Accidents happen. No seasoned analyst, admin or engineer I know hasn’t typed ‘reboot’ in the wrong term window, deleted the wrong file, created the odd network loop, etc. These are generally accidents: what was being done was judgmentally sound, however something wasn’t quite right.

When you omit the sound judgment, however, you didn’t have an accident anymore, you have generated work. In the world of geeks, accidents are things everyone laughs about after the fact, generating work will usually result in no one laughing with you ever again. Time is precious, respect doubly so, and making a bad decision that costs others’ cycles consumes both.

For the last few weeks, we’ve had some electrical contractors in our office building replacing our 1970s-vintage fire alarm system with something modern. Observing their group dynamics was fascinating, and reinforced the same principles. Some guy who set off the alarm system twice in one morning because of bad decisions wasn’t invited back after a couple days. Another guy who checkpointed his thoughts with the more seasoned crew before blundering was given more leeway. A third who didn’t seem to know how to do his job, and was constantly requesting help and making bad decisions requiring others to fix his shit, was asked at one point “are you really an electrician?” and generally ostracized by the senior crew.

I am a firm believer in the old adage “there are no stupid questions”. Without hesitation I offer my time to all sorts of people who are interested in learning, check-pointing, advancing or honing their knowledge in an array of topics. I enjoy pedagogy and dialogue: Most seasoned, polydisciplinaries do, especially those who are also autodidacts. Form intelligent questions, ask intelligent questions, save your reputation, expand your knowledge, develop sounder judgment, don’t generate work. Ask.

While I generally don’t post pictures taken by other people, this was taken by my Dad at Stone Valley yesterday, which one of my favorite local places to lose myself. You may need to look at the picture in full size to understand why I’m posting it. I have seen the subject many times, but never gotten a snap. Kudos to Dad.

My home PC (which is also my only Windows computer) has been having sporadic issues for a month or so. Moments of inexplicable slowness, programs crashing, etc. It has been tolerable, and while a nuisance, hadn’t prevented me from getting stuff done (or games played), until yesterday morning. On boot it threw a fit, as it sometime does, crashing a slew of startup up programs including the shell (Windows Explorer), but then it wouldn’t launch anything. I rebooted. Same thing. Reboot into “safe mode”, same thing. Cannot lunch applications.

My first thought was virus/malware. I don’t actively run anti-virus software on that system as I’m not a click-whore, and at home am behind an impregnable firewall. So I rebooted onto a rescue CD from F-Secure that runs linux and scans drives for the icky things most mortals fear, as well as looks at drive health (SMART), and picked up a book. Two-hours later, clean, but it noted some filesystem damage.

Great. I rebooted onto the Windows 7 install cd, which has an excellent recover/repair toolset including my favorite option: Command Prompt.

<tangent> I rue the increasing loss of control that consumers have over their computing devices. Increasingly corporate machines are forcing us to compute how they want, not giving us an operating system but rather a jail under the guise of keeping us safe from the “bad things” out there. Apple has taken this so far with the iPhone and now iPad (giggling) that you are UNABLE to install anything they haven’t approved and/or isn’t in their “store”. This isn’t to keep uSheep “safe”, it’s to control content and continue to profit by force, by 100% preventing competition. It’s a parasitic arrangement that uSheep and they have, and it is counter to every principle that helped build personal computing into what it is today. These disgusting tactics tug hard at the very fabrics of curiosity and innovation, stifling both while claiming to be their champion. Apple: We innovate, so you can’t. </tangent>

So, I’ve got a command prompt now, and run a chkdsk /f to find and fix logical errors, pick up a book and wait. A couple hours later, some indexes fixed, some orphans reconnected with their parents, but nothing obviously amazing. Reboot. Same problem. I reboot back onto the install CD and again wake my minion Command Prompt, and tell it to do something vile: chkdsk /r. *gasps, the sounds of doom everywhere*

The chkdsk tool is a lot like some fantasy-novel magic item: it can be used for great good, or great evil. It has been this way since DOS was your operating system. The /r option may very well hose your drive. R is not for “repair” or some similarly encouraging verb, but for “relocate”. It looks at the disk logically like /f, but after all that it looks at it physically, and relocates data on bad clusters to good clusters, ostensibly.

This may sound encouraging to you, dear reader, but let me help frame this with allegory: You have a plate covered with poop, and on that plate is some fruit. You relocate that fruit to another clean plate. Is all of the fruit still edible? Hell, is any of the fruit edible?! Tis the same with bad clusters: not all the bits in the bad cluster can be moved, thus corrupting files irrecoverably.

So I unleashed the Eater of Files on my drive, picked up my book, and waited. After 3 hours it started finding some serious problems, and commenced eating bits (or moving fecally-enhanced fruit, if you prefer). Three-hours after that it finished. I rebooted, asked the God of Bits for favor, and was not disappointed. Everything was very fast, and the only noticeable blowback was that software I use to control my computer via remote control (IMON) and Acrobat Reader were horribly damaged, and will need to be reinstalled.

But I got a lot of unexpected reading accomplished, so 11 hours was not wasted at all, just relocated.

Purloined wholesale from this /. story about this article, is an amazing piece of prose I wanted to draw attention too, as well as the relevant articles:

“Hot on the heels of the story in Publisher’s Weekly that ‘publishers could be losing out on as much $3 billion to online book piracy’ comes a sudden realization of a much larger threat to the viability of the book industry. Apparently, over 2 billion books were ‘loaned’ last year by a cabal of organizations found in nearly every American city and town. Using the same advanced projective mathematics used in the study cited by Publishers Weekly, Go To Hellman has computed that publishers could be losing sales opportunities totaling over $100 billion per year, losses which extend back to at least the year 2000. … From what we’ve been able to piece together, the book ‘lending’ takes place in ‘libraries.’ On entering one of these dens, patrons may view a dazzling array of books, periodicals, even CDs and DVDs, all available to anyone willing to disclose valuable personal information in exchange for a ‘card.’ But there is an ominous silence pervading these ersatz sanctuaries, enforced by the stern demeanor of staff and the glares of other patrons. Although there’s no admission charge and it doesn’t cost anything to borrow a book, there’s always the threat of an onerous overdue bill for the hapless borrower who forgets to continue the cycle of not paying for copyrighted material.”

Identity theft certainly exists. It’s when someone sets up their existence as someone else. Drivers license, bank accounts, passport, etc. etc. This is exceedingly rare these days, as it has become harder and harder over time to pull this sort of thing off. The motivation is generally less about making illicit money as hiding in the shadow of someone else.

Today, and for the last several years, we’ve heard more and more about “identity theft” because the words alone invoke fear. Companies make billions (yes, billions) of dollars off of your fear of “identity theft” for no reason other than that the words are scary. Someone has stolen my identity! That’s a scary thing. Thankfully it won’t happen. What does happen, and what the marketing engine tries to deflect, is fraud. Yup. There’s a not-scary word. That’s the word that the financial industry has used since the 1300’s to describe deceit and trickery for financial gain. Fraud. Someone has lied to someone else, and claimed to be you. That someone else didn’t do a good job making sure you are really you, and thus a fraud occurred. They didn’t steal your identity, they lied.

Besides making tons of money by propagating fear, it also deflects where the blame actually belongs. There are three parties involved: you, the fraudster, and your financial institution(s). You are the victim, we agree; he fraudster a liar; your financial institution … hmmm… why aren’t they doing a better job at protecting your assets/credit/reputation? Why are they believing the lie? Why are they complicit in the fraud? Because it’s cheaper to allow fraud than to defend against it. That’s the bottom line. The financial services industry is scaring you with the idea “identity theft” so that they can make money selling you garbage “identity theft protection” instead of doing their job and protecting your assets/credit/reputation from fraud. Period.

In systems programming (and application programming, and life in general) there are two ways to deal with potential problems: You can try to avoid them, or you can handle them intelligently. When we walk across the street, we look both ways: This is a simple avoidance. It’s easier to look both ways and not have to intelligently handle the oncoming car. We will, hopefully, however, intelligently handle the oncoming car in the event our avoidance strategy fails. We will jump out of the way, or throw a brick at the windshield, whatever. It continues to boggle my mind the lengths people will go to to avoid the most random corner-cases, but fail to intelligently handle even the most obvious of exceptions, even ones that continuously rear their heads.

Avoidance scenarios in code are often completely unnecessary and several orders more complex than handling the inevitable exception to begin with. The simplest solution is almost always the best, and always the best when you can’t rely on your avoidance strategy to begin with! If your program calls some code, and expects to get back a 0 or 1, what happens when it gets a 2? What happens when some field in a database you flagged as never null (an avoidance) is null for some records?

Avoidance is about assumptions. A lot of programmers love assumptions, they code for them often, sometimes even writing comments in their code “the blahblah field is set to ‘never null’ so we can assume that to be true” – Really? Until an admin turns off constraints and bulk loads some “bad” data. Instead of writing the cheeky comment making the assumption, you could have written a one-line handler to Do The Right Thing. If you toss what you think you know out the window, program with the facts in mind, and handle failure cases elegantly, you end up with a durable system.

Your system application sends a file from one system to another every day. Usually it works, sometimes it doesn’t. When it doesn’t, what happens? Nothing, because you never coded for that eventuality. Why would you? It’s not your fault if the other side doesn’t work, after all. Your code is perfect. The intelligent handling to this is two lines of code: after you send the file, check to make sure the file is actually on the other end, and retry if it’s not (or send you an e-mail, or write something to a log… SOMETHING). I can list over 60 different reasons that file may not be on the other side- you can add mitigators for those 60 reasons (plus the other 60 I didn’t bother thinking of), or you can intelligently handle the problem with two lines of code, actually, zero lines if you’re elegant.

SendFile(fileName,fromHere,toThere)

could be replaced with:

until(FileExists(“toThere/fileName”)) { SendFile(fileName,fromHere,toThere); }

That’s pseudo-code (‘until( .. )’ is the same as ‘while(not .. )’, if you’re using a primitive language) but it will work semantically the same in at least 16 different programming languages. This doesn’t mean you shouldn’t try to figure out why it’s failing and perhaps fix something that’s broken along the way, but your code doesn’t need that complexity, it just needs to Do The Right Thing.

Another classic example is how one handles multiple instances. Some software runs fine with multiple copies (like your web browser) some behave badly (like your e-mail client), some even worse. Frequently, even when they know that running two or more instances at the same time is very very bad  (will-destroy-data bad), they don’t handle that event, they avoid it. They say “well the code runs via a scheduler, and there’s enough time in-between runs that it should be done”. Should. You may destroy data and cause more work, for a “should”. The handler is a 2-line fix:

open ME, “<$0″ or exit;
flock ME, LOCK_EX | LOCK_NB or exit;

Lock yourself, and exit if you can’t get a lock. If locking yourself seems too conceptually scary, then pick a lock file (that’s what the /var/lock subsystem is for, by the way) and lock on that. The code is Perl, but the concept will work in at least a dozen different languages, and is bullet-proof (assuming your host OS supports flock).

There are a lot of reasons people don’t write durable programs- I don’t pretend to do it all of the time, either. Laziness and ignorance are probably tied for first, followed closely by apathy.  If your system is “critical” (to you, or someone else) and if/when it doesn’t function it generates work, there is almost always a 1-to-2 line solution to help it survive, or at the very least, not do the wrong thing.

In January of 2005 I wrote a small system for a client called ‘Batch NRPE’. The point of it was to have a drop-in replacement for the Nagios NRPE system that, instead of interrogating 50 services over the course of an hour, lets get all of the results at once, stuff them into a database (MySQL or flat-file), and then return the cached results from the database instead of chronically torturing the systems.

The contracted 5-year exclusivity on it is up, and it should be released GPLv2 next week. There isn’t much I want to change about it, just replace my argument processing with Getopt and add more documentation, really.

I only used the Nexus One for 20 minutes earlier this evening, here are my thoughts:

The screen is slightly larger than the Droid. Similarly clear and bright. It seems slower, side-by-side, than the Droid, although probably not measurably so.

Touch is “off” a bit. I can accurately hit tiny spaces on my Droid, but my meat digits seem to be unable to precisely hit the same target on the Nexus – maybe the increased screen resolution  has caused some precision narrowing? If so, Google can fix that with a software update if enough people complain.

The 3G speeds were nearly identical for looking at the same content (both Verizon). I didn’t test WiFi. Launching apps did take a “step” longer on the Nexus. It was very close, so who knows, but we observed it several times when switching from browser-to-mail-to-youtube-to-browser, etc.

The speaker isn’t as clear when playing music. I had the same MP3 playing on both and it was obvious the Droid had better range. (NOTE: I don’t care about this at all, just an observation)

The quality of calls in the ear-speaker is identical.

Android 2.1 is very nice, although the video backgrounds can take a leap- I’ll keep my battery life thankyouverymuch. I await the impending upgrade to the Droids… If I await too long I’ll probably field-upgrade mine, some of the real-estate changes are very drool-worthy.

“Memory” (flash) is surprisingly scarce at 4GB compared to the 16+GB on the Droid. Maybe Google figures since they didn’t include a physical keyboard you won’t ever use it?

So, yeah, no physical keyboard. Show-stopper. A mobile computer without a good physical keyboard is a complete waste of my time.

The point: If you don’t mind that little caveat (no keyboard), you can’t go wrong with it. It’s a near-visual-hardware-clone of the iPhone (albeit about 2billion times faster) with a much nicer interface (that you can get on a number of other devices) and gorgeous screen. Certainly nothing revolutionary over some of its recent kin, but still a very good device if you’re looking at getting a second-generation Android mobile.

Last week a couple colleagues in different environs offered to work with my latest-stable code in test environments, providing me volumes of great feedback. Over the last few days I’ve been enthusiastically implementing large swaths of those thoughts, taking 1/2 days from my dayjob to get things going. I wanted to publicly share the extra documentation I gave them, as it provides good insight into the changes in Endocrys, as well as where it is going:

Read more »

I remember when a “terrorist” was someone who drove a truck-bomb into a housing complex and killed a few-hundred soldiers. Today, anyone who says “fuck you” to a law enforcement officer, any child who scribbles hateful notes in the margins of their notebook, probably even me for writing this piece of disestablishmentarianism – is a “terrorist”.

I remember when a “friend” was someone who could hit you in the shoulder when you were being snarky, and whom you knew would have your back when others wouldn’t. Today it’s slung like manure on a corn field (I’ve been in Northern NY too long, methinks) to describe anyone someone may have come into some physical or digital contact with. I can count four people on this planet that I would deign to call a “friend”, some people seemingly have hundreds.

I chose these two words out of the bajillion I could have because they are on very different sides the adjective-spectrum, yet have still been turned into something rapidly approaching worthless.

We are diluting the value of words – lots of words that have good, solid meaning – simply because we want to use “extremes” instead of “moderates”. We don’t want to say “I’m sorry, you’re not a ‘friend’, but merely an ‘acquaintance’”; We don’t want to say “her classmates are obviously asshats” so we say “she’s a terrorist because she doesn’t like her classmates”. We’re cheapening poignant words with mediocrity, averaging meaning towards uselessness.

But I suppose you probably have 175 “friends” and don’t really think about “terrorists” because there isn’t a category for that on your social network.