Finding my copy of BASICball, re-reading my cute little grade-school comments (”Screen math is stupid”), remembering  anecdotes about what was going on at the time (”Our class gerbil is coming home with me for Thanksgiving break!!!!”), caught me up in an atypical wave of nostalgia. And in that wave, in looking back – really thinking about everything I have written over the years: millions upon millions of lines – that two things really stand out as being key to my success: Worse is Better, and Modularity is Ultimate.

I’ll write more about those things, I’m sure, but for now: Happy 20th Anniversary BASICball v6.0.

I just aged a bit, happily.

A little later, I really expected the Palm Pre to be the nirvana of compact, hyper-connected mobile devices. It wasn’t pretentious. It had a f%$#!%& keyboard. It was ahead of the curve in a number of features. But it wasn’t substantially different. Palm made the concious decision to keep the interface very similar to the old Palm OS, and let’s face it – user interfaces have evolved since then. But I got one. I like it. It’s a nice toy. If you want a compact, full-features smart phone, it’s still your best bet.

Motorola unveiled the Sholes over the summer. Beefy processor. Best-of-breed screen. Ridiculous connectivity solutions. A f%$#!%& keyboard. And CDMA (like the pre) so I don’t have to use the Ancient Telegram & Trash network. 5MP camera w/ LED flash. Ran Android (1.6 at the time). Super cool. At that point, they were still shopping for a vendor. I was cautiously optimistic it wouldn’t be a metro-only network like T-Mobile or Sprint.

Then came the onslaught of Droid Does during the baseball post-season. Speculation ran wild as to which phone it was, beneath the hype. BGR scooped it and pointed it out as a Sholes running Android 2.0. Thursday last I received mine.

Metal. Everywhere.

Ridiculously clear screen with outstanding pixel density.

Incredibly fast processor.

A f%$#!%& keyboard.

Seamless integration with all of the stuff I use (e-mail, calendar, contacts, etc. etc.).

Transparent movement between WIFI and the Verizon 3G network.

Incredibly fast processor.

Deep interface built atop a fully-accessible Linux system.

Scads of customizability.

Surprisingly good camera with shockingly bright flash.

Oh, and an incredibly fast processor.

If the pricetag is $100 too high for you, Verizon is also offering an HTC-based version called the Eris with no keyboard, more plastic, and a mid-level processor, with the same interface and general feature-set.

Oh yeah, and when you send e-mail, it’s not tagged “Sent from my <BlackBerry|iPhone|Other Pretentious Device>”.

But, since most people seem to enjoy those things: This post authored using a WordPress app from my Droid.

Endocrys has two primary modular components: Autocrys and Paracrys.

Autocrys is an extensible communication protocol atop XMPP. It governs the syntax of commands or queries sent to systems or groups, the responses of systems to those queries, how to manage their presence, and how to react to presence changes in others.

Paracrys is a database-driven deployment and configuration system. Paracrys allows module code and configuration data to be stored centrally and deployed to Endocrys nodes on-demand. Paracrys fully supports versioning, thus allowing changes to be rolled-back in the case of a major oopsie. How small can a Paracrys module be? Here’s an example that implements a command called ’shell’ that allows you to do, essentially, whatever you want on an Endocrys client:

BEGIN { $Endo::MODS{SHELL}++; $Endo::CMDS{SHELL} = \&shell; }
END { delete $Endo::MODS{SHELL}; delete $Endo::CMDS{SHELL}; }

sub shell {
 return `@_`;
}

Drop that puppy into the Paracrys MODULES table with some other data, issue a mass “fetch module SHELL; refresh;” command, and bingo, all of your systems now let you do very bad things. It’s that easy to create a command to do something… Hopefully something useful.

Of course you should note that there is no access control in the above code… How do we prevent Bad People from using our horrendously very bad shell command? That used to be managed by the Communication Masters using another database called EndoACL, but has been folded into Paracrys’ duties and drastically simplified. Each Endocrys client, when receiving the shell command, will now ask Paracrys if the user who sent it is authorized to issue that command. Previously, the clients never even received commands from users not authorized to send them, at great expense.

One of the major goals of the project originally was to have absolutely minimal dependencies on third-party code, so I reinvented the wheel in numerous places. Now that it’s mine again, those requirements are vapor and I’m ripping out large swaths of my code, and exchanging it for API calls into other code that is the de facto standard to do whatever. For example, I wrote a function that copies a file from one location to another. Ew. The File::Copy module is the Perl Way to do that, so that’s how we do it now. Less code I have to maintain, and less code you have to read to understand Endocrys.

Another major goal of the original project was absolute redundancy on all levels. With a requirement like that, I over-engineered what were called the Communication Masters (CMs) so that they heart-beated each other, transferred each other’s sessions, held elections to decide who was authoritative for which IP ranges, dealt with segmentation and partitioning, etc. All of this at the cost of highly-customized hybrid XMPP/SQL servers that weren’t readily upgradeable. Wednesday night I spent a lot of time diagramming, and tonight solidified the spec to separate the XMPP server from the SQL database, and rely on established high-availability tools like pen or an SLB appliance to ensure connectivity to a farm of XMPP servers if needed. Additionally, this separation has allowed me to use MySQL clusters for the Paracrys bits, which adds scary levels of redundancy to those very critical bits.

Lastly for this post, the entire ithread Endocrys implementation has been ripped out and replaced with EV and AnyEvent, and the Net::XMPP code has been replaced with AnyEvent::XMPP for one cohesive event loop that runs very very fast. Originally I envisioned an Endocrys client maintaining dozens of XMPP sessions while handling dozens of system events and receiving dozens of commands, so I stuck everything in threads, and allowed it to scream along on SMP boxes. While this works just fine, there is a LOT of extra complexity involved with sharing variables across threads, dealing with races, etc. and the benefits are dubious when compared against a good, strong, event-loop system. I’m not quite done yet, but the net loss should be about 30% of the main code modules, with reduced complexity for all sub-modules as well.

I don’t have an ETA as to when the code will be generally available, but I’ve had some pings from some bright people interested in hammering the retooled version in non-critical environments, so hopefully it will be this year.

This is one of my favorite products, and I’m more than a little excited to get it back and get it out. It’s been battle-tested for many years, and I’m very proud of it. I’m working on getting the code cleaned up and abstracted before releasing it under the GPLv2. Below are edited points from slides describing Endocrys and why you might be interested in it.

The Problems

Dozens… hundreds… of systems, physical and virtual, all going about their business. Then something happens: maybe a disk drive failed, maybe a process died, maybe someone ordered a ‘reboot’ or ‘halt’. Those systems don’t have a way of communicating that externally. There is no “Hey, I’m rebooting, BRB” in the server world.

Dozens… hundreds… of systems, physical and virtual, all going about their business. Then you have a question: How many of them have Western Digital harddrives listed in a recent recall? How many of them are running <2GB of RAM? How many of them are running a certain version of some software listed in a security advisory? There’s no way to ask that question to the farm. There is no “Dear Lazyweb, answer this question for me” in the server world.

The Purpose

At its most basic level, Endocrys is a conduit between all of the systems and you. Think of it like a gigantic Instant Messaging buddy list, where all of your buddies are systems. When they’re online, they are in the list and can set their status messages, send you messages, send each other messages, receive messages, etc. Endocrys leverages the eXtensible Messaging and Presence Protocol (XMPP) to tie this framework into existing clients, transports and APIs, enabling a near-infinite number of possible applications or functions you can deploy.

The Technology

Endocrys is built as a framework – an abstract set of rules that can be extended at any time by writing little modules. These modules can be applied across the Endrocrys network instantly, without any downtime.

By leveraging XMPP, the Endocrys network is highly-redundant with no single fail points. Any number of “Communication Masters” (XMPP servers) are online, but only one is needed to keep communication flowing. All network communication is encrypted and signed. Partitioning and segmentation is handled rationally.

Communication is very similar to Instant Messaging, there is relatively no latency, and XMPP assures delivery even to systems offline when the message was sent.

Monitoring and control systems can participate on Endocrys, automating the remediation of problems remotely and automatically.

The Protocol

XMPP sits atop TCP, and atop that sits the Endocrys Communication Protocol aka Autocrys. ECP is a fully-authenticated, fully-controlled, skeptical protocol that serves both for sending structured announcements as well as sending and processing commands. The entire ECP specification is listed in AUTOCRYS.TXT. Endocrys agents can be written in any programming language, attached to any other framework, at any OSI level, as long as they can speak XMPP and implement ECP appropriately.

The first diagram below (left) shows the near-final design diagram, and I’ve confirmed that everything fits. Everything except the rear exhaust fan block. There will have to be a wee bit of … “furniture modification” as the slot for cable ingress is too small. A liquid cooling system was added mainly to keep the sound level down. The pump is quieter than most fans, and the large CPU heatsink and exhaust fan pictured were able to be replaced with a small copper block and some tubes.

The second diagram below (right) outlines the machining specification for the replacement front door panel. Originally it was high-end non-thermal glass (to allow remote control IR to pass through), but had to be replaced with something I could cut. After a bunch of trial and error, I settled on 1/8″ acrylic. I’m still working on this piece- I’ve gone through a bunch of scrap trying to get the cuts and breaks right, and that is holding up the final tweaking (and pictures!), mostly.

The liquid cooling system (pump and exhaust fans) has a total noise of 11dB, all of which are in the rear of the cabinet. The 120MM intake fan is almost completely silent ~5dB, but inward-facing so it doesn’t expel much noise at all (<1db immediately outside the cabinet). The noise measure from The Comfiest Couch in the world with the entire system runnng is 5 dB – quiet enough to hear the harddrives prattling about, and substantially quieter (5-6 times) than the air-cooled equivalent.

I certainly empathize with your password management situation. Passwords are, actually, horrible security mechanisms and it is my opinion that they should be done away with altogether. Problem solved: No passwords means no password management headaches.

So, how to do prove you’re who you are? How do your systems trust who you say you are? A token. A “key”. A physical and logical item possessed by the user. Something they can lose or get stolen or drop in their coffee mug, but doesn’t matter because it’s useless without them leashed to it- and can be reproduced by authorized personnel in a jiffy.

The security industry likes calling it “two-factor authentication”: The two factors being something you have (the token) and something you know (the sentence uttered by your first girlfriend when she dumped you, song lyrics, the title of a book … whatever). Behind the scenes we shift from password management (gross and abhorrent) to key management (fun and exciting!)

Encrypted-key security is the only managed authentication scheme I have rolled out in client environments for the last 7…8 years. It can be “difficult” to wrench into an existing infrastructure, changing the culture, disrupting the status quo- but technologically is a vastly superior solution to identity management.

The defacto standard is PGP [1], although there are a lot of players in this market with varying quality of products, some aiming at various vertical markets. The link below gives a nice picture of how various systemic pieces tie together.

I know I didn’t answer your question- people tell me that a lot- but I can’t in good faith recommend password management. I haven’t been able to since 1999 or so, and certainly can’t as 2009 winds down. Sure, there are things you can do – the DoD uses the Mandylion [2], which you can buy on ThinkGeek [3] for $50 – but it doesn’t solve the actual problem of secure identity management: Please pardon the crudeness, but it’s like putting whipped-cream on dogshit.

[1] http://www.pgp.com/products/index.html
[2] http://www.mandylionlabs.com/
[3] http://www.thinkgeek.com/gadgets/security/91a2/

3 lbs boneless chicken breasts
1 lemon, cut in half
1/4 cup ghee, melted
4 large garlic cloves, chopped fine
thumb-sized piece of fresh ginger, peeled and chopped fine
2 tbsp ground paprika
1 tsp ground cinnamon
1 tbsp ground cumin
1 tsp ground coriander
1/2-1 tsp ground chili powder
1/2 tsp ground cloves
1/2 cup plain yogurt
4 drops red food coloring
2 drops yellow food coloring
2-3 small (or one large and seeded) chili peppers, chopped fine
14 oz can diced tomatoes
1 cup heavy cream
1/4 cup fresh cilantro (chopped)
salt
pepper

Preheat oven to 400F (will be using top rack). Place cleaned chicken in large glass bowl  and  stab repeatedly with pairing knife. Rub in 1/2 lemon and 1/2 tsp salt. Add in 1/2 of the chopped garlic, 1/2 of the cumin, 1 tbsp paprika,  and all of: ginger, yogurt, food coloring, cinnamon, coriander, chili powder, cloves. Mix with your hands until everything is evenly coated up to your elbows. If prepping ahead, this mix can go into the fridge for up to a week just fine.

Put chicken on cookie cooling rack over a solid, rimmed cookie sheet (dripping on the bottom of the oven are not fun). Some people claim a roasting pan works too. *shrug* With turkey baster, or a spoon if you’re boring, splurt 1/2 of the ghee over the top of  the chicken mess. Bake for 45 minutes. Broil for 10-20 minutes until top coating visibly blackening in spots (not burning!!). While broiling, make the sauce (next paragraph).

In large cast iron skillet, add remaining ghee over high heat until drops of water cause sizzling. Add remaining garlic and chili pepper, sauteing for 30 seconds or so. CAREFULLY add cream, diced tomatoes (with liquid) – ghee will be hot and addition of liquid may causing flashing!! Stir in remaining spices except cilantro (a few grinds of salt and pepper, too). Reduce heat to medium and simmer uncovered, stirring regularly,  for about 10  minutes or until sauce visibly thicker.

After chicken is done, remove and allow to cool for a couple minutes. Cut into small chunks and add to sauce.  Cover and reduce heat to low, cooking about 5 minutes.

Serve with rice and bread, sprinkling cilantro and a lemon wedge around plate edges for garnish and extra seasoning.

Serves 4. Prep time about 1 hour (serial). Cook time 70-85 minutes.

A while back, I got a new “TV”: An LG LH90. This TV, unlike my old TV (a Sony Wega that weighed 275lbs), could actually be placed atop furniture (as opposed to a steel-reinforced, purpose-built “Wega Stand”). Through the miracle of 4 HDMI ports on the TV, the cable box, Playstation 3 and my primary home PC can all be slaved  to a single gigantic screen: The TV also has a fiberoptic audio output which went smoothly into my old-yet-still-amazing surround-sound receiver, allowing me to use just the HDMI between devices for all video and audio. Consolidation nirvana!! But there was a problem:

My TV could be remote controlled from The Comfiest Couch In The World.

My PS3 could be remote controlled from The Comfiest Couch In The World.

My cable box could be remote controlled from The Comfiest Couch In The World.

My PC … couldn’t.

*twitch* *twitch* *sob* I had to get up and … *twitch* push a button.

So, I set upon the Intertubes with the goal of fixing that. Unfortunately, as forays into the morass of technology acquisition seem to go for me, I was sidetracked by a dangerous thing: potential.

Saga 2: Patiently Purveying Potential Possibilities… Primarily

While I should have been sufficiently pleased with acquiring a Thermaltake HTPC remote system, I kept listing restlessly at night, thinking of the other magical things I saw on the Interweb… Thinking of what I could do. After all, wouldn’t it be so cool if I built my PC into the furniture? That little cubby on the right was just begging to be modded- Begging to serve a higher purpose than banal storage.

But a mod of wood? Wood is a poor conductor of heat, and an amazing insulator: That would be a thermodynamic nightmare! And that point was precisely why I had to do it: I found a worthy challenge.

Specification

• Must not destroy, scratch or poke holes in the pricey (and pretty) furniture
• Must properly cool my 6-core system, and oversized graphics cards
• Must not be externally louder than current design (10-20dB)
• Must be completely controllable from The Comfiest Couch In The World
• Must be worthy

If you apply for jobs being vetted by morally superior -word removed- who pine for the “old days” of hunting witches in Salem , probably yes.

If not, then no. As someone who has done a lot of consulting for various gambling-related businesses, I can tell you that any software development company should prize former casino/gambling-related coders (that leave on good terms). It takes a lot of pedantic coding, edge-case handling, and vociferous documentation to make that industry run “right”, and those skills are hard to bake into someone.

I can’t speak to other countries, but in the US there are very few jobs that will – in and of themselves – cause you to get passed over for prospective employment. Except maybe working for the Revenue Service. We really don’t like them.

RooCluster is a command-and-control application designed for the special needs of  multiple robots operating in the same space, or over large multi-room spaces. Each Roomba is being fitted with an RFID tag, which, in coordiation with some more wireless access points, allows me to triangulate where a Roomba is and its travel vector (sometimes, math is cool). This information can help RooCluster avoid nasty Roomba-on-Roomba collisions, and also presents the possibility of meta-virtual walls.

If you have a Roomba, you probably have a virtual wall – the little pylon that sends out an infrared beam that the Roombas treat just like a wall. With some work, RooCluster should be able to honor coordinate-based lines (which could, in turn, form other shapes) and effectively “wall-off” areas without needing a physical barrier, or a battery-sucking virtual wall. You can also overlay the position and vector data onto floorplans, and see exactly where the Roombas are, and where they’re going.

Of course, you can also use it to make your Roombas dance with each other.

Or joust.