Contractor ghettos, Dunning Kruger, and oh sweet jesus the code!

I hear that's just what those coffee script guys look like. Give him a banana, will you?

Imagine a small development team. A few senior geeks, some cash, and some ideas. How do you take idea X, and develop it in timeframe Y, where your team alone can’t deliver?

Or, “Fred is the backbone of this team. If only I could clone Fred.”

1.

When I was still a raw n00b, back in the early 90s, the consensus from professionals and academics was that the days of coding as a cottage industry were behind us. Software was now a serious business, a serious engineering discipline, with methods for big teams to work together in hierarchies, according to set development phases, using serious expensive commercial tools and serious expensive commercial methodologies and wearing serious expensive business wear.

rofl, of course.

So here we are in 2012, and the software development landscape is bizarre/bazaar/hazaa! There are still borganisations in towers of glass and concrete and steel, taskmasters whipping their clone armies in the open plan pits. There are toytown lands of geek bliss, where the decor is by Fisher-Price and Nintendo, there is füssball in every room, and no one ever, ever sleeps. There are teams in far off lands that present themselves to the highest bidder in almost API-like fashion. There are cranky madmen in their code dungeons smashing out apps for a dollar a go on AppStore X, muttering and chanting incantations unheard since the Shareware dreaming. And of course there are the legions of the pure, their black screens aglow, manifesting the Nothing in its immanence and white fury as it consumes and evaporates all.

And combinations of these things, and other things I am entirely unaware of, and combinations of those, too.

There is a big picture, of course. That picture is the Great Evolutionary Bitgrinder. Great because the scale is immense, too large to fit in your head. Evolutionary, because that’s what it is, a concurrent search of the codescape, naturally selecting its way across the unfathomable gödelspaces in the manner of a biblical plague. Bitgrinder because, as with any selection algorithm, it relies fundamentally on failure and death at scale. You traverse a crenelated trap strewn mindfield by filling the holes and valleys and pits with corpses, a floodfill of the damned.

If you are building software in any capacity, you shouldn’t think about the macro level too often or for too long. It’s the creative person’s equivalent of staring directly into the sun. Or the abyss. There is always your mental health to consider, but more immediately and pragmatically, it changes nothing.

At the micro scale, each of our day to day scales, we still have to make things. We want to make things. Our circumstances and goals and approaches are different, but the creation requirement is invariant.

2.

Back in the early 2000s I came across rentacoder, and was intrigued by the idea of a market for coders. Someone specifies work, coders bid, someone gets it and does the work, gets paid. Loose ties for software development, at scale. Cool!

So I dived into the coding pool. It took a little while to get the stink off.

Because, of course, it was a ghetto. Here and there was a job I might have been able to do. But, the pay was really low, and the jobs were kind of awful or mad or both. If I wanted to write spambots for $11 an hour I probably could have been happy there. Or wanted to make Blue Orb (which, now that I think about it, you might even be able to do now, by chucking together a custom linux distro. This is a crazy, crazy world we live in.)

These sorts of sites are mature now. People seem to get actual work done through sites like Odesk, which implement the online version of traditional contracting, more or less. But I still get the sense that they’re a bit ghetto. No one ever raves to me “hey, I started using Odesk, it’s awesome”, they sound a lot more battleweary than that.

It’s odd to me that this is the case, because the idea itself, putting together people who need work done with people willing to do the work, seems sound on the face of it. Why is it still a poorly solved problem?

3.

Some years back, a good friend of mine sent me a link to a scientific paper, with the wonderful title “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments”, for which, wikipedia tells me, the authors won an IgNobel prize in 2000.

Here’s the abstract:

People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of the participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

This has now come to be known by the authorial eponym, “The Dunning-Kruger Effect“.

4.

So why are these contracting sites ghettos?

Consider how we manage work.

Say you have a big technical job to do. It breaks into two parts. One part you know how to do well. One you have no idea about. Your timeframe is short enough to mean you need help.

So of course you get a contractor in to do the part you don’t understand.

Later on, with your project in shambles, you grumble to your collegues about contractors or Elbonians or Odesk.

And Dunning and Kruger shake their heads. How can you judge what you can’t understand?

5.

Employing new people in tech projects is a minefield. When things are busy, when you’re under the pump, then it’s easy to hire out of desperation. Do you have time to supervise the new body? Will you be forward looking with choosing skillsets? Or will you shoot for the guy with a pulse, a good long resume in the thing you don’t understand, and enough plausibly geekish arrogance that it’s possible you might sit him in the corner and he’ll crank out the goods?

And he will sit there unsupervised. Why? Because you don’t have time. Luckily you don’t understand what he’s doing, so you wouldn’t know how to judge it, so you don’t feel guilty about it. You’re too busy to feel guilty!

And it’s ok for a while, isn’t it? Until the contract’s over, yet it doesn’t quite work right, and then you look at the code. Oh sweet jesus, the code…

6.

Where does it all go wrong?

We treat hires or contractors or odesk like a get out of jail free card for the prison of our own ignorance. But we should listen to Dunning and Kruger.

Not only do you not understand what you don’t understand, but you can’t recognise expertise in it, and can’t judge quality.

But when you hire in skills from outside, you need to do exactly these things.

How do you escape this trap?

By outsourcing only the things you are already good at.

By outsourcing ONLY the things you are ALREADY GOOD AT!

So when you had those two jobs that needed doing, one you understood and one you didn’t, you should have hired someone to do the one you understood, and learned the one you didn’t.

When that project comes up to build an ipad app to talk to the vast web app you built and maintain, think about hiring a web guy to do the extensions to your site, and buy yourself a macbook and a dev license.

7.

In some ways this is not a new idea. We’ve had teams led by tech leads for years, whose job is often to lead others in doing the well understood stuff, and get their heads around the unknown bits. Hang in there, though; this is only part 1 of a several part (2? 3?) series. Expect some more unorthodox thinking, culminating in a big idea. If you’re a professional senior developer, it might just be the recipe for the next decade or so of your career…

Advertisements
Contractor ghettos, Dunning Kruger, and oh sweet jesus the code!

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s