Software People Inspiring

Coding 'Til You Drop Gets The Thumbs-Down

 

I'm horrifically busy at the moment, which makes a change. But I just couldn't let this one go.

I tweeted this morning my career ambition. I would like to remain a programmer for the rest of my career. And I would like that to be seen as a good thing, and not a failure on my part to progress into management.

Programming is the most skilled job in IT. And programmers are the most skilled people. I would like to get better at it, and end my career on a programming high.

Because:

a. I enjoy it, and

b. Programming is kind of useful on software projects. The world needs working code.

Indeed, the world's appetite for working code grows ever more voracious. In the 30+ years since the first microcontroller was put in a car engine, we now have an average of 100 million lines of code in your luxury family saloon. I'm surrounded by devices that host millions of lines of code. My life, and probably yours, functions on working code every day. This is the information age, after all.

So if I enjoy programming, and the world needs programs that are useful, usable, reliable, maintainable, secure etc etc, then surely we have a marriage made in heaven.

And if I want to go on coding and not climb the career ladder into ever-more-lofty management positions, who could begrudge me that?

Well, someone could, it seems.

Bob Marshall (@flowchainsensei) retweeted it with a "-1" - basically, a thumbs down. He then tweeted:

"A focus on coding is self-indulgent and antithetical to the #lean ideas of value and flow, as well as to the #agile manifesto and principles"

Wow. I don't think Agile veterans like Uncle Bob Martin, Kent Beck, Ron Jeffries, Ward Cunningham, Steve Freeman, Nat Pryce, Ivan Moore, Joshua Kerievsky, Martin Fowler (and some bloke called Jason Gorman) got that memo. Somebody better tell them: if you want to be Agile, you need to stop coding beyond a certain age.

Now you'll have to pardon my French, but I couldn't give a flying fuck if I'm being "Lean", or frankly if I'm being "Agile", for that matter. I create working software. It is the software the customer asks for. Demonstrably so, as evidence by their acceptance tests - which are also expressed as code, because they need to be executable. So I have to deliver those, as well.

Now maybe in 2035 we'll all be creating software using pure thought channeled through a selection of genetically-modified winter vegetables, but that process will still be called "programming" and I want to be still doing it, and - if I pray to the coding pixies hard enough - maybe I'll be doing it better.

It's all very well saying "a focus on coding is self-indulgent", but no matter how unselfishly you manage your orchestra, you'll still need musicians who can "self-indulgently" make pleasing noises come out of their instruments using the enormous battery of skills they've built up over many years of dedicated practice. And nobody points at the pianist, or the lead violin or lead soprano and says "shouldn't you be a conductor by now?"

Nor do they accuse musicians of being "self-indulgent" in their ambition to become better musicians and not to persue a glittering management career at some record label or in the Arts Council.

If you love to code, and you're good at it, and if the world needs more and more working software, software of increasing sophistication and complexity, software that we come ever more to rely on in our daily lives, isn't it more selfish and self-indulgent to walk away from that calling?

Some people really need to go away and sit in a darkened room (or atop a distant mountain) and consider the realism of managing working software into existence. When the day comes when someone can demonstrate the creation of valuable software without programming (including "programming in pictures", before you MDA fans get too excited), then I'll hang up my IDE.

A Job For Programmers

 

Managing programmers is a job for programmers.


Hiring programmers is a job for programmers.


Educating programmers is a job for programmers.


Maybe you're a programmer, and you'd prefer to avoid these jobs. But the reality is that nobody else is qualified to do them.

Register For Software Craftsmanship 2010

 

Finally! You took your time, Jason!



Yes, Software Craftsmanship 2010 is open for business. You can register with your credit/debit card online for the jolly reasonable price of 85 quid. And, even better, all the profits from registration are going directly to help Bletchley Park.

We're also accepting submissions for sessions. The rules are simple:

1. Your session must involve a significant amount of live coding, and give participants the chance to do some coding, too
2. Your session must squeeze comfortably into a 30, 60 or 90 minutes slot
3. Your session must be about software craftsmanship (well, duh!)

Tutorials, katas, dojos, coding puzzles and games are all welcome. Maybe 30 minutes on seeing how many code smells we can cram into 100 lines of code. Or an hour playing Refactoring Golf. Or how about a 90-minute hands-on workshop on responsibility-driven design with TDD? Or pit your wits against your peers coding up battle bots in Ruby? If it gets everyone coding, and it can be done practically in the time and space provided, it's the sort of thing we're looking for.

To submit a session proposal, record a screencast of yourself demonstrating the practical elements (or practical illustrative examples, if the session is more exploratory) and post it on one of the many video or file hosting sites that abound these days (YouTube, Vimeo etc). Go to the SC2010 web site and fill out the submissions form, including links to your screencast videos.

You can submit and resubmit sessions as many times as you like up until the deadline (midnight on Sept 5th). After submissions have closed, everyone registered for SC2010 will be invited to vote for the sessions they'd most like to attend, based on your screencasts.

The most demanded sessions will find their way into the conference program, which will be announced on Sept 26th.

But don't worry if your session doesn't make it onto the schedule. SC2010 will leave plenty of space for ad hoc coding and discussion, both through open spaces and by making room for folk to sit down together and pair program outside of the sessions.

Whether you lead a session or not, you an be plenty busy with coding and sharing your experience and ideas throughout a packed day.

This what just some of the 100+ people who came to SC2009 had to say about the conference.



Couple that with the wonderful and historic surroundings of Bletchley Park, and we're sure SC2010 will be one of the best conference experiences you'll have this year.

Another Rant About Reinventing The Wheel And Against Yet More Navel-Gazing

 

Good morrow, gentle sir.

There's only so many times you can sit back and watch people "invent" stuff that's already been invented (several times over) before you start to realise how Neo must have felt when The Architect told him he was not the first to be called "The One".

Underlying any successful approach to software development is just a handful of core principles, all of which we've known about for decades. The recipe for working software - useful, reliable, maintainable working software - combines these staple ingredients with in a simmering reduction of talent (because any approach requires talent to work).

1. Tackle development in the smallest meaningful chunks that can be released to the customer. Any betting man will tell you that putting all your money on one horse is foolish in the extreme. Software development is very risky, so manage that risk by breaking one big gamble down into smaller, less risky punts. It also helps teams focus, by breaking requirements down into more manageable chunks.

2. Get meaningful feedback as often as possible, and adapt your software based on what that feedback teaches you. This is another pivotal element in successful - well - succesful anything. Iterate. We're not actually capable of solving complex problems any other way. It's been nature's solution right from the Big Bang. Life is a product of iterative design, as it happens. Oh, and get feedback by the most objective means you can. Just a hint: basing decisions to adapt your software on stuff people tell you is unwise. Because people mostly talk bollocks. Better to observe users actually trying to use the software you've delivered to do things you've agreed they should be able to do using the software. It help if you know why they're using your software. I know, it's a crazy idea! Which leads me neatly on to...

3. Have testable goals. Software development is expensive and hugely risky. Why gamble all that money if you don't have a clear picture of what it is you're spending that money for? What does your customer hope to achieve? How will they know if they've achieved it? At all levels from the business strategy that's driving the whole project or programme, down to understanding what a single method should do to contribute to those goals, it's incredibly useful to know precisely where the goalposts are. The best way to express goals testably is by writing tests. Just in case you were wondering. These could be project KPIs or business simulations and scenarios, or they could be behavioural specs written in RSpec, or unit tests written with JUnit. This is especially important when iterating. Feedback from tests can give you a very firm indication if you're getting warmer (or colder) with each pass.

4. Do the more important stuff sooner, especially if it's easier to do. Well, duh! Sounds obvious, right. Software development's a gamble. You've got a fixed amount to bet. If you leave your juiciest bets to last, you could run out of cash before you get the chance to place those juicy bets. It's not just a question of what a feature might be worth. Also bear in mind the cost and risk it might incur. Higher value, less costly/risky requirements should be further upstream in your requirements pipeline (or "backlog", to use the hip new street vernacular that I know many of you kids are into today).

5. If you find yourself doing something more than once, consider programming an evil robot to do it for you. Hey, you know what software's really good for? It's good for automating repetitive tasks. We've dedicated ourselves to taking the grind out of many people's jobs. Accountants can add up a balance sheet instantaneously and with total accuracy and consistency using spreadsheet software, for example. And think how long Avatar would have taken to render using coloured pencils. So when you're copying all those files across to the web server for the 27th time, consider how much time and trouble and risk might be saved by writing a program to do that for you. When you're writing yet another block of code to save and retrieve some object from a database, consider how much time you could save by automatically generating that kind of code. And so on. At the very least, train a monkey to do it.

6. Write your code so that it's actually possible to apply the first 5 principles, especially the one that adds most of the value - iterating your design. I'll make no bones about it. We have always known this. The biggest impediment to writing new code is the code you've already written. People in white coats carrying clipboards, probably working at the Laboratoire Garnier, have discovered that people like me and you spend most of our time staring at other people's code wondering what the buggery-hell it's doing. In this respect, the perennial killer germs are complexity, dependencies, duplication, poor regression test assurance and general lack of comprehensibility - often due to poor choices of names for variables, functions, classes and so on. While there's been something of a resurgence in disciplines that make code easier to understand an to change - like Domain-driven Design and Software Craftsmanship - we've actually known pretty much exactly what we should have been doing for several decades. So suck on that, Clean Code naysayers!

7. Use software developers who are good at developing software. No, seriously. It's all been for nought if, after you adopt your super-duper shiny new Agile Lean Kanban Unified Model-driven Development Method, you bus in a bunch of inexperienced, unpracticed, unmotivated, unprofessional code slobs. Just as sure as your super-duper shiny new Million Dollar Luxury Show Home will turn into a run-down hovel if you move in a bunch of first year med students. Indeed, I'd forgo 1-6 if I knew I had 7. Because a team of great software developers would probably do 1-6 anyway, instintictively and habitually. Of course, the fly in the ointment is that there's a very limited supply of such developers.

8. Get better at developing software. How many times have you done something for the first time and it's been just absolutely perfect? So perfect you couldn't possibly have done it better? Or done it quicker? Or done it cheaper? Or done it in a way that killed less horses? There's usually room for improvement in any activity or endevour, and this can be achieved by reflecting on what you've done and how you did it, aided by scientific principles to stop you jumping off the deep end into New Age Mumbo Jumbo Voodoo where you end up trying to improve design quality by homeopathic means or some such rubbish. Measurments can help, if they're good measurements and they're used in a grown-up way. Hard evidence generally is more useful than hearsay and anecdotal evidence. Check in histories, build reports, test reports, code metrics, screen captures, system logs, team height above sea level and average shoe size are just some of the useful data sources that can be analysed for clues as to how things could be done better. Smoke signals, aura readings and 100% discussion or opinion-based retrospectives tend to be the least useful. And practice needs to be part of the team's daily routine. Programming encompasses many theoretical discplines, but it's still by-and-large a practical skill, like speaking a foreign language. When you spot a deficiency highlihgted by your restrospectives, incorporating specially-designed exercises in your practice regime is a good way to turn a theoretical fix into a habitually-applied solution.


My point here is that there's no news here. All of these principles were known before most of us started our careers. Indeed, before many people had computers in their homes. What really would be novel is if we all started actually applying them. Instead of just dreaming up sexy new ways of talking about them. There's surely a lot more to discover about the art and science of software development. But when it comes to these core principles and "the old ways" that people like Knuth, Jacobson, Jackson, Hoare, Gilb and others were writing about while many of us were still in short trousers (or just glints in our parents' eyes), I think we're all talked-out.

Time for doing, not talking.

I Have A Dirty Secret. I'm A Software Craftsman

 

I have a dirty secret.

No, it's not the one about the Nuns and the winter vegetables. It's even worse than that.

I keep it off my CV and I never, ever mention it to potential employers when seeking contract work.

To my shame, and despite my best efforts not to, when I write code I routinely achieve very high test assurance. We're not talking 70% of the code or even 80%. No, we're looking at a truly obscene 95% or higher. And I'm not just talking about code coverage here. No, I graduated from that years ago. This is the hard stuff. I actually test my tests. I always run them to make sure they fail when they're supposed to, and I often use mutation testing to seek out any gaps in my tests, and once found, I just can't help plugging them. (I think someone might have just fainted. )

But it gets worse. My depravity doesn't stop at test assurance. I'm addicted to simplicity. Take two if... statements into the shower? Not likely. I like to Extract'N'Go. I get the cold sweats when I see methods longer than a few lines of code or that do more than one thing or have more than one branch. If the code's written by me, you'll find no methods longer than about ten lines of code, and vanishingly few with a cyclomatic complexity greater than two.

Oh, the humanity!

But it doesn't stop there. I'm obsessed with making my code readable, even to non-programmers. Try as I might, I just can't seem to get the hang of variable names like "t" and "x" and "m_strFltYrpOxfyRtn", method names like "returnNonNullString()" and "do()", and class names like "HelperManagerUtils" and "AstractXmlVisitorFactory2". I've just got this mental block. I can't help thinking my names should have something to do with the problem the user wants us to solve for them. I know. It's crazy! I've even tried using comments to hide my shame. But I can never think of anything useful to add that isn't already obvious from reading the code. EPIC FAIL.

This affliction of mine has now entered an advanced stage of perversity. If I could have just been satisfied with simple, readable and well-tested code maybe I could have controlled it. But, noooo. That wasn't enough. I had to go and get myself mixed up in the whole "coupling and cohesion" can of worms. So my code isn't just simple, readable and tested. Now all my classes and packages are loosely coupled and cohesive, too. And I'm not just talking about a bit of "tell, don't ask" or the occasional "dependency injection" here, either. This is the full-blown Object Oriented Design Principles (and then some!) shooting match: Single Responsibility, Open-Closed, Liskov Substitution, Interface Segregation, Dependency Inversion, Law of Demeter, Acyclic Dependencies, Stable Dependencies, Stable Abstractions, and a few I made up myself (because after a while I just wasn't getting the same high from the old stuff like I did at the start.)

Of course, I never mention any of this to prospective employers. They'd have me marched out of the door faster than you can say "perfectionist" (if you'll pardon my French). You only have to search on the big IT job sites to know that code of this quality simply will not be tolerated. And quite right, too. Software development's hard enough for the average team without some deranged OCD-type rampaging through the code like a bizarre kind of reverse-bull-in-a-china-shop, making it all nice and neat and maintainable. Especially, if, like me, they don't even have the decency to take a lot longer to deliver features while they're doing it.

The bottom line is that we're all on to a good thing here. I totally understand. No team is going to want someone like me producing high quality, reliable software. No team is going to want someone who can keep on producing valuable new features and responding to evolving business needs month after month and year after year.

It's that sort of nonsense that just gives customers ideas. And that would never do, now would it?

Join My UK Institute of Software Development. Or Don't. Whatever.

 

Greetings, puny earthling.

I have been appointed director of a brand new Institute of Software Development. Just now. By me. About 30 seconds after I wrote "Institute of Software Development" on a scrap of paper and thought "yes, that sounds suitably pompous".

It is, of course, my childish response to today's announcements by Gordon "Soon To Be Ex-Prime Minister" Brown made in a speech about so-called Digital Britain. It's all the usual vendor-driven, hype-oriented bullshit we're used to from politicians - especially when they're talking about anything to do with computers, because that, to them, is essentially magic beans and giant beanstalks territory. Mentions of "semantic web" (like the current web, only semantic, see) and "apps" and "digital content" and all the stuff people who pay someone to print off their email and read it out loud to them talk about.

He paints a picture of Digital Britain - empowered digital citizens connected by ultra-fast broadband to super-duper-whiz-bang-single-sign-on-will-even-make-the-tea-for-you online government services, and of empowered digital content creators (you may know them as "wankers with iPhones") leading the world in the creation of eye-popping visuals and aurals and very probably anals the likes of which none of us can frankly be bothered to imagine.

It all sounds great. Apart from the bit that seems to have been read directly from some vendor's latest brochure. Which it very probably was. But there's just one teensy-weensy little flaw in the grand plan. Just who in the Sam Hill is going to write all the software that will power Digital Britain?

Mr Brown claims that 250,000 new IT jobs will be created by his strategy, but that's a lot of skilled, talented professionals to conjure up in just a few years. Most importantly, where will all the new programmers come from? These people would have to be in schools and universities right now, as we speak, studying computer science, software engineering or other relevant disciplines. And they most definitely are not. Not in those numbers. Intake is down. And the quality of entrants into the profession is gradually eroding year on year.

The fact is, we're already scraping the bottom of the barrel. The dotcom bubble sucked in tens of thousands of wannabe programmers as share prices went stratospheric and anyone who could spell "A.S.P." was in with a shot at a career as a "web developer" (like a real software developer, only without the programming ability - which was seen as optional back then). After the bubble burst we found our talent pool chronically watered down with this low-quality dotcom intake - so much so that it could probably have been described as homeopathic. And any rational, educated scientist will tell you that homeopathy doesn't deliver.

It's bad enough today, when 9 out of every 10 developers you meet are frankly not up to the job and 9 out of 10 projects are consequently a nightmare for all involved. Another fairytale bubble could make that 99 out of 100 if the strategy doesn't make building the UK's software development capability it's number one priority. No good new programmers, no good new software. No good new software, no Digital Britain. It's really that simple.

And from the speech Mr Brown gave today and the Digital Britain report itself, it's apparant that not only isn't this a priority, it's not even worth a mention. I shouldn't be surprised. People who actually write software for a living are not represented in the priveleged circle of advisors the government has turned to to help shape the strategy. Instead, the report was shaped by academics, dotcom entrepreneurs, professional services companies, software vendors (not British ones, I should add) and more bloody celebrities.

Us codemonkeys are voiceless backroom boys (and gals, of course) in all of this. Just like we are in our own businesses. In our own IT departments. In our own teams. There's a clear pecking order from business leaders down through IT managers and project managers, past business analysts and architects and finally, right at the very bottom of the pile, the people who actually make software happen.

Which is entirely unnacceptable. If this was a strategy about Law in 2020, or the Future of Medicine, you could be damned sure that practicing lawyers and medical clinicians would be involved in the highest level discussions.

When I read Mr Brown's speech, I see hundreds, possibly thousands, of software projects being inadvertantly green-lighted. Potentially hundreds of millions of lines of code that will have to be written, tested, deployed, rewritten (several times) and adapted in the next ten years here in the UK. This truly dwarfs the economics of the online semantic social wotsisnames and digital content iThingy doodahs the government and their cohorts are salivating over, and they seem totally blind to it.

Blind, too, to the paradox they're defining here. As less and less people show an interest in learning how to tie their own shoelaces, let alone program a computer, we have a 10-year strategy that requires an unprecedented construction effort to create the critical software infrastructure that will power this digital revolution.

A programmer would not be blind to this, and many have tried to make their voices heard. But the government is not listening. Our existing institutions, like the BCS, are weak and ineffectual in such matters, and arguably are too widely-focused to speak specifically for people who write code.

We are not important. We are not famous academics (y'know, who've written books and been on telly and everything). We are not celebrated dotcom entrepreneurs. We are not Stephen Fry. (Love him though I definitely do). We have no voice in all of this, even though it's really going to be down to us at the end of the day to make it happen. As it always is when there's new software involved. We are the hard place to Mr Brown's digital rock. We must find a voice and make ourselves heard. We must do, because there is no "try".

So, in a petulant frenzy ("this is a petulant frenzy, I'm petulant and I'm having a frenzy" - bonus points to anyone who can identify the reference) I have literally just now formed my own Institute of Software Development so that when I write stinky missives to Number 10 about all of this I can put "Director, Institute of Software Development" on the letterhead and then maybe Gordon will think I'm as credible a source of guidance on these matters as Bono, Lorraine Kelly, Mr Kipling, Scooby Doo and any other celebs he's sought technology advice from recently.

Right now we only have one member. But you are very welcome to join. Membership is free to anyone who writes (or has written) code for a living. There is no newsletter and every year you won't get to vote on anything. You can put MISD after your name, if you think it will help. You know where to send the application. Usual address.

And an institute needs distinguished fellows, and perhaps just one or two celebs, of course. I have just taken a vote from the board of directors (me), and am delighted to announce the following fellowships:

John Daniels
Steve Freeman
Alan Cameron Wills
Steve Cook
C A O Hoare
Ivan Moore
Martin Fowler
Michael Winner
Bono

They don't get a say in it, of course. We have very strict rules about that sort of thing.

Message ends.

If You Want Things Done Right, Manage Yourselves

 

Hello, there. Are you one of those people? You know the ones I'm talking about. They sit around in meetings all day. They have Blackberrys. They take 6 weeks to respond to an email, despite spending most of their time checking their inbox, and especially if it's from someobody unimportant, like a programmer.

That's right. I'm talking about managers. We need managers, right? Our project would go off the rails without a firm hand to guide it. I mean, how will the programmers know what code to write? How will the customer know what features to ask for? How will the testers know what scenarios to scrutinize? How will the software get deployed? How will new programmers get hired?

Underlying the need for managers on software teams is a set of assumptions that there are things that won't happen unless someone is there to manage them, and that there are important things only a manager can do.

Talk to customers? You need a manager for that. Schedule work? You need a manager for that. Hire a programmer? Yes, that's management business alright.

It's true that projects need accounting and budgeting and stuff like that. I have an accountant. He's called Geoff. He does my books and tells me off when I lose a receipt. Guess what, though? He's not the boss of me. He doesn't go out and speak to clients on my behalf. He doesn't win business for me. He doesn't decide what days I work. He doesn't have any say in when I take my holidays, or which graphic designer I should use for the Boffoonery programme (I did it myself in the end, by the way, because I'm cheap). I do all of this myself. He just balances the books at the end of the year and tells me how much tax Her Majesty's bloodsuckers will have to prise from my cold, dead hands.

I'm a programmer. No, seriously. And I can pick up a phone and speak to a recruitment consultant about sending us some CVs for a programmer role on my team. I can run a stand-up meeting and I can run a planning session with both hands tied behind my back. I can order stationary. I can put up whiteboards. I can tie my own shoelaces and wipe my own arse. Because I'm a grown-up. Well, maybe.

Having had one foot firmly in the UML and use cases camp for well over a decade I can certainly handle requirements, and can talk directly with the customer - who is also a human - and help them articulate what they need from the software without the need for a translator.

I understand business. I run one. I get invites to join the Institute of Directors and everything. And you say "oh, but that's rare among programmers". No. No it isn't. I've met a few programmers in my time, and despite a handful of very interesting characters who wouldn't function in any social situtaion - including working with other programmers - I'm confidently calling bullshit on that myth. Programmers understand business every bit as well as their managers. Possibly even better, since their managers haven't had to describe how their business works in bloody C++ or whatever. If you really want to test your understanding of something, try explaining it to a computer.

Projects need administators, just like my business does. I'm not a qualified accountant, and I need someone who really knows their stuff to make sure my books are tickety-boo and Mr Tax Bastard is fed the correct number of pounds of flesh. But I run my business. And hundreds of thousands of freelance programmers are in the same boat. They have to understand business, because they are one. And we spend our lives immersed in the customer's business to a level of detail the customer themselves probably doesn't go into.

I believe in self-managing teams of professional programmers. They can do the hiring and firing. They can work with the customer on requirements, testing and planning/tracking. They can order the stationary they need. And, very much an upside for them, they can decide things like how much refactoring is really needed, how much time can be invested in learning and practice each week, what documentation is really needed, and all manner of things that really are best left to the people who are actually doing the work.

And when they're too busy doing code and stuff (bearing in mind how much of their time would be freed up not having to persue pointless management agendas), they can send their Project Gimp to do it for them. Leadership can and often does come from within the team. A leader can be a programmer. And we can all be leaders at some point, when we're best placed to make certain decisions or provide a certain kind of direction.

And for those programmers among you who are thinking "but I don't want to do all this stuff"; maybe you don't, but if that's the case then stop complaining when your managers waste your time doing dumb shit (as they are wont to do). If you want a job doing properly...

Code Metrics Show Convincing Picture Of Impact Of Peer-based Craftsmanship Coaching

 

I'm sure they won't mind my sharing this with you. I've been putting together a presentation for managers in TV Platforms at the BBC, where I've been running a Peer Group Learning & Assessment program to help engineers hone their TDD and refactoring chops.

It's been bubbling away in the background for about a year with 8 of their engineers, and is now rolling out to other peer groups, so by the end of this summer pretty much everybody writing code in TV Platforms will have been through the basic TDD practice regimen and hopefully successfully passed a gruelling practice-based peer assessment to demonstrate that they're truly in the habit of doing things like running the test to see it fail and not refactoring on a red light etc.

Anyhoo, for the presentation, Kerry Jones - TV Platform's key technical design authority and a thoroughly good craftsman in his own right - threw together some code metrics to give us a picture of the "before" and "after".

It's looking good, I have to say. Remember this is data culled from projects in which only 8 of the engineers have been through a TDD PGLA exercise - that's about a quarter of all the TVP engineers.

Test coverage is up by about 40%. Method length and complexity has dropped by roughly 10-15%. Duplicate code has dropped by more than 50%.

Now, their stats were by no means bad to start with (the BBC have a much higher-than-average quality of developer, I'm finding), and it's usually much more dramatic when teams are starting from rock bottom (low-hanging fruit and all that), so these improvements are all the more striking. This is by far and away the most encouraging result I've had from a coaching program, and I'm quietly confident that as we delve into the murky world of refactoring and legacy code, and incorporate data on dependencies on OO goodness/badness into the picture, we'll see yet more improvements.

Whether project managers will give two hoots remains to be seen, but when we make the link between factors like test assurance, complexity and duplication and productivity and cost of change (already well-established through SEI, IBM and other studies), it's a picture that's hard to dismiss.

If you want to know what PGLA is all about, I'll be giving the lowdown at QCon on March 10th, and Kerry and I are booked in to talk about PGLA and software craftsmanship at the BBC at XP2010 this summer.

When Does "Delivered" Not Mean Delivered?

 

On occasion, I take a calculated risk and buy something off eBay. Ninety nine times out of a hundred, I suppose, the sellers are on the level and ship my item as soon as my payment's cleared. Right now, I'm awaiting delivery of an item from the US, which I purchased about two weeks ago.

The seller shipped it by standard international parcel post and I've been able to track its progress via the USPS web site. Which is how I know it left the US on Feb 11th and arrived in the UK on Feb 17th. So far, so good.

But this is where things all go a bit Twilight Zone-ish. According to USPS tracking (which must now be gettings its data from Royal Mail tracking at our end), they "attempted" to deliver it at 11.20pm last night.

Uhuh? Are we doing parcel deliveries after the pubs shut now? I believe not. "Attempted" delivery I understand to mean that they stuck it in a delivery van and the van came to my house and the driver knocked on my door and there was no answer. Or they couldn't find the house. Or something like that. Basically, they tried to deliver it, but were unable to for some reason.

I was most definitely at home at 11.20pm last night, and wide awake, and sober, and would most definitely have noticed if someone had knocked on my door at that unsociable hour.

This is not the first time that an overdue parcel delivery has been tracked as "delivered" or "attempted delivery" via our esteemed postal service - the oldest (and now, it seems, the creakiest) in the world. I orderd a box of expensive cigars a few months back and it disappeared into their system for several days, while all the time showing as "delivered" on their tracking web site.

It eventually turned up, of course. And I pointedly asked the nice lady who delivered it why it was down on their system as "delivered" when it quite obviously hadn't been up until then. And she admitted, with little hint of embarassment, that this happens all the time and is done to keep their performance statistics looking good.

This is a growing malaise in British life. Targets and measures are routinely gamed by put-upon workers who frankly are given no reason to give a shit. Actually delivering the parcel is less important than being seen to have delivered it on paper. The management perception of performance and reliability weighs most heavily on the worker's shoulders - far more so than the customer's perception of the quality of service they're getting. So much so, in fact, that when I've dared to complain to sloppy service providers, they've called me a liar because their database systems clearly show a rosier picture than the one I, a mere customer, have painted.

This is all very well and good, having a Saturday morning moan and all. But there's a salient point at the end of all this. I've seen countless teams - Agile and otherwise - game their delivery data in much the same way. When you give the people doing the delivery the power to decide if something has been "delivered" and when their management relies on this highly subjective picture to assess the reliability of releases (by which I mean, when we say it's been delivered, has it really been delivered), we get a similarly disparity between the customer's perception and our own.

Of course, the postal delivery system is supposed to have checks and balances to make it more objective. How could something be "delivered" without a signature? Well, obviously it can, somehow. Perhaps the signature isn't really required. Or maybe any signature will do.

The solution is to place full responsibility for deciding if something has been successfully delivered with the end customer. And to establish very clear criteria as to what constitutes successful delivery. As it turns out, requiring a signature is by no means fullproof. What would probably work better is if customers were given a secret number or password to enter into the courier's little handheld computer thingy - or a barcode to scan - or something that can only be obtained by making contact with the person for whom the letter or parcel is intended.

In software projects, we need something that is a clear and unequivocal indicator that "the ball is in the hole". Not near the hole. In the hole. And this must be something that only the person who raised that specific requirement can control. Developers must not be allowed to make changes to the criteria. Only the customer should have that power. And these criteria must be beyond any doubt. They cannot be vague or handwavy, because that makes them game-able.

Practitioners of acceptance test-driven development will have some idea of what I'm getting at here. But I've watched teams doctor the test scripts without informing their customers, just to make a failing test pass and to meet a deadline, resolving to fix the real problems after they have "successfully delivered".

This cannot be allowed. ATDD works best when only customers have this power. Test scripts should be locked down in the customer's version control system, and nobody who touches the code should have the authority to check in new versions of the acceptance tests. If we want to move the goalposts, we must be forced to go to the customer and ask their permission.

Now, where's my bloody parcel?!

Wheel-driven Reinvention

 

One aspect of software development which is at once both amusing and troubling is the ability of us young whippersnappers to completely ignore what's gone before and reinvent established wisdom in our own image - often stealing the credit.

Take testing as an example. What do we know about testing software today that we didn't know, say, thirty years ago? Sure, we have new tools and testing has to fit within new approaches to the process of writing software as a whole, but fundamentally what have we discovered in the last decade or so?

Testing behaviour still works, by necessity, much as it has always worked by necessity. We must put the system under test in some desired initial state, then we must provide some stimulus to the system to trigger the behaviour we wish to test, then we must make observations about the final state of the system or about any behaviours that should have been invoked (e.g., a remote procedure call or a database request) in response to the combination of our stimulus and the initial conditions. And this process must be repeatable and predictable, like any good scientific test.

Though the culture of testing software may have evolved, much of it for the better, and the technology may have improved (though that is questionable), and though there are undoubtedly more people testing their systems today, when it comes to the business of writing and executing tests, there's really nothing new under the sun.

The same is true of many aspects of contemporary software development. Like it or nay, iterative and incremental development is older than C. We just weren't doing it back then, in the main.

Indeed, pick any "new" aspect of development and trace it back to its roots, and we discover that most novelties are actually much older than many of us thought. Objects are an invention from the sixties. Use cases hail from the seventies. Responsibility-driven design was being practiced before Frankie told us to Relax. UML existed in a fragmentary form before the Berlin Wall came down. People were writing code to satisfy tests back when those tests were stored on magnetic tape. Indeed, some of the descriptions of programming that was done for the very first computers rings bells with those of us who practice that black art today.

Younger developers like me, though, seem to readily believe that our time around is the first time around and feel no compunction to educate ourselves about the achievements of "old-timers", preferring instead to invent things anew - with sexier names and shinier tools, admittedly.

Our desire to reinvent goes as far as redefining words that already have a well-established definition. "Agile" no longer means "nimble" , "quick" or "spry". Today it apparantly means "communication, feedback, simplicity and courage". Or "iterative and incremental". Or "evolutionary". Or "Scrum-Certified". I caught someone the other day proferring their definition of "testable", which apparantly now requires us to go through "public interfaces". This is bad news for many scientists, who must now rewrite their peer-reviewed papers to incorporate the appropriate programming language with which to express the "testability" of their theories.

If software development was physics, we might expect newcomers to work through and understand the current body of knowledge before they start adding to it. That way, at the very least, we could avoid a great deal of duplication of effort. We may also avoid the tendency of our industry to throw "old-timers" on the scrapheap just because, even though they are probably just as current in their practical ability to deliver working software, they're not "down with the kids" on all the latest street slang for concepts that have been kicking around the block for decades.

The thinking of our elders and betters is far from irrelevent and outmoded. We can still learn a thing or two from the likes of Jacobson, Knuth and Hoare, should we choose to reject fashion in favour of substance in the approach we take to our work.