Software People Inspiring

Coding 'Til You Drop Gets The Thumbs-Down

 

I'm horrifically busy at the moment, which makes a change. But I just couldn't let this one go.

I tweeted this morning my career ambition. I would like to remain a programmer for the rest of my career. And I would like that to be seen as a good thing, and not a failure on my part to progress into management.

Programming is the most skilled job in IT. And programmers are the most skilled people. I would like to get better at it, and end my career on a programming high.

Because:

a. I enjoy it, and

b. Programming is kind of useful on software projects. The world needs working code.

Indeed, the world's appetite for working code grows ever more voracious. In the 30+ years since the first microcontroller was put in a car engine, we now have an average of 100 million lines of code in your luxury family saloon. I'm surrounded by devices that host millions of lines of code. My life, and probably yours, functions on working code every day. This is the information age, after all.

So if I enjoy programming, and the world needs programs that are useful, usable, reliable, maintainable, secure etc etc, then surely we have a marriage made in heaven.

And if I want to go on coding and not climb the career ladder into ever-more-lofty management positions, who could begrudge me that?

Well, someone could, it seems.

Bob Marshall (@flowchainsensei) retweeted it with a "-1" - basically, a thumbs down. He then tweeted:

"A focus on coding is self-indulgent and antithetical to the #lean ideas of value and flow, as well as to the #agile manifesto and principles"

Wow. I don't think Agile veterans like Uncle Bob Martin, Kent Beck, Ron Jeffries, Ward Cunningham, Steve Freeman, Nat Pryce, Ivan Moore, Joshua Kerievsky, Martin Fowler (and some bloke called Jason Gorman) got that memo. Somebody better tell them: if you want to be Agile, you need to stop coding beyond a certain age.

Now you'll have to pardon my French, but I couldn't give a flying fuck if I'm being "Lean", or frankly if I'm being "Agile", for that matter. I create working software. It is the software the customer asks for. Demonstrably so, as evidence by their acceptance tests - which are also expressed as code, because they need to be executable. So I have to deliver those, as well.

Now maybe in 2035 we'll all be creating software using pure thought channeled through a selection of genetically-modified winter vegetables, but that process will still be called "programming" and I want to be still doing it, and - if I pray to the coding pixies hard enough - maybe I'll be doing it better.

It's all very well saying "a focus on coding is self-indulgent", but no matter how unselfishly you manage your orchestra, you'll still need musicians who can "self-indulgently" make pleasing noises come out of their instruments using the enormous battery of skills they've built up over many years of dedicated practice. And nobody points at the pianist, or the lead violin or lead soprano and says "shouldn't you be a conductor by now?"

Nor do they accuse musicians of being "self-indulgent" in their ambition to become better musicians and not to persue a glittering management career at some record label or in the Arts Council.

If you love to code, and you're good at it, and if the world needs more and more working software, software of increasing sophistication and complexity, software that we come ever more to rely on in our daily lives, isn't it more selfish and self-indulgent to walk away from that calling?

Some people really need to go away and sit in a darkened room (or atop a distant mountain) and consider the realism of managing working software into existence. When the day comes when someone can demonstrate the creation of valuable software without programming (including "programming in pictures", before you MDA fans get too excited), then I'll hang up my IDE.

The Customer Holds The Key To The City of Agile

 

We all gripe and moan about how hard it is to get our customers to "go Agile" and play their role effectively. And it's an important role. Indeed, the customer is the key to Agile.


We have a hard sell on our hands changing the customer's mindset to embrace Agile values. But when it's the other way around - which, of course, it very rarely is - the customer doesn't have to do much to pretty much force teams to be Agile.


For example, if I was funding a software development venture (let's not call them "projects" any more) I would demand to see working software - growing working software - pretty frequently. A development team in the UK will cost you tens of thousands of pounds a week, so at the end of one week I'd want to know what that money bought me.


To deliver every week, cycles need to be short. If regression testing your application takes weeks or months, then a weekly show-and-tell will be nigh on impossible. A team would pretty much be driven into the comforting embrace of automated builds, tests and deployments. Either that or fail to meet my expectations, and risk getting canned.


And if a team or a supplier demands a complete detailed requirements specification up-front, or tries to freeze the requirements, again - as it's my money and my choice - they'd have to either adapt to work with high-level product goals for the long-term and detailed requirements for the work that's in focus for the next delivery, or be shown the door.


They would also have to accept that week-on-week, I can change the requirements based on what we've learned up to that point. I am the customer. It's my money. I reserve the right to change my mind. I'm willing to pay for those changes, so what's the problem?


What about quality? What if each drop is riddled with bugs and the devs end up spending most of their time fixing them? I'm not a programmer. How can I effect that?


Easy. I demand precise, executable acceptance tests. I can do that. I can insist. I can bring in my own test expert to help make sure it happens. She would work for me to ensure we all know exactly what should be delivered and how we'll know it works.


It's my money. So even if the devs normal modus operandi is code-and-fix development, I will not sign off on any delivery that doesn't pass my aceptance tests. Work is funded (by whatever model) week-by-week, and it's my money. Failure to deliver something that passes my acceptance tests will stop the flow of funding. Two or three painful, bug-filled deliveries in a row, and I'll withdraw my funding completely and the venture will end (or switch to a better team).


And if the team can't deliver everything we agreed in that week, well, I'll take a view on that. Is what they delivered worth what I paid for it? If, week after week, I feel I'm not getting value for money, then - yep, you guessed it - it's my money and I can decide to stop wasting it.


To meet my expectations as a customer - frequent deliveries of working software which I deem worth what I spent on it, verified by executable tests agreed with me, and adaptibility after each delivery cycle - I'm not aware of any way than to be Agile in the accepted sense of the word.


If the choice is between that and getting canned (and/or replaced with a better team), many teams will step up to the plate and find a way to do it. It can be done, this we know.


As the customer, I can set the values of the venture by setting the right expectations and doing my job in all of that. I can choose to make myself available (or someone I trust, at least) to make sure everybody gets the decisions and domain expertise they need when they need them.


The keys to the city of Agile are in my hands.


If I forget that, please remind me. It's my money, and I insist that you do. Even if I act like I don't want you to.

Object Oriented Design Master Class, London Aug 21-22

 

This summer's run of budget-friendly weekend master classes in software craftsmanship ends with a workshop on object oriented design on August 21-22.


In this very hands-on course, you'll learn about OO design principles (S.O.L.I.D. and more), using real code examples to illustrate how to refactor designs that violate these principles. You'll also get practical experience of using code analysis tools to measure OO design quality and detect problems more easily, even in large code bases. The workshop will also give you a chance to apply simple OO analysis and design techniques in a test-driven approach to development, as we as looking at how up-front design can be balanced with refactoring to produce optimal designs without falling into the shark-infested waters of "Big Design Up-Front" and "no conscious design at all".


As with all the workshops I'm running this summer, there'll be the minimum of talk from me and the maximum of hands-on practical experience and learning by doing, working in pairs with other attendees.


And - as if the price wasn't good enough already - everyone who attends both the TDD and refactoring courses will get to go on this one absolutely FREE. I must be mad! Book your place now before I change my mind!

Codemanship Academy Alumni - I Salute You

 

For the last couple of years, I've been dedicating myself to an experimental coaching and learning process that is peer-led and practice-based called Peer Group Learning & Assessment.

During that time, several groups of developers have been going through PGLA programs in Test-driven Development, and more recently, Refactoring. I've been overseeing their PGLA programs and validating results in my capacity as an all-round brilliant person (and love machine). The results have been encouraging. Teams that did the TDD PGLA program saw real results in their code. Code was simpler and cleaner, and - unsurprisingly - better tested and more reliable. We have hard data to back this up from a dozen or so projects that paint a clear picture of the before and after. We continue to monitor that code to see the effect of scaling the practice up to 3 groups (with a fourth about to start), and tackling refactorings in more detail.

There are enough people engaged in PGLA now to warrant publicly acknowledging their achievements. So I've set up a web page on the Codemanship site where you can find out who's done it.

Some of you are probably thinking "hmmm, this looks suspiciously like certification to me". And, yes, it does a bit. But this is not a list of people who attended a course one weekend. I have spent months (and months and more months) working with these groups and their organisations. I've paired with all of them, and they've spent many hours over those many months pairing with each other and their internal coaches. It's no easy feat. Ask any of them whether PGLA was as easy as, say, getting to be a Certified Scrum Master. Many struggled to make the time. It was a serious commitment that they gave, and they put in some serious time to honouring it.

They also had one final hurdle to jump. And it's quite a high hurdle. To complete their PGLA program, they had to spend a day doing what they'd been practicing to do. And doing it to a standard that most of us would struggle to maintain for even an hour, let alone over the course of a long and gruelling day (the last hour is the worst, trust me!)

If you want to know how tough that might be, try this little exercise. Write down a list of a dozen good habits you believe you should apply when you're doing, say, TDD. It could be things like "write the assertion first and work backwards" or "never refactor if a test is failing" or "always run the test to check it fails in the way you expect it to", and so on.

Now, pick a simple coding exercise. For example, write a program that will generate a sequence of prime numbers of a specified length. Or that will calculate scores for a bowling game. Then try to implement the code needed to solve the problem doing TDD "by the book" (i.e., sticking to all the good habits on your list). Record a video of your desktop while you do it, then watch it back very carefully. Every time you catch yourself failing to apply one of those habits (e.g., you didn't run a test to see it fail), that's one strike. If you can spend an hour practicing TDD and get less than three strikes, and do that three hours in a row, you'll be doing very well. Especially if you can stick with the habits even at the end of a long and tiring day.

I'll bet you a shiny penny that it's harder than you might think it would be. Y'see, most of us know what we should be doing, but few of us actually do those things - not habitually, anyway. PGLA is about building those habits and instincts. It takes a long time and it's no small achievement.

The lovely people listed on the Codemanship Academy page have proved their metal in a way that very few developers have. They've proved that, even when they're tired and stressed and the pubs are open and it's time to go home, they can still do TDD to a standard higher than most of us apply when we're full of vim and vigour and raring to go.

And how many of us can honestly say we do that, too?

Another Rant About Reinventing The Wheel And Against Yet More Navel-Gazing

 

Good morrow, gentle sir.

There's only so many times you can sit back and watch people "invent" stuff that's already been invented (several times over) before you start to realise how Neo must have felt when The Architect told him he was not the first to be called "The One".

Underlying any successful approach to software development is just a handful of core principles, all of which we've known about for decades. The recipe for working software - useful, reliable, maintainable working software - combines these staple ingredients with in a simmering reduction of talent (because any approach requires talent to work).

1. Tackle development in the smallest meaningful chunks that can be released to the customer. Any betting man will tell you that putting all your money on one horse is foolish in the extreme. Software development is very risky, so manage that risk by breaking one big gamble down into smaller, less risky punts. It also helps teams focus, by breaking requirements down into more manageable chunks.

2. Get meaningful feedback as often as possible, and adapt your software based on what that feedback teaches you. This is another pivotal element in successful - well - succesful anything. Iterate. We're not actually capable of solving complex problems any other way. It's been nature's solution right from the Big Bang. Life is a product of iterative design, as it happens. Oh, and get feedback by the most objective means you can. Just a hint: basing decisions to adapt your software on stuff people tell you is unwise. Because people mostly talk bollocks. Better to observe users actually trying to use the software you've delivered to do things you've agreed they should be able to do using the software. It help if you know why they're using your software. I know, it's a crazy idea! Which leads me neatly on to...

3. Have testable goals. Software development is expensive and hugely risky. Why gamble all that money if you don't have a clear picture of what it is you're spending that money for? What does your customer hope to achieve? How will they know if they've achieved it? At all levels from the business strategy that's driving the whole project or programme, down to understanding what a single method should do to contribute to those goals, it's incredibly useful to know precisely where the goalposts are. The best way to express goals testably is by writing tests. Just in case you were wondering. These could be project KPIs or business simulations and scenarios, or they could be behavioural specs written in RSpec, or unit tests written with JUnit. This is especially important when iterating. Feedback from tests can give you a very firm indication if you're getting warmer (or colder) with each pass.

4. Do the more important stuff sooner, especially if it's easier to do. Well, duh! Sounds obvious, right. Software development's a gamble. You've got a fixed amount to bet. If you leave your juiciest bets to last, you could run out of cash before you get the chance to place those juicy bets. It's not just a question of what a feature might be worth. Also bear in mind the cost and risk it might incur. Higher value, less costly/risky requirements should be further upstream in your requirements pipeline (or "backlog", to use the hip new street vernacular that I know many of you kids are into today).

5. If you find yourself doing something more than once, consider programming an evil robot to do it for you. Hey, you know what software's really good for? It's good for automating repetitive tasks. We've dedicated ourselves to taking the grind out of many people's jobs. Accountants can add up a balance sheet instantaneously and with total accuracy and consistency using spreadsheet software, for example. And think how long Avatar would have taken to render using coloured pencils. So when you're copying all those files across to the web server for the 27th time, consider how much time and trouble and risk might be saved by writing a program to do that for you. When you're writing yet another block of code to save and retrieve some object from a database, consider how much time you could save by automatically generating that kind of code. And so on. At the very least, train a monkey to do it.

6. Write your code so that it's actually possible to apply the first 5 principles, especially the one that adds most of the value - iterating your design. I'll make no bones about it. We have always known this. The biggest impediment to writing new code is the code you've already written. People in white coats carrying clipboards, probably working at the Laboratoire Garnier, have discovered that people like me and you spend most of our time staring at other people's code wondering what the buggery-hell it's doing. In this respect, the perennial killer germs are complexity, dependencies, duplication, poor regression test assurance and general lack of comprehensibility - often due to poor choices of names for variables, functions, classes and so on. While there's been something of a resurgence in disciplines that make code easier to understand an to change - like Domain-driven Design and Software Craftsmanship - we've actually known pretty much exactly what we should have been doing for several decades. So suck on that, Clean Code naysayers!

7. Use software developers who are good at developing software. No, seriously. It's all been for nought if, after you adopt your super-duper shiny new Agile Lean Kanban Unified Model-driven Development Method, you bus in a bunch of inexperienced, unpracticed, unmotivated, unprofessional code slobs. Just as sure as your super-duper shiny new Million Dollar Luxury Show Home will turn into a run-down hovel if you move in a bunch of first year med students. Indeed, I'd forgo 1-6 if I knew I had 7. Because a team of great software developers would probably do 1-6 anyway, instintictively and habitually. Of course, the fly in the ointment is that there's a very limited supply of such developers.

8. Get better at developing software. How many times have you done something for the first time and it's been just absolutely perfect? So perfect you couldn't possibly have done it better? Or done it quicker? Or done it cheaper? Or done it in a way that killed less horses? There's usually room for improvement in any activity or endevour, and this can be achieved by reflecting on what you've done and how you did it, aided by scientific principles to stop you jumping off the deep end into New Age Mumbo Jumbo Voodoo where you end up trying to improve design quality by homeopathic means or some such rubbish. Measurments can help, if they're good measurements and they're used in a grown-up way. Hard evidence generally is more useful than hearsay and anecdotal evidence. Check in histories, build reports, test reports, code metrics, screen captures, system logs, team height above sea level and average shoe size are just some of the useful data sources that can be analysed for clues as to how things could be done better. Smoke signals, aura readings and 100% discussion or opinion-based retrospectives tend to be the least useful. And practice needs to be part of the team's daily routine. Programming encompasses many theoretical discplines, but it's still by-and-large a practical skill, like speaking a foreign language. When you spot a deficiency highlihgted by your restrospectives, incorporating specially-designed exercises in your practice regime is a good way to turn a theoretical fix into a habitually-applied solution.


My point here is that there's no news here. All of these principles were known before most of us started our careers. Indeed, before many people had computers in their homes. What really would be novel is if we all started actually applying them. Instead of just dreaming up sexy new ways of talking about them. There's surely a lot more to discover about the art and science of software development. But when it comes to these core principles and "the old ways" that people like Knuth, Jacobson, Jackson, Hoare, Gilb and others were writing about while many of us were still in short trousers (or just glints in our parents' eyes), I think we're all talked-out.

Time for doing, not talking.

Software Is Both Art & Science. Can We Move On Now?

 

Bonjour, mes enfants.

SEMAT has gotten me thinking. Any mention of "engineering" and "science" in software development seems to polarise opinion.

Undoubtedly, there's a large section of the software development community who believe those words simply do not apply. Software development is not a science. It's an art. Or a craft. Like basket weaving. It's not engineering. No sir!

And there's an equally large section of the community who believe the exact opposite. Software development is a science. It is engineering. We can apply scientific principles to shape predictable, well-engineered end products.

Of course, they're both wrong.

Anyone who dismisses the notion of any kind of scientific basis for software development is running away from reality. Everything that exists has a scientific basis. Even American Idol. You just have to understand it. That we don't fully understand the science of software does not mean that no such science exists or that we'll never understand. I just can't help being reminded of UFOlogists who claim that "science cannot be applied to the study of UFOs". What they mean, of course, is "I've got a good thing going here selling my unscientific ideas to these schmucks, and I'd like to keep it that way, thanks".

And anyone who believes that software development can be completely tamed by science is equally deluded. There are sciences - emerging in recent decades - that teach us that there are many things in the world that, while it's possible that we can fully understand them, we will never be able to control them. Chaos is science. And software development is mostly chaos. Any vision of "software engineering", where pulling lever A causes X and pulling lever B causes Y with certainty, is the product of naivity at the macro, project scale. Life just isn't like that. Clockwork might be, but life isn't. Software development is intractably complex and unpredictable.

In the real world, biology has come up with processes that deal effectively - but not predictably - with intractably complex problems. Evolution is one such process. Evolution solves complex, multi-dimensional problems by iterating through generations of potential solutions. That is science. We can understand it. But we cannot even begin to predict what solution a process of evolution will reach, or how long it will take to reach it.

The clockwork, Newtonian paradigm of "software process enginering" is fundamentally flawed. And anyone who believes that it's possible to attain "value" deterministically is deeply mistaken. "If we pull lever A, we'll ship an extra 10,000 units". Give me a break!

Flawed, too, is any notion that this means that there's no engineering at all to be done in creating good software. I doubt anyone would claim that making rock music is "engineering" - well, anyone sane, at least. But there is science that can be applied within this process.

It's possible mathematically to predict what effect the choice of software compressor and the compression settings used will have on the amplitude of a recording across a certain frequency range. Indeed, it is helpful in getting the best-sounding mix. There is such a thing as "audio engineering" within the music production process. Granted, it's chiefly a creative process. But there is useful science we can appky within in to help tweak the results closer to perfection.

Similarly, while software design is chiefly a creative process, it can be useful to know if the code we're writing "smells" in any significant way. Some code smells can be detected just by looking at the code and using our judgement. Others are more subtle and harder to spot, but just as damaging to maintainability. Code analysis tools, as they grow more sophisticated, can complement our "eye for good design" every bit as much as audio engineering tools complement our "ear for a good mix" and music composition aids can complement our "ear for a good tune".

The trick, I believe, is to find the balance and work within the limitations of science and engineering and creative disciplines. Trying to figure out the formula for "valuable software" is every bit as futile as looking for a formula for making "hit records". And relying entirely on your eyes and ears to refine an end product has severe limitations. For millenia, we've used tools and theory to tweak and refine all manner of creative end products. We even have a word for it: "machine-tooled". The fact that you can fit a computer more powerful than all of the computers in the world were 30 years ago in your breast pocket, and it looks good, is testament to this symbiosis of science and art.

We can continue to refine and extend our scientific understanding of code and coding through research and exploration, just like any science. And new and useful tools will emerge from that understanding that will make it possible for us to produce better quality code, for sure.

And we can also continue to develop and refine the art of software development. Through reflection, practice and sharing.

There is such a thing as "software engineering", but it has limited scope, just like "audio engineering". Specifically, it's limited to things we can predict and can control, like what happens to coupling and cohesion if we move a class from one package to another.

The bigger picture of "delivering value" is a complex human endevour, and creativity, judgement and more than a sprinkling of luck is all we have that we can bring to bear in any meaningful way at this level. We may be capable of understanding, with the benefit of hindsight, why Feature X was used more than Feature Y when the software was released. But then, with hindsight, we understand quite a lot about volcanos and hurricanes, too. These are things that can only be understood with hindsight, really. We don't see them coming until they're almost upon us, and we have two choices - stay and risk everything or get out of the way and live to fight another day.

In years to come, I'll probably notice more and more a difference between "hand-rolled" software and software that has been written with some help from "software engineering" tools - the more grown-up descendants of tools like XDepend and Jester.

But I sincerely doubt I will ever be able to tell at the start of a project whether the resulting software will enjoy success or not. Sure, I'll be able to look back on projects and say "hey, y'know what we got wrong?" But far in advance, the outcome is every bit as unknowable as a hurricane, volcanic eruption or hit record. So where things like requirements and processes and "enterprise architeture" are concerned, I'll stick with arts and crafts.

Training 4 Bletchley - Peer Group Learning & Assessment Workshop, Bletchley Park, Sept 16th

 

Bono estente, et scorchio!

Firstly, a very heartfelt thanks to everyone who has generously pledged their pay for September 16th to help Bletchley Park. We're well on our way to achieving my initial target of 50 pledges by the end of this week. If you haven't pledged, please take a moment and consider helping. And remember, for every pound you donate, it could unlock a further 4 quid in lottery funds. And then there's Gift Aid...

On that subject, I've also pledged my earnings for September 16th. And, like many coaches and trainers, I have good days and better days (financially speaking). I'm keen to make September 16th one of the better days, which usually means running a training workshop.

And where better to run a training workshop in aid of Bletchley Park than, well, Bletchley Park? The room is booked and I already know what the workshop will be focusing on.

In the last 18 months or so, as some of you will know, I've been focusing on a new approach to coaching and helping teams of developers to learn, internalise that knowledge and build good habits through long-term practice and refelection with your peers.

The results the pilot groups at various clients have been achieving look very promising indeed, and the approach has been validated with code reviews and metrics that show clear improvements in key areas of maintainability like code complexity, duplication and regression test assurance, as well as the reductions in bug counts we might expect.

This approach of Peer Group Learning & Assessment has worked especially well at the BBC, and I and my counterpart there, Kerry Jones, are "doing the rounds" of a few conferences and writing up our experinces for the odd journal, which can give you a flavour of what it's all about.

Of course, there's more to it than can be conveyed in a 45-minute talk or in a 6-page article. To run your own Peer Group Learning & Assessment program, you really need a practical, hands-on introduction to the key elements of the program, as well as some useful tips I've picked up during these last 18 months.

So I'm proposing to run a one day workshop on Peer Group Learning & Assessment at Bletchley Park on September 16th, where people can get a proper handle on what it is, how it works, how you might run such a program in your organisation, and a heads-up on some the pitfalls I've encountered in a small - but diverse - range of clients.

This is going to be real training - I'm going to prepare and everything - and I'll be charging a real training fee for registration, but instead of writing your cheque to my company, you'll be paying your money directly to the Bletchley Park Trust and helping to preserve and transform a hugely important part of computing history. It'll be a packed day, and you will really get your money's worth.

The tea and sandwiches will be on me!

Usual address for details.

If You Want Things Done Right, Manage Yourselves

 

Hello, there. Are you one of those people? You know the ones I'm talking about. They sit around in meetings all day. They have Blackberrys. They take 6 weeks to respond to an email, despite spending most of their time checking their inbox, and especially if it's from someobody unimportant, like a programmer.

That's right. I'm talking about managers. We need managers, right? Our project would go off the rails without a firm hand to guide it. I mean, how will the programmers know what code to write? How will the customer know what features to ask for? How will the testers know what scenarios to scrutinize? How will the software get deployed? How will new programmers get hired?

Underlying the need for managers on software teams is a set of assumptions that there are things that won't happen unless someone is there to manage them, and that there are important things only a manager can do.

Talk to customers? You need a manager for that. Schedule work? You need a manager for that. Hire a programmer? Yes, that's management business alright.

It's true that projects need accounting and budgeting and stuff like that. I have an accountant. He's called Geoff. He does my books and tells me off when I lose a receipt. Guess what, though? He's not the boss of me. He doesn't go out and speak to clients on my behalf. He doesn't win business for me. He doesn't decide what days I work. He doesn't have any say in when I take my holidays, or which graphic designer I should use for the Boffoonery programme (I did it myself in the end, by the way, because I'm cheap). I do all of this myself. He just balances the books at the end of the year and tells me how much tax Her Majesty's bloodsuckers will have to prise from my cold, dead hands.

I'm a programmer. No, seriously. And I can pick up a phone and speak to a recruitment consultant about sending us some CVs for a programmer role on my team. I can run a stand-up meeting and I can run a planning session with both hands tied behind my back. I can order stationary. I can put up whiteboards. I can tie my own shoelaces and wipe my own arse. Because I'm a grown-up. Well, maybe.

Having had one foot firmly in the UML and use cases camp for well over a decade I can certainly handle requirements, and can talk directly with the customer - who is also a human - and help them articulate what they need from the software without the need for a translator.

I understand business. I run one. I get invites to join the Institute of Directors and everything. And you say "oh, but that's rare among programmers". No. No it isn't. I've met a few programmers in my time, and despite a handful of very interesting characters who wouldn't function in any social situtaion - including working with other programmers - I'm confidently calling bullshit on that myth. Programmers understand business every bit as well as their managers. Possibly even better, since their managers haven't had to describe how their business works in bloody C++ or whatever. If you really want to test your understanding of something, try explaining it to a computer.

Projects need administators, just like my business does. I'm not a qualified accountant, and I need someone who really knows their stuff to make sure my books are tickety-boo and Mr Tax Bastard is fed the correct number of pounds of flesh. But I run my business. And hundreds of thousands of freelance programmers are in the same boat. They have to understand business, because they are one. And we spend our lives immersed in the customer's business to a level of detail the customer themselves probably doesn't go into.

I believe in self-managing teams of professional programmers. They can do the hiring and firing. They can work with the customer on requirements, testing and planning/tracking. They can order the stationary they need. And, very much an upside for them, they can decide things like how much refactoring is really needed, how much time can be invested in learning and practice each week, what documentation is really needed, and all manner of things that really are best left to the people who are actually doing the work.

And when they're too busy doing code and stuff (bearing in mind how much of their time would be freed up not having to persue pointless management agendas), they can send their Project Gimp to do it for them. Leadership can and often does come from within the team. A leader can be a programmer. And we can all be leaders at some point, when we're best placed to make certain decisions or provide a certain kind of direction.

And for those programmers among you who are thinking "but I don't want to do all this stuff"; maybe you don't, but if that's the case then stop complaining when your managers waste your time doing dumb shit (as they are wont to do). If you want a job doing properly...

When Does "Delivered" Not Mean Delivered?

 

On occasion, I take a calculated risk and buy something off eBay. Ninety nine times out of a hundred, I suppose, the sellers are on the level and ship my item as soon as my payment's cleared. Right now, I'm awaiting delivery of an item from the US, which I purchased about two weeks ago.

The seller shipped it by standard international parcel post and I've been able to track its progress via the USPS web site. Which is how I know it left the US on Feb 11th and arrived in the UK on Feb 17th. So far, so good.

But this is where things all go a bit Twilight Zone-ish. According to USPS tracking (which must now be gettings its data from Royal Mail tracking at our end), they "attempted" to deliver it at 11.20pm last night.

Uhuh? Are we doing parcel deliveries after the pubs shut now? I believe not. "Attempted" delivery I understand to mean that they stuck it in a delivery van and the van came to my house and the driver knocked on my door and there was no answer. Or they couldn't find the house. Or something like that. Basically, they tried to deliver it, but were unable to for some reason.

I was most definitely at home at 11.20pm last night, and wide awake, and sober, and would most definitely have noticed if someone had knocked on my door at that unsociable hour.

This is not the first time that an overdue parcel delivery has been tracked as "delivered" or "attempted delivery" via our esteemed postal service - the oldest (and now, it seems, the creakiest) in the world. I orderd a box of expensive cigars a few months back and it disappeared into their system for several days, while all the time showing as "delivered" on their tracking web site.

It eventually turned up, of course. And I pointedly asked the nice lady who delivered it why it was down on their system as "delivered" when it quite obviously hadn't been up until then. And she admitted, with little hint of embarassment, that this happens all the time and is done to keep their performance statistics looking good.

This is a growing malaise in British life. Targets and measures are routinely gamed by put-upon workers who frankly are given no reason to give a shit. Actually delivering the parcel is less important than being seen to have delivered it on paper. The management perception of performance and reliability weighs most heavily on the worker's shoulders - far more so than the customer's perception of the quality of service they're getting. So much so, in fact, that when I've dared to complain to sloppy service providers, they've called me a liar because their database systems clearly show a rosier picture than the one I, a mere customer, have painted.

This is all very well and good, having a Saturday morning moan and all. But there's a salient point at the end of all this. I've seen countless teams - Agile and otherwise - game their delivery data in much the same way. When you give the people doing the delivery the power to decide if something has been "delivered" and when their management relies on this highly subjective picture to assess the reliability of releases (by which I mean, when we say it's been delivered, has it really been delivered), we get a similarly disparity between the customer's perception and our own.

Of course, the postal delivery system is supposed to have checks and balances to make it more objective. How could something be "delivered" without a signature? Well, obviously it can, somehow. Perhaps the signature isn't really required. Or maybe any signature will do.

The solution is to place full responsibility for deciding if something has been successfully delivered with the end customer. And to establish very clear criteria as to what constitutes successful delivery. As it turns out, requiring a signature is by no means fullproof. What would probably work better is if customers were given a secret number or password to enter into the courier's little handheld computer thingy - or a barcode to scan - or something that can only be obtained by making contact with the person for whom the letter or parcel is intended.

In software projects, we need something that is a clear and unequivocal indicator that "the ball is in the hole". Not near the hole. In the hole. And this must be something that only the person who raised that specific requirement can control. Developers must not be allowed to make changes to the criteria. Only the customer should have that power. And these criteria must be beyond any doubt. They cannot be vague or handwavy, because that makes them game-able.

Practitioners of acceptance test-driven development will have some idea of what I'm getting at here. But I've watched teams doctor the test scripts without informing their customers, just to make a failing test pass and to meet a deadline, resolving to fix the real problems after they have "successfully delivered".

This cannot be allowed. ATDD works best when only customers have this power. Test scripts should be locked down in the customer's version control system, and nobody who touches the code should have the authority to check in new versions of the acceptance tests. If we want to move the goalposts, we must be forced to go to the customer and ask their permission.

Now, where's my bloody parcel?!

Wheel-driven Reinvention

 

One aspect of software development which is at once both amusing and troubling is the ability of us young whippersnappers to completely ignore what's gone before and reinvent established wisdom in our own image - often stealing the credit.

Take testing as an example. What do we know about testing software today that we didn't know, say, thirty years ago? Sure, we have new tools and testing has to fit within new approaches to the process of writing software as a whole, but fundamentally what have we discovered in the last decade or so?

Testing behaviour still works, by necessity, much as it has always worked by necessity. We must put the system under test in some desired initial state, then we must provide some stimulus to the system to trigger the behaviour we wish to test, then we must make observations about the final state of the system or about any behaviours that should have been invoked (e.g., a remote procedure call or a database request) in response to the combination of our stimulus and the initial conditions. And this process must be repeatable and predictable, like any good scientific test.

Though the culture of testing software may have evolved, much of it for the better, and the technology may have improved (though that is questionable), and though there are undoubtedly more people testing their systems today, when it comes to the business of writing and executing tests, there's really nothing new under the sun.

The same is true of many aspects of contemporary software development. Like it or nay, iterative and incremental development is older than C. We just weren't doing it back then, in the main.

Indeed, pick any "new" aspect of development and trace it back to its roots, and we discover that most novelties are actually much older than many of us thought. Objects are an invention from the sixties. Use cases hail from the seventies. Responsibility-driven design was being practiced before Frankie told us to Relax. UML existed in a fragmentary form before the Berlin Wall came down. People were writing code to satisfy tests back when those tests were stored on magnetic tape. Indeed, some of the descriptions of programming that was done for the very first computers rings bells with those of us who practice that black art today.

Younger developers like me, though, seem to readily believe that our time around is the first time around and feel no compunction to educate ourselves about the achievements of "old-timers", preferring instead to invent things anew - with sexier names and shinier tools, admittedly.

Our desire to reinvent goes as far as redefining words that already have a well-established definition. "Agile" no longer means "nimble" , "quick" or "spry". Today it apparantly means "communication, feedback, simplicity and courage". Or "iterative and incremental". Or "evolutionary". Or "Scrum-Certified". I caught someone the other day proferring their definition of "testable", which apparantly now requires us to go through "public interfaces". This is bad news for many scientists, who must now rewrite their peer-reviewed papers to incorporate the appropriate programming language with which to express the "testability" of their theories.

If software development was physics, we might expect newcomers to work through and understand the current body of knowledge before they start adding to it. That way, at the very least, we could avoid a great deal of duplication of effort. We may also avoid the tendency of our industry to throw "old-timers" on the scrapheap just because, even though they are probably just as current in their practical ability to deliver working software, they're not "down with the kids" on all the latest street slang for concepts that have been kicking around the block for decades.

The thinking of our elders and betters is far from irrelevent and outmoded. We can still learn a thing or two from the likes of Jacobson, Knuth and Hoare, should we choose to reject fashion in favour of substance in the approach we take to our work.