Software People Inspiring

Wheel-driven Reinvention

 

One aspect of software development which is at once both amusing and troubling is the ability of us young whippersnappers to completely ignore what's gone before and reinvent established wisdom in our own image - often stealing the credit.

Take testing as an example. What do we know about testing software today that we didn't know, say, thirty years ago? Sure, we have new tools and testing has to fit within new approaches to the process of writing software as a whole, but fundamentally what have we discovered in the last decade or so?

Testing behaviour still works, by necessity, much as it has always worked by necessity. We must put the system under test in some desired initial state, then we must provide some stimulus to the system to trigger the behaviour we wish to test, then we must make observations about the final state of the system or about any behaviours that should have been invoked (e.g., a remote procedure call or a database request) in response to the combination of our stimulus and the initial conditions. And this process must be repeatable and predictable, like any good scientific test.

Though the culture of testing software may have evolved, much of it for the better, and the technology may have improved (though that is questionable), and though there are undoubtedly more people testing their systems today, when it comes to the business of writing and executing tests, there's really nothing new under the sun.

The same is true of many aspects of contemporary software development. Like it or nay, iterative and incremental development is older than C. We just weren't doing it back then, in the main.

Indeed, pick any "new" aspect of development and trace it back to its roots, and we discover that most novelties are actually much older than many of us thought. Objects are an invention from the sixties. Use cases hail from the seventies. Responsibility-driven design was being practiced before Frankie told us to Relax. UML existed in a fragmentary form before the Berlin Wall came down. People were writing code to satisfy tests back when those tests were stored on magnetic tape. Indeed, some of the descriptions of programming that was done for the very first computers rings bells with those of us who practice that black art today.

Younger developers like me, though, seem to readily believe that our time around is the first time around and feel no compunction to educate ourselves about the achievements of "old-timers", preferring instead to invent things anew - with sexier names and shinier tools, admittedly.

Our desire to reinvent goes as far as redefining words that already have a well-established definition. "Agile" no longer means "nimble" , "quick" or "spry". Today it apparantly means "communication, feedback, simplicity and courage". Or "iterative and incremental". Or "evolutionary". Or "Scrum-Certified". I caught someone the other day proferring their definition of "testable", which apparantly now requires us to go through "public interfaces". This is bad news for many scientists, who must now rewrite their peer-reviewed papers to incorporate the appropriate programming language with which to express the "testability" of their theories.

If software development was physics, we might expect newcomers to work through and understand the current body of knowledge before they start adding to it. That way, at the very least, we could avoid a great deal of duplication of effort. We may also avoid the tendency of our industry to throw "old-timers" on the scrapheap just because, even though they are probably just as current in their practical ability to deliver working software, they're not "down with the kids" on all the latest street slang for concepts that have been kicking around the block for decades.

The thinking of our elders and betters is far from irrelevent and outmoded. We can still learn a thing or two from the likes of Jacobson, Knuth and Hoare, should we choose to reject fashion in favour of substance in the approach we take to our work.

Physicist Tests Journal of Cultural Studies - Finds Relativism Gone Mad!

 

Alan D. Sokal, a physicist at NYU, submitted a wordy article to a leading journal of cultural studies packed full of outrageous and totally unsupported claims suggesting that physical reality is a social construct and even that science should be led by a political agenda. And it got published. Without any corrections.

As Sokal puts it:

"What concerns me is the proliferation, not just of nonsense and sloppy thinking per se, but of a particular kind of nonsense and sloppy thinking: one that denies the existence of objective realities, or (when challenged) admits their existence but downplays their practical relevance. At its best, a journal like Social Text raises important questions that no scientist should ignore -- questions, for example, about how corporate and government funding influence scientific work. Unfortunately, epistemic relativism does little to further the discussion of these matters.

In short, my concern over the spread of subjectivist thinking is both intellectual and political. Intellectually, the problem with such doctrines is that they are false (when not simply meaningless). There IS a real world; its properties are not merely social constructions; facts and evidence DO matter. What sane person would contend otherwise? And yet, much contemporary academic theorizing consists precisely of attempts to blur these obvious truths -- the utter absurdity of it all being concealed through obscure and pretentious language."

One can't help wondering if many of our leading professional publications would be just as easily bamboozled by relativist nonsense. I have read articles that, at the time, I thought must surely be spoofs.

Software development abounds with outrageous and totally unsupported claims and the "epistemic relativism" Sokal accuses Social Text of championing. Most specifically, the pernicious and ultimately disastrous notion that your way is as good as my way is as good as their ways (so everybody gets to be right - hoorah for everybody!) and that, in software projects, there is no objective reality and all that matters is perspective and discourse. In less fancy language, we get to sit around yapping all day and everybody's opinion is equally valid - and evidence doesn't matter.

Indeed, as critics of metrics are fond of pointing out, reliance on evidence leads to oppression. I would argue that misinterpretation and/or misapplication of - often poor quality - evidence can and does lead to bad things in all kinds of organisational life. But I don't infer from this that seeking evidence is therefore always wrong (far from it), or that the reality of software and software development itself is somehow beyond empiricial understanding in certain key respects.

This is currently a very unfashionable view, and one for which I'm often chided by my peers. But I increasingly believe it to be important, and see a dark future for our profession if we continue down the slippery slope of woolly thinking that we're on.

The Movement Formerly Known As "Software Craftsmanship"

 

As with all intellectual movements, it seems the mere act of choosing a name is enough to stoke up heated debate.

It's actually not possible to use words currently in existence because they all have some kind of semantic baggage, and someone, somewhere is going to take offence somehow.

One possible solution is to take a leaf out of pop singer Prince's book and adopt an abstract symbol that's currently not in use.


The Movement Formerly Known As "Software Craftsmanship"

But then again, if you've ever taken the ink blot experiment, you'll know that even abstract shapes can get people hot under the collar.

On a related note, while some argue over the name, others continue to argue over what it is we actually believe in. Already there's a manifesto that's generating fun and games on discussion groups and blogs around and about the countryside. Meanwhile, a few folk seem to be venturing out into the Cursed Earth to preach the good word of Software Craftsmanship - from what bible I do not know, because we're still writing the outline.

Certainly, there seem to be different schools of emerging. Some, for example, see apprenticeship as a key requirement. I don't. Don't get me wrong, I think apprenticeships is a potentially great way to learn and improve. No argument there. But are there other ways? And are the people who tread that other path really "software craftsman"? (Sorry, are they really ...) Yes, I rather think they probably are.

Which all brings me to the conclusion that we may be overthinking things. Again. Because we're like that.

My Lovely But Still Overdue SC2009 Summary

 

So I finally got around to following up on last week's amazing Software Craftsmanship conference. With the formalities out of the way, I just wanted to add some personal notes about the experience, because that - apparantly - is what blogs are for.



First of all, I just want to go on record and say that I got a real kick out of organising this conference. I had some great help from Peter Camfield, Kerry Jones, Robin Doran and many others that meant that pretty much everything went smoothly on the day. It was an absolute pleasure to put together, and hearing the very positive - some might even dare to say "gushing" feedback on and after the day was the icing on the cake.

I was really chuffed with the programme we ended up with, and - contrary to how it almost always works - doubly chuffed to see expectations being exceeded in the execution of those sessions. I think we got a kick-arse, A1 bunch of sessions and session leaders, personally. I couldn't have wished for a better start to what may well turn into a new series of conferences and other related events.

Talking of feedback, I really, really think we're on to something here. Software craftsmanship is not a flash in the pan, I suspect. It's not just Le Fad De Jeur. Being surrounded by 100 committed professionals who genuinely care about what they're doing and who are 100% dedicated to learning and improving has been one of the highlights of my checkered programming career. I want more!

And I'm going to get it! SC2010 is definitely on the cards now, and I'm feeling very confident about pushing the envelope next year and taking this conference boldy to places where no conference has been before.

But 2010 is still a loooooong way aways yet. SC2009 had a palpable energy and momentum. You can feel it when you watch the Vox Pops video. And a sentiment that came out in much of the feedback I heard on the day is "let's make sure this doesn't fizzle out". So I'm looking with some urgency into setting up a regular get-together here in London where we can get hands on together, run dojos and katas over a coffee or a beer. Frankly there's been too much lips-flapping and not enough key-bashing of late, and we need more opportunities to - as Frank Zappa so eloquently put it - Shut Up And Play Our Guitars!

But London's just one little part of the world where craftsmanship has caught fire. There are other flames being fanned in far-flung places like Chicago, for example. Ironic that in a city named after their politician's empty rhetoric (the "windy city") we have a strong and rapidly maturing tradition of software craftsmanship growing thanks to folk like Micah Martin at 8th Light, Dave Hoover at Obtiva and - lest we forget - Uncle Bob Martin and Mike Feathers at Object Mentor. Micah crossed the pond especially for SC2009, I'm told, keen to see what's going down here in London. I'm equally keen to forge links with craftsman in Chicago and wherever else in the world they may be found.

In these days of global warming and rising fuel costs, though, we should look for ways to achieve rich international collaboration and sharing through technologies like Skype, desktop sharing, videoconferencing and atomic carrier pigeons (okay, I may have made that last one up). Hopping on planes and seeing folk in the flesh is going to be a necessity, I'm sure. But it's not a scalable or sustainable route to the level of sharing that I think we're going to need to really drive craftsmanship forward across borders.

On a totally personal note, it was very refreshing to see the "A" word taking a back seat at a conference for once. sure, lots of Agile folk there and lots of Agile practices, but the word that dominated the day was "craftsmanship" - which is as it should be now.

So, my summary summarised - By Jingo, I Think We May Be On To Something, Here!

Well, that's enough superlatives for one day. I'm off to read a whitepaper on some Model-driven Architecture nonsense.
Until next week, toodle pip!

Software Craftsmanship 2009 Follow-Up

 

This went out as an email to delegates, but is reproduced here for completeness

First of all, if you made it to SC2009 on Feb 26th then a big hearty thanks for helping to make the day the success that your generous feedback suggests it was.



An especially big thanks goes to everyone who ran the sessions. Your hard work is much appreciated.

Also, many, many thanks to Robin Doran from BBC Backstage, Peter Camfield from BBC Worldwide and Kerry Jones from BBC Future Media & Technology for all your invaluable assistance in putting the event together. Thanks, too, to all the BBC delegates who helped out on the day. You did a sterling job.

The event was very generously sponsored by BBC Worldwide and BBC Backstage. You should check them out, 'cause they're doing some pretty cool stuff these days:

http://backstage.bbc.co.uk

http://www.bbcworldwide.com

A few announcements:

* Were you at Keith Braithwaite's excellent TDD As If You Meant It? session? Do you still have a copy of the code you worked on? If the answer is Yes, then you might want to consider donating it to science! Please send zipped-up copies of your code to Dr Sue Black from University of Westminster so she and Steve Counsell from Brunel University can perform deranged experiments on it that hopefully will produce further insights into software maintainability.

* Don't forget to pay a visit to Sue's website dedicated to Saving Bletchley Park and see if you can help.

* A reminder about the upcoming Software Practice Advancement conference, which is being held in London this year at the BCS offices in Covent Garden starting on April 5th. I'm running a session on scaling up design reviews using automated code analysis. Apart from that, the rest of the programme looks pretty good, though ;-)

* Another reminder about Rachel Davies' Agile coaches gathering event which is being held at Bletchley Park on 22-23rd May

Following up:

Session Materials/Outputs

Immo Huneke has posted the outputs from his excellent and intimate session on My Favourite Keyboard Shortcuts on the - now publicly accessible - conference Wiki:

http://softwarecraftsmanship.wikidot.com/myfavouritekeyboardshortcutsoutputs

Gojko Adzic has posted slides and related material from his well-received session on Specification Workshops:

http://gojko.net/2009/02/27/links-and-slides-from-the-specifications-workshops-talk-at-the-software-craftsmanship-conference

Nat Pryce's notes for his fascinating session on Testing Asynchronous Systems can be find here:

http://www.natpryce.com/articles/000755.html

Ivan Sanchez posts his session materials for 5 Reasons To Have Coding Dojo here:

http://isanchez.net/2009/03/04/links-and-slides-from-my-session-at-the-software-craftsmanship-2009/

Feedback from the Blogosphere

Kerry Buckley very nicely summarises some of the sessions:

http://www.kerrybuckley.org/2009/03/02/software-craftsmanship-2009/

Gojko Adzic's great write-up of TDD As If You Meant It:

http://gojko.net/2009/02/27/thought-provoking-tdd-exercise-at-the-software-craftsmanship-conference/

Markus Gaertner's summary of the conference:

http://blog.shino.de/index.php?/archives/33-Software-Craftsmanship-conference-wrap-up.html

Diego Pino shares his thoughts here:

http://blogs.igalia.com/dpino/?p=22

Richard Fennell gets his thoughts down on virtual paper:

http://blogs.blackmarble.co.uk/blogs/rfennell/archive/2009/02/26/intent-is-the-key-thoughts-on-the-way-home-form-software-craftsmanship-2009.aspx

BBC Worldwide's Rob Bowley blogs about SC2009 and posts a great pic that illustrates how busy lunch was!

http://blog.robbowley.net/2009/02/28/software-craftsmanship-2009-round-up/

.NET journeyman Tim Ross posts:

http://timross.wordpress.com/2009/02/27/software-craftsmanship-2009/

SC2009 Vox Pops video

What Next?

Will there be a Software Craftsmanship 2010 conference? Very probably, yes. Watch my blog for developments, and get your thinking caps on for session ideas.

Where do we continue this dialogue? A good start would be to sign up for the Software Craftsmanship Google Group. Many of the folk you might have met on feb 26th are known to frequent the Extreme Tuesday Club in London. Also, you'll find us at Software Practice Advancement, XPDay and many other Agile-leaning events.

What about a specific regular meet-up on Software Craftsmanship? Bingo! Great idea. What a clever fellow you are for suggesting it :-) Yes, we shouldn't let the energy and momentum we gathered at SC2009 fixxle out, so I'm going to propose a regular meeting that will be a sort of mini-SC2009 where we specifically meet-up with our laptops for hands-on learning, sharing, practice and alcohol. (The four major food groups, I think you'll find.) The XTC venue is too noisy, really. So I'll be seeking out a friendly venue with a decent-sized function room and maybe even a projector and screen. I'm also leaning very heavily towards a weekend timeslot, because my brain is usually pretty frazzled by 7pm on a weekday! Keep an eye on my blog, and on the Google group, for announcements very soon.

I'm sure I've forgotten something, but isn't that always the way. Drop me a line with any thoughts, complaints or offers of easy money.

Are We No Better Than Those UFO Nuts? The Case for a Software "Enlightenment"

 

Back to the subject of UFOs and the paranormal...

One thing that I find very entertaining about the UFO community is how authoratitive they manage to sound whilst piling speculation on top of conjecture on top of supposition on top of a very shaky bedrock of fundamental assumptions.

While less colourful students of the phenomenon dare only to ask "are THEY here?", many UFO "researchers" settled that debate decades ago in their own minds and are now preoccupied with more advanced questions like "are the small grey insect-like aliens at war with the tall lizard-like ones?" and "what does the High President of the Galactic Federation think about this whole Isreal/Palestine thing?"

Check out this classic example of the ouvre from self-styled (and what a style that is!) UFO expert Ed Komarek. How many totally unsupported claims did you count?

It's easy to laugh, of course. And, yes, I do think these folk are either completely ga-ga, or just plain crooked and they know they're making it up. The poor, ignorant saps who eat this stuff up from UFO fanzines ('cause that's what they are, folks - it's a cult of personality) and blogs and discussion groups are being led down a very long and windy garden path by charletans and nutcases who are every bit as screwy as astrologers and mediums - and priests, let's not forget.

At best we get pseudoscience. At worst, just plain drivel without even the shallowest attempt to justify it scientifically. UFO lore is 99% "he said, she said", with researchers relying mostly on anecdotes and spurious official memos to back up their outrageous claims.

But when I read the books, magazines, blogs and message boards of our illustrious software development community, I'm ashamed to say that we are equally piling speculation upon conjecture upon supposition on top of a very, very shaky foundation of assumptions. Pseudoscience is rampant in software development. Long-since debunked (if they ever were talen seriously) pop psychology and sociology, and totally antiquated or beyond-the-edge-of-the-fringe management theories abound. Everything from Myers-Briggs Types to the masterpiece of pseudoscience that is the Theory of Constraints - our profession is stuffed to the gills with unproven management theories that are actually nothing more than old wives tales and voodoo.

And it doesn't stop there: alarmingly, much - actually most - of what we say about writing software itself is also just a load of old mumbo-jumbo when you scratch beneath the thin veneer. A lot of the received wisdom about software design has not been validated by experiment, and the causal mechanisms behind so much of what we consider to be good or bad design choices are simply not understood.

And for every one of us who seeks empirical answers through some kind of testable theory of "code mechanics", there are 100 more who are quite happy to believe that you should not use the Singleton Pattern simply because "the book said so".

Just as for every person who seeks scientific proof of alien visitation there are 100 more who are happy to accept that they're here - or, indeed, not here (because both positions are unscientific if they're based only on faith) - purely because Stanton T Friedman or Nick Pope says they are.

As software developers, we are living in a dark age of ignorance and superstition, and one of my great hopes for the next decade is a resurgence in interest in the science of software (as well as a surge in practice of the craft of software development - because the two are by no means mutually exclusive) and perhaps one day we'll look back and call this transitional period the software industry's very own Enlightenment.

We're Doing Agile. So Why Is Our Code Still Crap?

 

There's a debate bubbling away on James Shore's blog post about The Decline And Fall of Agile

I waded in a couple of times, and feel so strongly about the points I've raised that I thought I'd copy and paste the second comment here for your delight and delectation:

I work with a lot of teams who are struggling with the apparant paradox "but we're doing Agile, so why is our code still crap?" Very often a little bit of analysis of their end product reveals that they're actually not really doing many of the practices they claim to be.

They might, for example, believe that they're being test-driven, and yet a simple test coverage report might reveal that the majority of their code must have been written without a failing test that required it. Or they might think they're doing refactoring, but a bit of digging reveals a huge backlog of code smells and crippling levels of technical debt, for example. (And, let's be honest, when you actually pair with them, "refactoring" usually just means "hacking about randomly in the code").

It's the most common anti-pattern I see among Agile teams: the "oh, but we're different" syndrome, as they point and laugh at "lesser" non-Agile teams who are, in most practical senses, actually producing software that's no worse.

The first episode of the TV comedy "Nathan Barley" - written by the brilliant Charlie Brooker - has some useful insights into this. In the first episode, down-at-heel journalist Dan Ashcroft writes a scathing attack on fashion victims and new media trendies called "Rise of the Idiots", which is then feted by the very people he was attacking (Nathan Barley in particular), who genuinely believe he was writing about "someone else".

Talking about brilliant Guardian columnists, Malcom Gladwell has written a very thought-provoking piece this weekend on the relationship between practice/application and "genius". The magic number seems to be 10,000 hours - the time it roughly takes to become brilliant in some complex discipline.

Rant II over...

Totally Test-driven Scrum Delivers Better Objectivity In Measuring Real Progress

 

One of the more questionable precepts behind the Scrum virus is the way it does estimating at release and iteraton ("Scrum") levels.

At the release level, when we're sizing our product backlog, we use story points to estimate the relative complexity (or difficulty, or improbability) of user stories. This makes sense, because estimating in terms of effort or time tends to encourage the illusion of commitments being made about when stuff will be delivered. So it's best to avoid the temptation, especially at such a high and woolly level of estimation.

When we get to the iteration level, we suddenly ditch our "let's not mention hours or days" ideals and we're encouraged to break down our stories into engineering tasks and estimate those in - yep, you've guessed it - time required for completion.

I have two problems with this: one major and one minor. My major grip is actually about switching between two distinct models - deliverables and complexity, vs. tasks and effort. I actually have found through years of experience that the first model is the most useful.

There are two dangers with the tasks and effort model for planning iterations: one is that tasks are the "how" and it's quite possible to be 90% of the way through the tasks for a user story and have delivered nothing of any value to the users. So planning and tracking iterations through tasks can lead to the notoriously misleading "90% done" syndrome that has bitten so very many projects on the beehive in the past (and continues to do so.)

The other problem I have with the tasks and effort model is that there's an ever-present danger that when a developer estimates that some task will take "2 days", that is magically transformed by managers into a firm commitment to have it completed exactly 2 days from now.

I much prefer to break user stories down into tests and complexity points - sort of like a mini product backlog (let's call it the story backlog for now).

At the start of each Scrum, we do just enough analysis and design for each story to identify - and, I should stress, not to design in detail - the key test scenarios we'll need to tackle to implement that story to the point where the product owner agrees is useful.

On a board or a bit of free wall space - if you have any left (because I know how much you Scrum cats like to cover every square inch of every surface with your crazy crayon doodlings) - create a column for each user story, and in each column stick up a card for each named test scenario. Each story has an estimate of relative complexity. The sum of all the story ponts makes up the total complexity of the iteration.



Correspondingly, every test has complexity points. The sum of the points for all the tests for a story - the story backlog - adds up to 100% of the story points estimated for that story.

E.g., you have a story estimated at 23 story points. It has three tests with 23, 8 and 13 points respectively. If you have completed the first test, then you have completed 23/(23 + 8 + 13) * 23 story points for that iteation. Do you catch my drift?

Progress is measured entirely in terms of story points collected for tests passed. There is no "A for Effort" in my way of doing things. It's either testably delivered or it's not.

The analogy I use - and, yes, it's a golfing analogy - is to differentiate by measuring the completeness of a hole by whether or not the players took the shots they planned to vs. whether or not the ball actually finished up in the hole.

Using an acceptance test dashboard like the one illustrated above can be a very powerful way to communicate progress to project stakeholders and ensure a much greater objectivity in tracking progress within an iteration.

Everything Is Agile. Agile Is Everything.

 

After a recent flurry of coincidental trollings from various folk about the contentious term "Post-Agilism", I've come to a decision:

It's just not worth arguing about

The arguments are all of the circular "oh, but that is Agile, isn't it?" variety. To the extent that one correspondant claims that if doing waterfall development works for you, than that is Agile. Which is errent nonsense, surely.

And patently many folks believe that us post-Agilists (all 2 of us) should stick with the Agile movement even when we're not finding what we need in the Agile movement. Even when the Agile community are hostile to our line of enquiry.

I'm sensing a very real desire here to plant a flag for The Big "A" in everything good that comes along. If usability is good, then it's Agile. If Fagan inspections help your project, then they're Agile. If listening to Britney Spears and wearing your Spider-man pyjamas makes you code better, than that's Agile, too.

"I am Alpha and Omega."

There comes a point where I think that, for such a little thing, it's just not worth entering into debates. The people who don't get it, won't get it. To them, everything is Agile, and Agile is everything. And that's an end to the discussion.

It was only ever intended to be a useful contention. A diverting whimsy. A flim flam to stimulate a bit of free thinking.

Instead, it's opened a can of worms and stimulated a hostile - and occasionally vitriolic - defense of the Agile status quo, which - they argue - is not the status quo at all, but will be forever young, vital and omnipotently relevant.

So I guess we're going to have to agree to disagree, and I hope you'll pardon me from any further discussion or debate on this subject.

Another Post-Agile Job Advertisement

 

"Significant experience using the Agile, XP and post-Agilism development framework"

(From http://www.mima.org/jobs/index.asp?jobID=2103 )

So they're looking for a project manager with experience of constructive anarchy, then?

Good luck with that one.