Software People Inspiring

OO Design Principles In Practice

 

From the Codemanship blog:

I'm running the first of the newly redesigned OO design principles workshops this weekend in London (and a -day version on Sept 16th at Bletchley Park - places still available and all proceeds to charity).

OO design principles run the risk of being a bit too abstract and theoretical, and the major challenge has been to reframe my original training to be as hands-on and practical as possible. Indeed, this may be where so many OO courses and books have faltered.

A particular challenge is to apply the principles on a day-to-day basis. How does S.O.L.I.D. translate into coding habits?

Well, I've had a stab.

Read More...

Object Oriented Design Master Class, London Aug 21-22

 

This summer's run of budget-friendly weekend master classes in software craftsmanship ends with a workshop on object oriented design on August 21-22.


In this very hands-on course, you'll learn about OO design principles (S.O.L.I.D. and more), using real code examples to illustrate how to refactor designs that violate these principles. You'll also get practical experience of using code analysis tools to measure OO design quality and detect problems more easily, even in large code bases. The workshop will also give you a chance to apply simple OO analysis and design techniques in a test-driven approach to development, as we as looking at how up-front design can be balanced with refactoring to produce optimal designs without falling into the shark-infested waters of "Big Design Up-Front" and "no conscious design at all".


As with all the workshops I'm running this summer, there'll be the minimum of talk from me and the maximum of hands-on practical experience and learning by doing, working in pairs with other attendees.


And - as if the price wasn't good enough already - everyone who attends both the TDD and refactoring courses will get to go on this one absolutely FREE. I must be mad! Book your place now before I change my mind!

Software Is Both Art & Science. Can We Move On Now?

 

Bonjour, mes enfants.

SEMAT has gotten me thinking. Any mention of "engineering" and "science" in software development seems to polarise opinion.

Undoubtedly, there's a large section of the software development community who believe those words simply do not apply. Software development is not a science. It's an art. Or a craft. Like basket weaving. It's not engineering. No sir!

And there's an equally large section of the community who believe the exact opposite. Software development is a science. It is engineering. We can apply scientific principles to shape predictable, well-engineered end products.

Of course, they're both wrong.

Anyone who dismisses the notion of any kind of scientific basis for software development is running away from reality. Everything that exists has a scientific basis. Even American Idol. You just have to understand it. That we don't fully understand the science of software does not mean that no such science exists or that we'll never understand. I just can't help being reminded of UFOlogists who claim that "science cannot be applied to the study of UFOs". What they mean, of course, is "I've got a good thing going here selling my unscientific ideas to these schmucks, and I'd like to keep it that way, thanks".

And anyone who believes that software development can be completely tamed by science is equally deluded. There are sciences - emerging in recent decades - that teach us that there are many things in the world that, while it's possible that we can fully understand them, we will never be able to control them. Chaos is science. And software development is mostly chaos. Any vision of "software engineering", where pulling lever A causes X and pulling lever B causes Y with certainty, is the product of naivity at the macro, project scale. Life just isn't like that. Clockwork might be, but life isn't. Software development is intractably complex and unpredictable.

In the real world, biology has come up with processes that deal effectively - but not predictably - with intractably complex problems. Evolution is one such process. Evolution solves complex, multi-dimensional problems by iterating through generations of potential solutions. That is science. We can understand it. But we cannot even begin to predict what solution a process of evolution will reach, or how long it will take to reach it.

The clockwork, Newtonian paradigm of "software process enginering" is fundamentally flawed. And anyone who believes that it's possible to attain "value" deterministically is deeply mistaken. "If we pull lever A, we'll ship an extra 10,000 units". Give me a break!

Flawed, too, is any notion that this means that there's no engineering at all to be done in creating good software. I doubt anyone would claim that making rock music is "engineering" - well, anyone sane, at least. But there is science that can be applied within this process.

It's possible mathematically to predict what effect the choice of software compressor and the compression settings used will have on the amplitude of a recording across a certain frequency range. Indeed, it is helpful in getting the best-sounding mix. There is such a thing as "audio engineering" within the music production process. Granted, it's chiefly a creative process. But there is useful science we can appky within in to help tweak the results closer to perfection.

Similarly, while software design is chiefly a creative process, it can be useful to know if the code we're writing "smells" in any significant way. Some code smells can be detected just by looking at the code and using our judgement. Others are more subtle and harder to spot, but just as damaging to maintainability. Code analysis tools, as they grow more sophisticated, can complement our "eye for good design" every bit as much as audio engineering tools complement our "ear for a good mix" and music composition aids can complement our "ear for a good tune".

The trick, I believe, is to find the balance and work within the limitations of science and engineering and creative disciplines. Trying to figure out the formula for "valuable software" is every bit as futile as looking for a formula for making "hit records". And relying entirely on your eyes and ears to refine an end product has severe limitations. For millenia, we've used tools and theory to tweak and refine all manner of creative end products. We even have a word for it: "machine-tooled". The fact that you can fit a computer more powerful than all of the computers in the world were 30 years ago in your breast pocket, and it looks good, is testament to this symbiosis of science and art.

We can continue to refine and extend our scientific understanding of code and coding through research and exploration, just like any science. And new and useful tools will emerge from that understanding that will make it possible for us to produce better quality code, for sure.

And we can also continue to develop and refine the art of software development. Through reflection, practice and sharing.

There is such a thing as "software engineering", but it has limited scope, just like "audio engineering". Specifically, it's limited to things we can predict and can control, like what happens to coupling and cohesion if we move a class from one package to another.

The bigger picture of "delivering value" is a complex human endevour, and creativity, judgement and more than a sprinkling of luck is all we have that we can bring to bear in any meaningful way at this level. We may be capable of understanding, with the benefit of hindsight, why Feature X was used more than Feature Y when the software was released. But then, with hindsight, we understand quite a lot about volcanos and hurricanes, too. These are things that can only be understood with hindsight, really. We don't see them coming until they're almost upon us, and we have two choices - stay and risk everything or get out of the way and live to fight another day.

In years to come, I'll probably notice more and more a difference between "hand-rolled" software and software that has been written with some help from "software engineering" tools - the more grown-up descendants of tools like XDepend and Jester.

But I sincerely doubt I will ever be able to tell at the start of a project whether the resulting software will enjoy success or not. Sure, I'll be able to look back on projects and say "hey, y'know what we got wrong?" But far in advance, the outcome is every bit as unknowable as a hurricane, volcanic eruption or hit record. So where things like requirements and processes and "enterprise architeture" are concerned, I'll stick with arts and crafts.

I Have A Dirty Secret. I'm A Software Craftsman

 

I have a dirty secret.

No, it's not the one about the Nuns and the winter vegetables. It's even worse than that.

I keep it off my CV and I never, ever mention it to potential employers when seeking contract work.

To my shame, and despite my best efforts not to, when I write code I routinely achieve very high test assurance. We're not talking 70% of the code or even 80%. No, we're looking at a truly obscene 95% or higher. And I'm not just talking about code coverage here. No, I graduated from that years ago. This is the hard stuff. I actually test my tests. I always run them to make sure they fail when they're supposed to, and I often use mutation testing to seek out any gaps in my tests, and once found, I just can't help plugging them. (I think someone might have just fainted. )

But it gets worse. My depravity doesn't stop at test assurance. I'm addicted to simplicity. Take two if... statements into the shower? Not likely. I like to Extract'N'Go. I get the cold sweats when I see methods longer than a few lines of code or that do more than one thing or have more than one branch. If the code's written by me, you'll find no methods longer than about ten lines of code, and vanishingly few with a cyclomatic complexity greater than two.

Oh, the humanity!

But it doesn't stop there. I'm obsessed with making my code readable, even to non-programmers. Try as I might, I just can't seem to get the hang of variable names like "t" and "x" and "m_strFltYrpOxfyRtn", method names like "returnNonNullString()" and "do()", and class names like "HelperManagerUtils" and "AstractXmlVisitorFactory2". I've just got this mental block. I can't help thinking my names should have something to do with the problem the user wants us to solve for them. I know. It's crazy! I've even tried using comments to hide my shame. But I can never think of anything useful to add that isn't already obvious from reading the code. EPIC FAIL.

This affliction of mine has now entered an advanced stage of perversity. If I could have just been satisfied with simple, readable and well-tested code maybe I could have controlled it. But, noooo. That wasn't enough. I had to go and get myself mixed up in the whole "coupling and cohesion" can of worms. So my code isn't just simple, readable and tested. Now all my classes and packages are loosely coupled and cohesive, too. And I'm not just talking about a bit of "tell, don't ask" or the occasional "dependency injection" here, either. This is the full-blown Object Oriented Design Principles (and then some!) shooting match: Single Responsibility, Open-Closed, Liskov Substitution, Interface Segregation, Dependency Inversion, Law of Demeter, Acyclic Dependencies, Stable Dependencies, Stable Abstractions, and a few I made up myself (because after a while I just wasn't getting the same high from the old stuff like I did at the start.)

Of course, I never mention any of this to prospective employers. They'd have me marched out of the door faster than you can say "perfectionist" (if you'll pardon my French). You only have to search on the big IT job sites to know that code of this quality simply will not be tolerated. And quite right, too. Software development's hard enough for the average team without some deranged OCD-type rampaging through the code like a bizarre kind of reverse-bull-in-a-china-shop, making it all nice and neat and maintainable. Especially, if, like me, they don't even have the decency to take a lot longer to deliver features while they're doing it.

The bottom line is that we're all on to a good thing here. I totally understand. No team is going to want someone like me producing high quality, reliable software. No team is going to want someone who can keep on producing valuable new features and responding to evolving business needs month after month and year after year.

It's that sort of nonsense that just gives customers ideas. And that would never do, now would it?

Training 4 Bletchley - Peer Group Learning & Assessment Workshop, Bletchley Park, Sept 16th

 

Bono estente, et scorchio!

Firstly, a very heartfelt thanks to everyone who has generously pledged their pay for September 16th to help Bletchley Park. We're well on our way to achieving my initial target of 50 pledges by the end of this week. If you haven't pledged, please take a moment and consider helping. And remember, for every pound you donate, it could unlock a further 4 quid in lottery funds. And then there's Gift Aid...

On that subject, I've also pledged my earnings for September 16th. And, like many coaches and trainers, I have good days and better days (financially speaking). I'm keen to make September 16th one of the better days, which usually means running a training workshop.

And where better to run a training workshop in aid of Bletchley Park than, well, Bletchley Park? The room is booked and I already know what the workshop will be focusing on.

In the last 18 months or so, as some of you will know, I've been focusing on a new approach to coaching and helping teams of developers to learn, internalise that knowledge and build good habits through long-term practice and refelection with your peers.

The results the pilot groups at various clients have been achieving look very promising indeed, and the approach has been validated with code reviews and metrics that show clear improvements in key areas of maintainability like code complexity, duplication and regression test assurance, as well as the reductions in bug counts we might expect.

This approach of Peer Group Learning & Assessment has worked especially well at the BBC, and I and my counterpart there, Kerry Jones, are "doing the rounds" of a few conferences and writing up our experinces for the odd journal, which can give you a flavour of what it's all about.

Of course, there's more to it than can be conveyed in a 45-minute talk or in a 6-page article. To run your own Peer Group Learning & Assessment program, you really need a practical, hands-on introduction to the key elements of the program, as well as some useful tips I've picked up during these last 18 months.

So I'm proposing to run a one day workshop on Peer Group Learning & Assessment at Bletchley Park on September 16th, where people can get a proper handle on what it is, how it works, how you might run such a program in your organisation, and a heads-up on some the pitfalls I've encountered in a small - but diverse - range of clients.

This is going to be real training - I'm going to prepare and everything - and I'll be charging a real training fee for registration, but instead of writing your cheque to my company, you'll be paying your money directly to the Bletchley Park Trust and helping to preserve and transform a hugely important part of computing history. It'll be a packed day, and you will really get your money's worth.

The tea and sandwiches will be on me!

Usual address for details.

When Does "Delivered" Not Mean Delivered?

 

On occasion, I take a calculated risk and buy something off eBay. Ninety nine times out of a hundred, I suppose, the sellers are on the level and ship my item as soon as my payment's cleared. Right now, I'm awaiting delivery of an item from the US, which I purchased about two weeks ago.

The seller shipped it by standard international parcel post and I've been able to track its progress via the USPS web site. Which is how I know it left the US on Feb 11th and arrived in the UK on Feb 17th. So far, so good.

But this is where things all go a bit Twilight Zone-ish. According to USPS tracking (which must now be gettings its data from Royal Mail tracking at our end), they "attempted" to deliver it at 11.20pm last night.

Uhuh? Are we doing parcel deliveries after the pubs shut now? I believe not. "Attempted" delivery I understand to mean that they stuck it in a delivery van and the van came to my house and the driver knocked on my door and there was no answer. Or they couldn't find the house. Or something like that. Basically, they tried to deliver it, but were unable to for some reason.

I was most definitely at home at 11.20pm last night, and wide awake, and sober, and would most definitely have noticed if someone had knocked on my door at that unsociable hour.

This is not the first time that an overdue parcel delivery has been tracked as "delivered" or "attempted delivery" via our esteemed postal service - the oldest (and now, it seems, the creakiest) in the world. I orderd a box of expensive cigars a few months back and it disappeared into their system for several days, while all the time showing as "delivered" on their tracking web site.

It eventually turned up, of course. And I pointedly asked the nice lady who delivered it why it was down on their system as "delivered" when it quite obviously hadn't been up until then. And she admitted, with little hint of embarassment, that this happens all the time and is done to keep their performance statistics looking good.

This is a growing malaise in British life. Targets and measures are routinely gamed by put-upon workers who frankly are given no reason to give a shit. Actually delivering the parcel is less important than being seen to have delivered it on paper. The management perception of performance and reliability weighs most heavily on the worker's shoulders - far more so than the customer's perception of the quality of service they're getting. So much so, in fact, that when I've dared to complain to sloppy service providers, they've called me a liar because their database systems clearly show a rosier picture than the one I, a mere customer, have painted.

This is all very well and good, having a Saturday morning moan and all. But there's a salient point at the end of all this. I've seen countless teams - Agile and otherwise - game their delivery data in much the same way. When you give the people doing the delivery the power to decide if something has been "delivered" and when their management relies on this highly subjective picture to assess the reliability of releases (by which I mean, when we say it's been delivered, has it really been delivered), we get a similarly disparity between the customer's perception and our own.

Of course, the postal delivery system is supposed to have checks and balances to make it more objective. How could something be "delivered" without a signature? Well, obviously it can, somehow. Perhaps the signature isn't really required. Or maybe any signature will do.

The solution is to place full responsibility for deciding if something has been successfully delivered with the end customer. And to establish very clear criteria as to what constitutes successful delivery. As it turns out, requiring a signature is by no means fullproof. What would probably work better is if customers were given a secret number or password to enter into the courier's little handheld computer thingy - or a barcode to scan - or something that can only be obtained by making contact with the person for whom the letter or parcel is intended.

In software projects, we need something that is a clear and unequivocal indicator that "the ball is in the hole". Not near the hole. In the hole. And this must be something that only the person who raised that specific requirement can control. Developers must not be allowed to make changes to the criteria. Only the customer should have that power. And these criteria must be beyond any doubt. They cannot be vague or handwavy, because that makes them game-able.

Practitioners of acceptance test-driven development will have some idea of what I'm getting at here. But I've watched teams doctor the test scripts without informing their customers, just to make a failing test pass and to meet a deadline, resolving to fix the real problems after they have "successfully delivered".

This cannot be allowed. ATDD works best when only customers have this power. Test scripts should be locked down in the customer's version control system, and nobody who touches the code should have the authority to check in new versions of the acceptance tests. If we want to move the goalposts, we must be forced to go to the customer and ask their permission.

Now, where's my bloody parcel?!

Skepticism Is What Software Development Needs Now

 

I've blogged in the past about the rather alarming amount of hearsay and pseudoscience in software development.

Today I tried a little social experiment. On a social network. Which I thought was fitting.

I asked on Twitter whether typing speed was a significant factor in programmer productivity. As far as I'm aware, there's no data on this, so the answer of a scientist would have to be "we don't know". You may have your theories about whether it would help with flow if you typed faster, or whether writing more code in less time would mean you make more mistakes and waste even more time fixing them. But none of these theories has been put to the test.

Bottom line is, we just don't know.

I got quite a mix of responses, many of which had similar theories. But few said those magic words: I don't know.

As it happens, there's a whole bunch of stuff we just don't know in software development. Most of the accepted wisdom can be traced back to proclamations made with little or no evidence to back them up.

Refactoring makes change easier? Does it? I mean, really, does it? Where's the evidence to support that?

Sure, there's plenty of anecdotal evidence. And I'm every bit as guilty of saying with an air of certainty that refactoring makes change easier and other spurious proclamations. But there's plenty of anecdotal evidence for alien abductions, faith healing and the Loch Ness monster.

Now, I'm pretty sure refactoring does make it easier to change software, if you do enough of it and do it effectively. But I have not a shred of convicing evidence for that. I just believe it. Very strongly. Probably as strongly as Tony Blair believes bombing civilians in Iraq was the right decision. He is very, very wrong. How do I know I'm not wrong about refactoring? I don't. I take it on faith. Which isn't good enough, really, I'm afraid.

If we're to distinguish ourselves from the loonies sittings out in the Nevada desert wearing tin-foil hats waiting for aliens to show up and give us world peace, we need to start adopting a more practical kind of skepticism, and start subjecting some of these old wives tales to a more scientific kind of scrutiny.

Performance Metrics Often Hide The Reality From Managers

 

Over the years I've taken a special interest in the design and application of metrics. My interest was piqued way back in 1998 when I consulted on a project at The Post Office to develop tools for capturing, consolidating and analysing performance data from mail sorting operations. I read voraciously on the subject of performance measurement, and - even if I say so myself - became something of an armchair expert on it.

The only wisdom I tend to spout about designing metrics that might influence behaviour is "be careful what you wish for".

If someone's bonus, or even their whole job, depends upon showing good performance, as measured by a metric of our design, then we should fully expect them to steer their efforts directly towards achieving the best numbers, and - people being people - to do so with the minimal effort and application possible. In that sense, when performance metrics are applied seriously, the metric itself tells people exactly what it is you want them to achieve. Metrics are to performance what tests are to TDD. They are precise specifications.

It's ironic, then, that recently I've been witnessing some pretty blatant gaming of performance metrics from the very same organisation that inspired my interest in them. On two occasions I've ordered items online and paid a premium for Royal Mail's "special delivery" service. Special delivery items are "guaranteed" to be delivered before an agreed time the following working day. On the first occasion, the item didn't show up for several days - probably because of strike action. I got a tracking reference from the company I ordered the item from, and used Royal Mai's online Track & Trace feature, which showed that the item had been delivered the previous day. Literally as I was on the phone to the item's supplier, a Royal Mail driver appeared at my door with the item that had already been delivered. As I signed for it, I asked how come their own tracking system had it recorded that this item had been delivered the previous morning. She admitted that they often recorded delayed items as having been delivered to keep their performance numbers looking good.

Two week's later I'm expecting another item by special delivery that is currently a day later than was "guaranteed" (again, probably due to strike action), and this time the Track & Trace entry shows it as having been "returned to sender". I've been working at home both yesterday and today and nobody has called at the house, and no little card has been popped through my letterbox to tell me anyone tried to deliver it, which is what would happen if someone tried to deliver it but found nobody to sign for the item.

Again, I suspect the item will turn up in a day or two (two or three days later than they "guaranteed"), and their records will show the usual exemplary service.

It's highly likely that nobody in senior management is any the wiser to this metrics scam, or that the excellent service that their meaures tell them they're achieving might be the result of gaming rather than actual good service.

Indeed, I don't believe Royal Mail are alone in this. I suspect there are thousands upon thousands of organisations who - when you get above the level of the people being measured - have no clue that the quality of their products and services, in reality, is anything less than what their reports tell them. So it doesn't get addressed. And if you dare to complain, they will point to the 99% of satisfied customers who prove that it is you who is probably at fault here.

Deliveries are a big deal for businesses. And when customers pay a premium and their items don't show up on time, the supplier - and not the delivery firm - suffers, as does the customer. To sort out these problems, the first step might be to lift the veil on what's actually happening with these deliveries.

The design and application of metrics needs to go beyond simple, trite indicators like "the guy who drives the van says it's been delivered". You could drive a bus through the potential loopholes. A good way of tightening up the metric would be to apply a set of simple secondary tests to help identify this sort of gaming. Mystery customers are a great tool in these kinds of cases. If a small representitive sample - e.g., a few hundred - of the millions of deliveries Royal Mail handle were tracked not by computer but by actual human beings who can tell you whether the items actually were delivered when the record shows they were, then delivery drivers would soon realise that fabricating delivery times is too risky an option.

One wonders if they do something like that. From these experiences, I suspect not.

Codemanship Is Coming

 

Those of you whose hobbies include typing in infinite numbers of random domain names into your browser and seeing what web sites come up may have noticed that this blog is now the holding page for www.codemanship.com

I'm going to be building a new family of ideas and services around the Codemanship brand, and like all good families, we already have a motto:

Curo. Cognosco. Factito. Communico.

"To care. To learn. To practice. To share."

Being Skeptical About Metrics != Dismissing All Metrics

 

There's a growing and vocal tranche within the software development community who believe that all software metrics are bad.

Curiously, they don't believe that metrics are bad when their doctor tells them that their blood pressure is abnormally high, or when their union representative tells them that their salary is below the national average for their job description.

But attach any kind of number to anything related to the business of writing software and it's a different story. Metrics, to some, are the Devil's work and must be exorcised from software development.

It's true that metrics can and often are abused in management. Decades of cooked books, skewed studies and notoriously optimistic "government statistics" have conditioned many of us to mistrust numbers. And rightly so. Whenever anyone presents you with a measurement, it's wise to be skeptical and question what it might mean and how it was collected. But dismissing all measurements as a rule is not skepticism. It's just as much an act of blind faith as believing every statistic you see.

Software development without metrics is woolly and very, very highly subjective. It's like the Olympics without stopwatches or cookery without a set of scales. Metrics debunkers sometimes paint a picture of the other extreme as being the reality of metrics - namely total reliance on arbitrary numbers, even when they fly in the face of our subjective reality.

I would not advocate blindly accepting what your metrics tell you. But I would definitely not advocate ignoring the numbers, either. If Max Planck had thought like that, we'd never have ended up with quantum mechanics, which - measured in terms of the accuracy of its predictions - is our most successful science. Sometimes the reality is what the numbers say it is, and not what our eyes or our instincts tell us.

And sometimes, and especially in very complex matters, simple measurements hide a more subtle truth, and our instincts serve us best - provided we've had the time to develop them to the point where they can be trusted. Estimating is a great example of something where the first instinct of a senior programmer is just as likely - if not more likely - to be right than any number derived through some kind of empirical COCOMO nonsense.

But that doesn't apply to every aspect of software development. If a coverage tool tells you that 50% of your code isn't executed during a test run then you know that 50% of your code isn't being tested at all. If a static analysis tool tells you that one of your methods has 500 lines of code and a cyclomatic complexity of 80 then you know it will be practically untestable and very difficult to change. If JDepend tells you that you have a package full of concrete classes that is very heavily depended upon then you know that making changes to code in that package is likely to have a bigger knock-on effect because of the lack of substitutability.

Sure. Absolutely. I won't deny that metrics throw up false positives and can sometimes send us on wild goose chases. But I'll tell you this: I have never seen a project team that delivered high quality software that wasn't monitoring quality using some kind of measurements. I suspect that's because testable goals tend to be easier to aim for.

By all means, be a metrics skeptic. Question every number you see. If a metric tells you something about reality, go look at reality and use your hard-earned judgement to see if you agree with the metric.

But dismiss all software metrics out of hand at your peril. Because I know that software teams who don't measure quality tend not to deliver very good software. Successful development teams use metrics. Of course, there are plenty of teams who don't measure anything and who think they're successful. There are plenty of people who think they have psychic powers, too.