Software People Inspiring

Another Rant About Reinventing The Wheel And Against Yet More Navel-Gazing

 

Good morrow, gentle sir.

There's only so many times you can sit back and watch people "invent" stuff that's already been invented (several times over) before you start to realise how Neo must have felt when The Architect told him he was not the first to be called "The One".

Underlying any successful approach to software development is just a handful of core principles, all of which we've known about for decades. The recipe for working software - useful, reliable, maintainable working software - combines these staple ingredients with in a simmering reduction of talent (because any approach requires talent to work).

1. Tackle development in the smallest meaningful chunks that can be released to the customer. Any betting man will tell you that putting all your money on one horse is foolish in the extreme. Software development is very risky, so manage that risk by breaking one big gamble down into smaller, less risky punts. It also helps teams focus, by breaking requirements down into more manageable chunks.

2. Get meaningful feedback as often as possible, and adapt your software based on what that feedback teaches you. This is another pivotal element in successful - well - succesful anything. Iterate. We're not actually capable of solving complex problems any other way. It's been nature's solution right from the Big Bang. Life is a product of iterative design, as it happens. Oh, and get feedback by the most objective means you can. Just a hint: basing decisions to adapt your software on stuff people tell you is unwise. Because people mostly talk bollocks. Better to observe users actually trying to use the software you've delivered to do things you've agreed they should be able to do using the software. It help if you know why they're using your software. I know, it's a crazy idea! Which leads me neatly on to...

3. Have testable goals. Software development is expensive and hugely risky. Why gamble all that money if you don't have a clear picture of what it is you're spending that money for? What does your customer hope to achieve? How will they know if they've achieved it? At all levels from the business strategy that's driving the whole project or programme, down to understanding what a single method should do to contribute to those goals, it's incredibly useful to know precisely where the goalposts are. The best way to express goals testably is by writing tests. Just in case you were wondering. These could be project KPIs or business simulations and scenarios, or they could be behavioural specs written in RSpec, or unit tests written with JUnit. This is especially important when iterating. Feedback from tests can give you a very firm indication if you're getting warmer (or colder) with each pass.

4. Do the more important stuff sooner, especially if it's easier to do. Well, duh! Sounds obvious, right. Software development's a gamble. You've got a fixed amount to bet. If you leave your juiciest bets to last, you could run out of cash before you get the chance to place those juicy bets. It's not just a question of what a feature might be worth. Also bear in mind the cost and risk it might incur. Higher value, less costly/risky requirements should be further upstream in your requirements pipeline (or "backlog", to use the hip new street vernacular that I know many of you kids are into today).

5. If you find yourself doing something more than once, consider programming an evil robot to do it for you. Hey, you know what software's really good for? It's good for automating repetitive tasks. We've dedicated ourselves to taking the grind out of many people's jobs. Accountants can add up a balance sheet instantaneously and with total accuracy and consistency using spreadsheet software, for example. And think how long Avatar would have taken to render using coloured pencils. So when you're copying all those files across to the web server for the 27th time, consider how much time and trouble and risk might be saved by writing a program to do that for you. When you're writing yet another block of code to save and retrieve some object from a database, consider how much time you could save by automatically generating that kind of code. And so on. At the very least, train a monkey to do it.

6. Write your code so that it's actually possible to apply the first 5 principles, especially the one that adds most of the value - iterating your design. I'll make no bones about it. We have always known this. The biggest impediment to writing new code is the code you've already written. People in white coats carrying clipboards, probably working at the Laboratoire Garnier, have discovered that people like me and you spend most of our time staring at other people's code wondering what the buggery-hell it's doing. In this respect, the perennial killer germs are complexity, dependencies, duplication, poor regression test assurance and general lack of comprehensibility - often due to poor choices of names for variables, functions, classes and so on. While there's been something of a resurgence in disciplines that make code easier to understand an to change - like Domain-driven Design and Software Craftsmanship - we've actually known pretty much exactly what we should have been doing for several decades. So suck on that, Clean Code naysayers!

7. Use software developers who are good at developing software. No, seriously. It's all been for nought if, after you adopt your super-duper shiny new Agile Lean Kanban Unified Model-driven Development Method, you bus in a bunch of inexperienced, unpracticed, unmotivated, unprofessional code slobs. Just as sure as your super-duper shiny new Million Dollar Luxury Show Home will turn into a run-down hovel if you move in a bunch of first year med students. Indeed, I'd forgo 1-6 if I knew I had 7. Because a team of great software developers would probably do 1-6 anyway, instintictively and habitually. Of course, the fly in the ointment is that there's a very limited supply of such developers.

8. Get better at developing software. How many times have you done something for the first time and it's been just absolutely perfect? So perfect you couldn't possibly have done it better? Or done it quicker? Or done it cheaper? Or done it in a way that killed less horses? There's usually room for improvement in any activity or endevour, and this can be achieved by reflecting on what you've done and how you did it, aided by scientific principles to stop you jumping off the deep end into New Age Mumbo Jumbo Voodoo where you end up trying to improve design quality by homeopathic means or some such rubbish. Measurments can help, if they're good measurements and they're used in a grown-up way. Hard evidence generally is more useful than hearsay and anecdotal evidence. Check in histories, build reports, test reports, code metrics, screen captures, system logs, team height above sea level and average shoe size are just some of the useful data sources that can be analysed for clues as to how things could be done better. Smoke signals, aura readings and 100% discussion or opinion-based retrospectives tend to be the least useful. And practice needs to be part of the team's daily routine. Programming encompasses many theoretical discplines, but it's still by-and-large a practical skill, like speaking a foreign language. When you spot a deficiency highlihgted by your restrospectives, incorporating specially-designed exercises in your practice regime is a good way to turn a theoretical fix into a habitually-applied solution.


My point here is that there's no news here. All of these principles were known before most of us started our careers. Indeed, before many people had computers in their homes. What really would be novel is if we all started actually applying them. Instead of just dreaming up sexy new ways of talking about them. There's surely a lot more to discover about the art and science of software development. But when it comes to these core principles and "the old ways" that people like Knuth, Jacobson, Jackson, Hoare, Gilb and others were writing about while many of us were still in short trousers (or just glints in our parents' eyes), I think we're all talked-out.

Time for doing, not talking.

Software Is Both Art & Science. Can We Move On Now?

 

Bonjour, mes enfants.

SEMAT has gotten me thinking. Any mention of "engineering" and "science" in software development seems to polarise opinion.

Undoubtedly, there's a large section of the software development community who believe those words simply do not apply. Software development is not a science. It's an art. Or a craft. Like basket weaving. It's not engineering. No sir!

And there's an equally large section of the community who believe the exact opposite. Software development is a science. It is engineering. We can apply scientific principles to shape predictable, well-engineered end products.

Of course, they're both wrong.

Anyone who dismisses the notion of any kind of scientific basis for software development is running away from reality. Everything that exists has a scientific basis. Even American Idol. You just have to understand it. That we don't fully understand the science of software does not mean that no such science exists or that we'll never understand. I just can't help being reminded of UFOlogists who claim that "science cannot be applied to the study of UFOs". What they mean, of course, is "I've got a good thing going here selling my unscientific ideas to these schmucks, and I'd like to keep it that way, thanks".

And anyone who believes that software development can be completely tamed by science is equally deluded. There are sciences - emerging in recent decades - that teach us that there are many things in the world that, while it's possible that we can fully understand them, we will never be able to control them. Chaos is science. And software development is mostly chaos. Any vision of "software engineering", where pulling lever A causes X and pulling lever B causes Y with certainty, is the product of naivity at the macro, project scale. Life just isn't like that. Clockwork might be, but life isn't. Software development is intractably complex and unpredictable.

In the real world, biology has come up with processes that deal effectively - but not predictably - with intractably complex problems. Evolution is one such process. Evolution solves complex, multi-dimensional problems by iterating through generations of potential solutions. That is science. We can understand it. But we cannot even begin to predict what solution a process of evolution will reach, or how long it will take to reach it.

The clockwork, Newtonian paradigm of "software process enginering" is fundamentally flawed. And anyone who believes that it's possible to attain "value" deterministically is deeply mistaken. "If we pull lever A, we'll ship an extra 10,000 units". Give me a break!

Flawed, too, is any notion that this means that there's no engineering at all to be done in creating good software. I doubt anyone would claim that making rock music is "engineering" - well, anyone sane, at least. But there is science that can be applied within this process.

It's possible mathematically to predict what effect the choice of software compressor and the compression settings used will have on the amplitude of a recording across a certain frequency range. Indeed, it is helpful in getting the best-sounding mix. There is such a thing as "audio engineering" within the music production process. Granted, it's chiefly a creative process. But there is useful science we can appky within in to help tweak the results closer to perfection.

Similarly, while software design is chiefly a creative process, it can be useful to know if the code we're writing "smells" in any significant way. Some code smells can be detected just by looking at the code and using our judgement. Others are more subtle and harder to spot, but just as damaging to maintainability. Code analysis tools, as they grow more sophisticated, can complement our "eye for good design" every bit as much as audio engineering tools complement our "ear for a good mix" and music composition aids can complement our "ear for a good tune".

The trick, I believe, is to find the balance and work within the limitations of science and engineering and creative disciplines. Trying to figure out the formula for "valuable software" is every bit as futile as looking for a formula for making "hit records". And relying entirely on your eyes and ears to refine an end product has severe limitations. For millenia, we've used tools and theory to tweak and refine all manner of creative end products. We even have a word for it: "machine-tooled". The fact that you can fit a computer more powerful than all of the computers in the world were 30 years ago in your breast pocket, and it looks good, is testament to this symbiosis of science and art.

We can continue to refine and extend our scientific understanding of code and coding through research and exploration, just like any science. And new and useful tools will emerge from that understanding that will make it possible for us to produce better quality code, for sure.

And we can also continue to develop and refine the art of software development. Through reflection, practice and sharing.

There is such a thing as "software engineering", but it has limited scope, just like "audio engineering". Specifically, it's limited to things we can predict and can control, like what happens to coupling and cohesion if we move a class from one package to another.

The bigger picture of "delivering value" is a complex human endevour, and creativity, judgement and more than a sprinkling of luck is all we have that we can bring to bear in any meaningful way at this level. We may be capable of understanding, with the benefit of hindsight, why Feature X was used more than Feature Y when the software was released. But then, with hindsight, we understand quite a lot about volcanos and hurricanes, too. These are things that can only be understood with hindsight, really. We don't see them coming until they're almost upon us, and we have two choices - stay and risk everything or get out of the way and live to fight another day.

In years to come, I'll probably notice more and more a difference between "hand-rolled" software and software that has been written with some help from "software engineering" tools - the more grown-up descendants of tools like XDepend and Jester.

But I sincerely doubt I will ever be able to tell at the start of a project whether the resulting software will enjoy success or not. Sure, I'll be able to look back on projects and say "hey, y'know what we got wrong?" But far in advance, the outcome is every bit as unknowable as a hurricane, volcanic eruption or hit record. So where things like requirements and processes and "enterprise architeture" are concerned, I'll stick with arts and crafts.

Wheel-driven Reinvention

 

One aspect of software development which is at once both amusing and troubling is the ability of us young whippersnappers to completely ignore what's gone before and reinvent established wisdom in our own image - often stealing the credit.

Take testing as an example. What do we know about testing software today that we didn't know, say, thirty years ago? Sure, we have new tools and testing has to fit within new approaches to the process of writing software as a whole, but fundamentally what have we discovered in the last decade or so?

Testing behaviour still works, by necessity, much as it has always worked by necessity. We must put the system under test in some desired initial state, then we must provide some stimulus to the system to trigger the behaviour we wish to test, then we must make observations about the final state of the system or about any behaviours that should have been invoked (e.g., a remote procedure call or a database request) in response to the combination of our stimulus and the initial conditions. And this process must be repeatable and predictable, like any good scientific test.

Though the culture of testing software may have evolved, much of it for the better, and the technology may have improved (though that is questionable), and though there are undoubtedly more people testing their systems today, when it comes to the business of writing and executing tests, there's really nothing new under the sun.

The same is true of many aspects of contemporary software development. Like it or nay, iterative and incremental development is older than C. We just weren't doing it back then, in the main.

Indeed, pick any "new" aspect of development and trace it back to its roots, and we discover that most novelties are actually much older than many of us thought. Objects are an invention from the sixties. Use cases hail from the seventies. Responsibility-driven design was being practiced before Frankie told us to Relax. UML existed in a fragmentary form before the Berlin Wall came down. People were writing code to satisfy tests back when those tests were stored on magnetic tape. Indeed, some of the descriptions of programming that was done for the very first computers rings bells with those of us who practice that black art today.

Younger developers like me, though, seem to readily believe that our time around is the first time around and feel no compunction to educate ourselves about the achievements of "old-timers", preferring instead to invent things anew - with sexier names and shinier tools, admittedly.

Our desire to reinvent goes as far as redefining words that already have a well-established definition. "Agile" no longer means "nimble" , "quick" or "spry". Today it apparantly means "communication, feedback, simplicity and courage". Or "iterative and incremental". Or "evolutionary". Or "Scrum-Certified". I caught someone the other day proferring their definition of "testable", which apparantly now requires us to go through "public interfaces". This is bad news for many scientists, who must now rewrite their peer-reviewed papers to incorporate the appropriate programming language with which to express the "testability" of their theories.

If software development was physics, we might expect newcomers to work through and understand the current body of knowledge before they start adding to it. That way, at the very least, we could avoid a great deal of duplication of effort. We may also avoid the tendency of our industry to throw "old-timers" on the scrapheap just because, even though they are probably just as current in their practical ability to deliver working software, they're not "down with the kids" on all the latest street slang for concepts that have been kicking around the block for decades.

The thinking of our elders and betters is far from irrelevent and outmoded. We can still learn a thing or two from the likes of Jacobson, Knuth and Hoare, should we choose to reject fashion in favour of substance in the approach we take to our work.

What Makes Social Networks Grow Also Kills Them

 

I think social networks are like holiday destinations. Once they get really popular, they cease to be the idyllic, unspoilt paradises that made them so attractive in the first place.

Twitter's a bit like Cyprus. It may once have had a rustic mediterranean charm, all unspoiled beaches and quirky local tavernas and mile after mile of olive groves. But today it's just one big unfinished building site; a sea of English-themed pubs serving chips with everything and mile after mile of rooftop satellite dishes, clustered around beaches that are full to bursting with people that expect chips with everything, the same TV they get at home, and all the road signs and menus to be in English.

12 months ago, Twitter was great. It had a simple charm and was relatively unspoiled by porn, spam, advertising and the dickheads that have ruined other social networks like MySpace and Facebook. Most Twitter users were actual people. I mean real people. Who weren't selling porn or viagra. Real people who weren't dickheads. And it was a pretty swish neighbourhood, too. On Twitter, you could send a message to Stephen Fry or Jonathan Ross or Robert Llewellyn, and there was a decent chance that they'd actually respond. How jolly civilised and egalitarian.

Now it's all gone Pete Tong. The porn merchants have moved in, the spammers and scammers have started to operate in our previously fairly crime-free neighbourhood, and every other bloody person that follows me now is either a "social media guru" (and there's a four-letter word beginning with "c" that has the same essential meaning) or a robot with the body of an 18 year-old girl. And the beaches of Twitter are now so hopelessly overcrowded that someone like Stephen Fry, with thrumpty zillion followers all clamoring for his attention, is effectively just broadcasting to the majority of Twitter users - which kind of defeats the object of a social network, I suspect.

I've seen this process - from charming, unspoiled paradise to dirty, overcrowded destination for lager louts and sex tourists - happen many times on many social networks. I actually think that it's a natural lifeycle that they go through. And I think this is the main weakness of the current social networking business model. Your Web 2.0 site will turn into the Costa del Sol eventually. No matter what precautions you take, you must leave your network open enough to be useful to real people who want to connect with other real people. That openness - most vividly demonstrated by Twitter, which is pretty much a house made of glass with very feeble locks on the doors - is what makes your network grow. And it's also what makes your network grow ugly and eventually die.

A smart entrepreneur would recognise this fact of Web 2.0 life, and start putting their money into new destinations the moment they recognise the early signs of saturation.

They also need to accept that monetizing their social network will eventually kill it

Of course, a sneeky alternative might be to develop social networks that allow users to develop their own island paradises, far from the madding crowd. Handing over control of social networks, and treating them as a utility application, like email or calendar functions, might be one way to move forward. I reconnected with my old school friend Martin on MSN. Then on FriendsReunited. Then on Facebook. Now on Twitter. But he's still my old school friend and we own that relationship. Social networking sites come and go, but actual social networks - the ones that last and have some kind of real meaning - tend to persist.

And isn't that just like tourism? Holiday destinations may pass in and out of vogue. But what really makes a holiday is the company.

Macs Are Better (Because They Suck)

 

I'm not a Mac user. I have never owned a Mac, and the only device in my possession that was manufactured by Apple was a present; an iPod Nano from my brother.

People constantly tell me that I should buy a Mac. They say the Mac is more stable than Windows. That it is more secure.

I say, yes. I completely agree. Macs very probably are more stable, and they very probably are more secure. I don't need to own a Mac to know that. It's simple statistics.

According to various folk I've known who worked at or with Microsoft over the years, the underlying source of the bulk of reported Windows bugs is third party device drivers. Essentially, device drivers are hosted by the OS. In that sense, they're like Enterprise Java Beans. They are designed to present a specific interface and run inside a container. To do this safely, they must satisfy all the behavioural rules - including rules about concurrency and timing, which is always a bit fiddly to get right - required by the container. And not all 3rd party device drivers satisfy all of the rules all of the time. Every new device driver presents a finite risk to the stability of the OS. The more device drivers your OS supports, the greater the accumulated risk. Put simply, Macs are more stable than Windows because they support far fewer third party devices. Less choice, greater stability.

And of course Macs are more secure than PCs. Far fewer people are trying to attack the security of Macs. Why go to the expense of writing viruses and wotnot for such a niche platform? And also, there are far more ways of attacking a PC because there are far more ways to write code that will run on a PC. Again, with a Mac, less choice means greater security.

Macs aren't more stable and more secure because they're better engineered. The real reason why Macs are more stable and more secure than PCs is because they're less popular and offer less choice to their users.

In other words, Macs are better because, actually, they suck.

Screen Captures Hold Key To Raising Your Game

 

I've been experimenting with techniques to help us assess how effectively we apply good habits, and one technique in particular seems to be bearing early and very promising fruit.

Inspired by the example being set by Antony Marcano and Andy Palmer's pairwith.us project, I've been using screen captures to assess how well developers - myself included - apply basic Test-driven Development practices on short coding exercises.

Here's how we've been doing it:

Agree a set of no more than a dozen good TDD habits. These must be simple and enforceable (i.e., it is relatively easy to know when you're not doing them). They could be things like:

* Write the test assertion first and work backwards

* Make sure the test fails for the right reason(s) before you try to pass it

* Don't refactor when there are failing tests

* Refactor out any duplicate code after passing each test

And so on.

Your list of good habits should roughly represent what you (and your peers, if you're doing this in groups) believe is "good basic test-driven development".

What we want to know is, given a sufficiently representative amount of code to produce, how often do you break these good habits? How often do you catch yourself doing little refactorings when you should be just trying to pass a failing test? How often do you leave duplicate code after passing a test? How often do you forget to check that the test actually fails for the reasons you would want or expect it to?

The answer, if my own experience is anything to go by, is more often than you think.

Pick a simple coding problem. Here's a good sample one: write some code that will generate the Fibonacci Sequence of a specified length (e.g., when the length specified is 10, it must generate {0, 1, 1, 2, 3, 5, 8, 13, 21, 34}), where the minimum length allowed is 8 and the maximum length is 50.

That takes most programmers about 30-60 minutes to do using TDD. Using a screen capture tool like Camtasia or CamStudio, record yourself doing this exercise. If you're a fan of the pomodoro style of working, then you can break it into 25 minute recording blocks. Give yourself an hour (or two 25 min blocks) to complete the exercise.

When you've finished, take a break to help refresh your mind and then sit down with a pencil and a piece of paper on which you've listed all the good TDD habits you wish to assess yourself on. Watch the screen recordings back. Watch them very carefully indeed.

Every time you catch yourself breaking one of the habits, put a big X against that habit, and note the time at which you spotted it in the recording and note briefly what you caught yourself doing (or not doing, as the case may be).

Your aim should be to get less than 3 crosses in any hour of programming. It's 3 strikes and you're out, basically. I like to follow the "shoplifter rule". Wait until they have tried to leave the store before apprehending them. If the programmer does a refactoring and doesn't immediately run the tests afterwards, wait until they move on (e.g., to another test) before giving them an X.

If you're doing this in a peer group, then you can asses each other's screen recordings. And you can also vary the exercises and increase the length or the number of hours to make it even more of a challenge. A peer group in one of my clients will be doing 3 hours - in one of which they'll program in pairs and assess each other in pairs, and BOTH programmers get a X each if the pair slips up, meaning that the navigator really has to stop the driver from making mistakes.

Doing assessments is also great practice for pair programming, especially if the screen recording is captured without any audio. With just video, assessors have only the code to go on, which is a bit like a pair programming situation without talking or any other kind of communication. The driver must say what they have to say through the medium of code, making it clear enough that the assessor can understand what it is they're trying to do, and the assessor must concentrate and follow the rhythm of the TDD process so they can clearly see when they are at red light, green light or should be refactoring.

It might even be possible to do a kind of mutation testing by showing assessors a screen recording in which a variety of deliberate small mistakes are made to see how many they pick up on. You have to watch the code like a hawk, believe me!

As with all of these things, there's a bit more to it than I'm able to explain here. In the true spirit of software craftsmanship, you really have to see it to understand it. And you have to do it to really understand it.

The early results have been very exciting, though. It's becoming clear to me that this requires a level of concentration and focus on good practice that is head and shoulders above certainly what I thought was "rigorous" this time two years ago. And the results also strongly suggest that knowledge and cramming don't help. It's all about the habits. If you're not in the habit of doing these things, then at some point you'll slip up.

Which is why it certainly looks as though you need months of regular, focused practice - a few hours a week - to reach a point where you will be able to do 2-3 or more hours of solid TDD to the kind of standard this exercise demands.

It's all about increasing self-awareness. By examining closely what we do and how we do it, we tend to notice all sorts of little (and sometimes big) areas that need improving that might otherwise have gone unnoticed. It's a bit like practicing in front of a mirror for dancers. To do things right, they really need to get a clear view of themselves doing it.

The groups I'm working with now will soon be graduating on to "basic" refactoring - though, as I look more and more at what they'll be doing, they'll still be taking refactoring to a level very few people have mastered. Exciting times!

Twitter Re-Educates Me On "Value"

 

Twitter is currently teaching me a valuable lesson in - er - value.

Whenever I try to log into the service at the moment, be it via the web site, from my mobile phone or via TweetDeck, it's 50/50 that I'll actually get a response from the site.

If I am successfully logged in, when I submit an update it's 50/50 again that it'll actually get successfully posted.

Add to that some "logic quirks" with the functionality itself and some very dodgy AJAX nonsense going on, and we have one demonstrably flakey Web 2.0

And yet current speculation from those in the know (though the crisis in our financial markets maybe suggests that these people know nothing) values Twitter at some ridiculous, astronomical figure. We're talking the GDP of a small, but developed, economy here.

I am still convinced that a team of 4-6 good developers could deliver Twitter pretty much as it is on the Google App Engine in less than a fortnight. And deliver a far more reliable and robust implementation at that.

But how does the relationship between reliability and value work in this case? It calls into question the simplistic predictable relationship - a commoditised version of "value", if you like - that we've been assuming when we talk about things like "value streams" and "sustainable delivery of value".

If Twitter runs on just a few thousand lines of code, and I suspect there's very little under the bonnet in reality - then, if the service is valued in the billions, each line of code could be carrying the weight of millions of dollars in equity.

Imagine writing a line of code knowing that! How much time and effort would put in to making it right, and into making it scale?

Except, of course, Twitter's developers had no idea when they wrote it. And neither, I suspect, do any of us when we're writing code. No idea at all.

Make Roles Explicit: Another Brand New Idea From the Distant Past

 

One of the more frustrating consequences of having been a programmer for a certain number of years is that you've been around long enough to see the wheel being reinvented several times.

I was watching a video of someone speaking at this year's QCon in London about a "new" design principle: make roles explicit.

What he describes in his talk sounds very much like Interface Segregation. Classes should present multiple small client-specific (and by implication, role-specific) interfaces so that a change to one role it plays with respect to one client doesn't affect clients who depend on other methods relating to other roles. (e.g., the role of an Employee played by a Person with respect to an Organisation, which might use public methods like appraise() or calculateBonus() vs. the role of Parent played by the same Person class with respect to a School, which might use methods like inviteToParentsEvening() or requestPermissionForSchoolTrip() and so on).

Curiously, he describes how he learned this "new" principle from an "old" programmer. That should have been his first clue :-)

Also of interest is the use of generics, which tends to have the reverse effect he's seeking (instead of just having an ICollection, you specify an ICollection of Wotnots, which is less easy to substitute.)

Are We No Better Than Those UFO Nuts? The Case for a Software "Enlightenment"

 

Back to the subject of UFOs and the paranormal...

One thing that I find very entertaining about the UFO community is how authoratitive they manage to sound whilst piling speculation on top of conjecture on top of supposition on top of a very shaky bedrock of fundamental assumptions.

While less colourful students of the phenomenon dare only to ask "are THEY here?", many UFO "researchers" settled that debate decades ago in their own minds and are now preoccupied with more advanced questions like "are the small grey insect-like aliens at war with the tall lizard-like ones?" and "what does the High President of the Galactic Federation think about this whole Isreal/Palestine thing?"

Check out this classic example of the ouvre from self-styled (and what a style that is!) UFO expert Ed Komarek. How many totally unsupported claims did you count?

It's easy to laugh, of course. And, yes, I do think these folk are either completely ga-ga, or just plain crooked and they know they're making it up. The poor, ignorant saps who eat this stuff up from UFO fanzines ('cause that's what they are, folks - it's a cult of personality) and blogs and discussion groups are being led down a very long and windy garden path by charletans and nutcases who are every bit as screwy as astrologers and mediums - and priests, let's not forget.

At best we get pseudoscience. At worst, just plain drivel without even the shallowest attempt to justify it scientifically. UFO lore is 99% "he said, she said", with researchers relying mostly on anecdotes and spurious official memos to back up their outrageous claims.

But when I read the books, magazines, blogs and message boards of our illustrious software development community, I'm ashamed to say that we are equally piling speculation upon conjecture upon supposition on top of a very, very shaky foundation of assumptions. Pseudoscience is rampant in software development. Long-since debunked (if they ever were talen seriously) pop psychology and sociology, and totally antiquated or beyond-the-edge-of-the-fringe management theories abound. Everything from Myers-Briggs Types to the masterpiece of pseudoscience that is the Theory of Constraints - our profession is stuffed to the gills with unproven management theories that are actually nothing more than old wives tales and voodoo.

And it doesn't stop there: alarmingly, much - actually most - of what we say about writing software itself is also just a load of old mumbo-jumbo when you scratch beneath the thin veneer. A lot of the received wisdom about software design has not been validated by experiment, and the causal mechanisms behind so much of what we consider to be good or bad design choices are simply not understood.

And for every one of us who seeks empirical answers through some kind of testable theory of "code mechanics", there are 100 more who are quite happy to believe that you should not use the Singleton Pattern simply because "the book said so".

Just as for every person who seeks scientific proof of alien visitation there are 100 more who are happy to accept that they're here - or, indeed, not here (because both positions are unscientific if they're based only on faith) - purely because Stanton T Friedman or Nick Pope says they are.

As software developers, we are living in a dark age of ignorance and superstition, and one of my great hopes for the next decade is a resurgence in interest in the science of software (as well as a surge in practice of the craft of software development - because the two are by no means mutually exclusive) and perhaps one day we'll look back and call this transitional period the software industry's very own Enlightenment.

Twittizenship - A Metric-In-Progress

 

Just for jolly, I've been signed up to micro-blogging and social networking site Twitter for about three weeks now. Although the signal-to-noise ratio is quite poor (especially emenating from me), in just those few weeks some very interesting connections have been made and some very interesting outcomes achieved by "Tweeting" (the term Twitter uses for posting updates).

If you're not familiar with Twitter, it's pretty simple, really. There's the micro-blogging part, where you can post updates about what you're up to, or respond to other user's posts, or just post simple thoughts or links or wotnot. Because Twitter accepts posts by SMS, the length of a post is a maximum of 140 characters. So brevity is called for. (And, yes, it is a struggle, thanks for asking.)

Then there's the social networking part, where users can sign up to follow another user's updates, and connected users can send each other private direct messages. Thus a directed graph of followers and followed is constructed along which memes can propogate - which is the whole pay-off, I think.

Most users have a few dozen to a few hundred followers, and follow roughly as many. There's a kind of sense of obligation when someone follows you to check out their profile and - if they're not obviously a"marketing guru" or a "social media expert" (both euphamisms for "social network spammer") - you follow them back. Well, it's only polite, after all.

If someone you are following Tweets something of reusable interest, you can "Re-Tweet" (or "RT" for short) their post so that people following you (and not the original Tweeter) can see it. As with my famous forest fires, Re-Tweets may be Re-Tweeted, and as such can spread throughout the social network like wildfire when someone hits on a hot topic. Yesterday, Saving Bletchley Park was one of the most Re-Tweeted topics, thanks to a Re-Tweet by someone who we'll get to a bit later.

There are web sites that monitor Twitter traffic and report on various interesting trends, like what topics have been especially popular, or what posts have been re-Tweeted most that day, and so on. Thanks to it's simple API, Twitter data is pretty openly available. Unless they've specifically blocked a user (or all users) from seeing their timeline, any user's updates can be accessed, as well as lists of who they are following and who's following them.

This makes for some potentialy fascinating reading. For example, this list compiled for NOOP.NL of The Top 50 Twitterers To Follow For Software Development has given me hours of fun comparing the profiles and stats (and Tweeting styles) of the great and the good in our profession.

It's a commonly-held notion in Twitter culture, for instance, that Twitter is not a broadcast medium. It is considered the height of bad form to dish out update after update about yourself and mostly ignore the rest of the social network in which you are embedded. It's also considered a poor show if you are followed by lots and lots of people, but follow very few yourself. This means, essentially, that you are largely one-way traffic.

So, being a curious soul, I've devised a little metric - which I call Twittizenship - to help gauge just how good or bad a Twitter user has been at connecting with other users. Would you like to know how NOOP.NL's top 10 scored?

I've devised a formula thus:

TWITTIZENSHIP = (LOG(Users Followed)/LOG(Users Following)) x Number of Updates Addressing Other Users in Most Recent 25 Updates

In other words, the greater the proportion of people you follow to people who follow you, and the greater the number of times you reference other users in your updates, the greater your Twittizenship.

I've chosen to use Logs because there's a common sense issue here - as you get more and more followers, it becomes harder and harder to engage with them all, so I think I've made it fairer and more realistic by not keeping the ratio of following to followers linear.



Okay, so it's not an ideal metric. Just sampling the 25 most recent updates, for example, means that Twittizenship could fluctuate dramatically day to day. If I had the time, I would use the Twitter API to calculate the average per 25 Tweets across all of them.

And it fails to take into account the possibility of someone addressing the same tiny subset of Tweeters in multiple updates. Again, some jiggery-pokery with the Twitter API could calculate the reach as well as the frequency of their interactions with other Tweeters.

But the numbers I've come up with very roughly reflect my own qualitative impressions from examining these profiles. Kathy Sierra does seem to engage and interact noticeabley more than Jeff Atwood, for example.

Finally, for a bit of fun - as BBC elections pundit Peter Snow might say - let's do some interesting benchmarks that I hope will put these figures in some sort of context.

I have 131 followers and I'm following 133 other users. Out of the most recent 25 updates, 12 address other Tweeters. My Twittizenship is therefore 12.03. Let me assure you, I have not updated yet today or followed any new users, so I've not had a chance to game the numbers. Acclording to my metric, I'm a better Twittizen than the best in NOOP.NL's top 10. And, you know what, I reckon I jolly well may be, too.

Stephen Fry (I said we'd get to him) is an internationally famous writer, broadcaster and film director and has a massive Twitter following - 125,000 and growing very rapidly. His Twittizenship comes out at 8.84, but his stats are such an abberation that, like abnormally high IQ's (and I suspect he may be the owner of one of those, too) the figures are much harder to trust.

Jonathan Ross, another very famous UK Tweeter with a huge following, has a Twittizenship of 3.50

Okay, okay! I'm still working on it!