The Moral Architect

I started my career in the telecommunications division of the General Electrical Company (GEC) as a software engineer designing digital signalling systems for Private Branch Exchanges based on the Digital Private Network Signalling System. As part of that role I represented GEC on the working party that defined the DPNSS standard which was owned by British Telecom. I remember at one of the meetings the head of the working party, whose name I unfortunately forget, posed the question: what would have happened if regimes such as those of Nazi Germany or the Stalinist Soviet Union had access to the powerful (sic) technology we were developing? When I look back at that time (early 80’s) such “powerful technology” looks positively antiquated – we were actually talking about little more than the ability to know who was calling whom using calling line identification! However that question was an important one to ask and is now one we should be asking more than ever today.One of the roles of the architect is to ask the questions that others tend to either forget about or purposely don’t ask because the answer is “too hard”. Questions like:

  • So you expect 10,000 people to use your website but what happens if it really takes off and the number of users is 10 or 100 times that?
  • So you’re giving your workforce mobile devices that can be used to access your sales systems, what happens when one of your employees leaves their tablet on a plane/train/taxi?
  • So we are buying database software from a new vendor who will help us migrate from our old systems but what in-house skills do we have to manage and operate this new software?
  • Etc

In many ways these are the easy questions, for a slightly harder question consider this one posed by Nicholas Carr in this blog post.

So you’re happily tweeting away as your Google self-driving car crosses a bridge, its speed precisely synced to the 50 m.p.h. limit. A group of frisky schoolchildren is also heading across the bridge, on the pedestrian walkway. Suddenly, there’s a tussle, and three of the kids are pushed into the road, right in your vehicle’s path. Your self-driving car has a fraction of a second to make a choice: Either it swerves off the bridge, possibly killing you, or it runs over the children. What does the Google algorithm tell it to do?

Pity the poor architect who has to design for that particular use case (and probably several hundred others not yet thought of)! Whilst this might seem to be someway off, the future, as they say, is actually a lot closer than you think. As Carr points out, the US Department of Defence has just issued guidelines designed to:

Minimize the probability and consequences of failures in autonomous and semi-autonomous weapon systems that could lead to unintended engagements.

Guidelines which presumably software architects and designers, amongst others, need to get their heads around.

For anyone who has even the remotest knowledge of the genre of science fiction this is probably going to sound familiar. As far back as 1942 the author Isaac Asimov formulated his famous three laws of robotics which current and future software architects may well be minded to adopt as an important set of architectural principles. These three laws, as stated in Asimov’s 1942 short story Runaround, are:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

As stated here these laws are beautifully concise and unambiguous however the devil, of course, will be in the implementation. Asimov himself went on to make quite a career of writing stories that tussled with some of the ambiguities that could arise from the conflicts between these laws.

So back to the point of this blog. As our systems become ever more complex and infringe on more and more of our everyday lives are ethical or moral requirements such as these going to be another set of things that software architects need to deal with? I would say absolutely yes. More than ever we need to understand not just the impact on humanity of those systems we are building but also those systems (and tools) we are using everyday. As  Douglas Rushkoff says in his book Program or be Programmed:

If you don’t know what the software you’re using is for, then you’re not using it but being used by it.

In a recent blog post Seth Godin poses a number of questions of what freedom in a digital world really means. Many of these are difficult moral questions with no easy answer and yet systems we are building now, today are implicitly or explicitly embedding assumptions around some of these questions whether we like it or not. One could argue that we should always question whether a particular system should be built or not (just because we can do something does not necessarily mean we should) but often by the time you realise you should be asking such questions it’s already too late. Many of the systems we have today were not built as such, but rather grew or emerged. Facebook may have started out as a means of connecting college friends but now it’s a huge interconnected world of relationships and likes and dislikes and photographs and timelines and goodness knows what else that can be ‘mined’ for all sorts of purposes not originally envisaged.

One of the questions architects and technologists alike must surely be asking is how much mining (of personal data) is it right to do? Technology exists to track our digital presence wherever we go but how much should we be making use of that data and and to what end? The story of how the US retailer Target found out a teenage girl was pregnant before her father did has been doing the rounds for a while now. Apart from the huge embarrassment to the girl and her family this story probably had a fairly harmless outcome however what if that girl had lived in a part of the world where such behavior was treated with less sympathy?

It is of course up to each of us to decide what sort of systems we are or are not prepared to work on in order to earn a living. Each of us must make a moral and ethical judgment based on our own values and beliefs. We should also take care in judging others that create systems we do not agree with or think are “wrong”. What is important however is to always question the motives and the reasons behind those systems and be very clear why you are doing what you are doing and are able to sleep easy having made your decision.

Advertisements

Making Smart Architecture Decisions

Seth Godin whose blog can be found here, posted a wonderful entry called The extraordinary software development manager yesterday which contains some great axioms for managing complex software projects, the last of which is something I wish I’d written! Here is what it says:

Programming at scale is more like building a skyscraper than putting together a dinner party. Architecture in the acquisition of infrastructure and tools is one of the highest leverage pieces of work a tech company can do. Smart architecture decisions will save  hundreds of thousands of dollars and earn millions. We’ll only make those decisions if we can clearly understand our options.

As I’ve said elsewhere architecture decisions are one of the most crucial social objects to be exchanged between stakeholders on a software development project. For a very thorough description of what architecture decisions are and how they can be captured see the article Architecture Decisions: Demystifying Architecture by Jeff Tyree and Art Akerman. However you don’t need to make every decision to such a precise level of detail. At a minimum you just need to make sure you have looked at all options and assumptions and have captured the rationale as to why the decision was made. Only then can you move forward reasonably safe in the knowledge that your decisions will stand the test of time and, as a bonus, will be transparent for all to see and to understand the rationale behind them.

Good architecture decisions will form part of the system of record for the project and the system itself, once it is built. They will allow future software archeologists to delve deeply into the system and understand why it is the way it is (and maybe even help fix it). They also help newcomers to the project understand why things are the way they are and help bring them up to speed more quickly on the project. Finally, and possibly most importantly of all, they stop others making different decisions later on with possibly great financial implications to the project. If you can read and understand the rationale behind why a decision was made you are less likely to overturn the decision or ignore it all together.

Architecting and Social Media

The arguments for and against the rise of user generated content on the web continue to rage. Depending on which side of the debate you take we are either on an amoral downward spiral of increasingly meaningless content being generated by amateurs (for free) that is putting the professional writers, musicians, software developers, photographers etc out of work as well as ruining our brains or are entering a new age where the combination of an unbounded publishing engine and the cognitive surplus many people now have means we are able to build a better and more cooperative world.Like most things in life the truth will not be at one of these polar opposites but somewhere in between. Seth Godin makes an interesting point in a recent blog entry that “lifestyle media isn’t a fad it’s what human beings have been doing forever, with a brief, recent interruption for a hundred years of professional media along the way”. He goes on to say “we shouldn’t be surprised when someone chooses to publish their photos, their words, their art or their opinions. We should be surprised when they don’t.”

After all, given the precarious nature of the press in the UK at the moment with stories being obtained through all sorts of dubious means the professionals can hardly be seen as holding the ethical or moral high ground.

The possibilities for creativity, and building interesting and innovative solutions out of this mixed bag of social media self-publishing is going to be the place where architects are going to have a fertile ground over the coming years. A nice example of this is Flipboard which if you have an iPhone or iPad you should definitely download. This free app is a “social magazine” that extends links your friends and contacts are sharing on Facebook, Twitter, Linkedin, 500px and others into beautifully packaged “articles”. It can also pull in content from a raft of other online content. It’s a great example of what architects should be doing, namely taking existing components and assembling them in interesting and innovative ways.

Creativity at Scale

I stumbled across this article by Grady Booch today whilst doing some research on complex systems. It’s quite old (well 3 years is an age in this profession) but what it has to say about developing complex systems is, I believe, timeless. In particular I loved Grady’s point about the importance of “developing social structures that encourage innovation whilst still preserving predictability”. Interestingly another post by Seth Godin says:  

The challenge of our time may be to build organizations and platforms that engage and coordinate the elites, wherever they are. After all, this is where change and productivity come from.

Seth’s definition of an “elite” is anyone who is “actively engaged in new ideas, actively seeking out change, actively engaging.

The internet is a great platform for bringing people together. The challenge is in how to scale the creativity, that can be easily bought to focus in a small, co-located team, across the internet. The boundaries of the enterprises that used to do this are beginning to crumble. There is often more creativity outside the enterprise than is in it and new architectures (new platforms) need to learn how to make use of this. This is more than just what has been happening with the open source movement, where a group of people come together to build new products, its about how to use creativity at scale to build new platforms.

~~~~

Spookily (well it is Halloween) and with a great bit of webendipity just as I was about to push the publish button on this article up popped in my twitter feed this article by my colleague Jeremy Caine about platforms (e.g. Facebook, iTunes, Twitter) and the creatives that build them.

The Legacy Issue

Much of the work we do as architects involves dealing with the dreaded “legacy systems”. Of course legacy actually means the last system built, not one that is necessarily 5, 10, 20 or more years old. As soon as a system goes into production it is basically “legacy”. As soon as new features get added that legacy system gets harder to maintain and  more difficult to understand; entropy (in the sense of an expression of disorder or randomness) sets in.

Apple have recently been in the news again for the wrong reasons because some of the latest iPod’s do not work with previous versions of Mac OSX. Users have been complaining that they are being forced to to upgrade to the latest version of OSX in order to get their shiny new iPods to work. To make matters worse however Apple do support the relatively ancient version of Windows XP. Apple have always taken a fairly hard line when it comes to legacy by not supporting backwards compatibility particularly well when their OS gets upgraded. The upside is the operating systems does not suffer from the “OS bloat” that Windows seems to (the last version of OSX actually had a smaller footprint than the previous version).

As architects it is difficult to focus both on maintaining legacy systems and also figuring out how to replace them. As Seth Godin says:“Driving with your eyes on the rearview mirror is difficult indeed”. At some point you need to figure out whether it is better to abandon the legacy system and replace it or soldier on supporting an ever harder to maintain system. There comes a point where the effort and cost in maintaining legacy is greater than that needed to replace the system entirely. I’m not aware of any formal methods that would help answer this particularly hard architectural decision but it’s one I think any architect should try and answer before embarking on a risky upgrade program that involves updating existing systems.

The Essence of Being an Architect

There are many skills frameworks out there that tell us what skills we should have for ‘doing architecture’. My company has (at least) one and I’m sure yours does as well. There are also organisations that specialise in creating such frameworks (check out the Skills Framework for the Information Age for example). Whilst there are some very specific skills that software architects need (developing applications using xyz programming language, building systems using a particular ERP package and so on) which come and go as technology evolves there are some enduring skills which I believe all architects must acquire as they progress in their careers. What I refer to as being the essence of being an architect. Seth Godin recently posted a blog entry called What’s high school for? where he listed 10 things all schools should be teaching that should sit above any of the usual stuff kids get taught (maths, chemistry, history etc). A sort of list of meta-skills if you like. Borrowing from, and extending, this list gives me my list of essential architect skills.

  1. How to focus intently on a problem until it’s solved. There is much talk these days about how the internet, the TV networks and the print media are leading to a dumbed down society in which we have an inability to focus on anything for longer than 30 minutes. Today’s business problems are increasingly complex and often require prolonged periods of time to really focus on what the problem is before charging in with a (software) solution. Unfortunately the temptation is always often to provide the cheapest or the quickest to market solution. You need to fight against these pressures and stay focused on the problem until it is solved.
  2. How to read critically. As architects we cannot hope to understand everything there is to know about every software product, technology or technique that is out there. Often we need to rely on what vendors tell us about their products. Clearly there is a danger here that they tell us what we, or our clients, want to hear glossing over faults or features that are more ‘marketechture’ than architecture. Learn how to read vendor product descriptions and whitepapers with a critical eye and ask difficult questions.
  3. The power of being able to lead groups of peers without receiving clear delegated authority. The role of an architect is to build solutions by assembling components in new and interesting ways. You are the person who needs to both understand what the business wants and how to translate those ‘wants’ into technology. Business people, by and large, cannot tell you how to do that. You need to lead your peers (both business people and technologists) to arrive at an effective solution.
  4. How to persuasively present ideas in multiple forms, especially in writing and before a group. Obvious really, you can have the greatest idea in the world but if you cannot present it properly and effectively it will just stay that, an idea.
  5. Project management, self-management and the management of ideas, projects and people. How to manage your and others time to stay focused and deliver what the client wants in a timely fashion.
  6. An insatiable desire (and the ability) to learn more. Forever! This job cannot be done without continuous learning and acquiring of knowledge. Everyone has their own learning style and preferences for how they acquire knowledge, find out what your style is and deploy it regularly. Don’t stick to IT, I’ve discussed the role of the versatilist extensively (see here for example). Be ‘V’ shaped not ‘T’ shaped.
  7. The self-reliance that comes from understanding that relentless hard work can be applied to solve problems worth solving. Belief in ones ideas and the ability to deploy them when all around you are doubting you is probably one of the hardest skills to acquire. There is a fine balance between arrogance and self-belief. In my experience this is not an easily repeatable skill. Sometimes you will be wrong!
  8. Know how to focus on what is important and to ignore what is not. If you have not heard of Parkinson’s Law of Triviality take a look at it.
  9. Know who the real client is and focus on satisfying him/her/them. There can be lots of distractions in our working lives, and I’m not just talking about twittering, blogging (sic) and the rest of the social networking gamut. Projects can sometimes become too inward focused and lose track of what they are meant to be delivering. We live in a world where numbers have achieved ascendency over purpose. We can sometimes spend too much time measuring, reviewing and meeting targets rather than actually doing. I love this quote from Deming: “If you give a manager a numerical target, he’ll make it, even if he has to destroy the company in the process”. There is little merit in a well executed project that no one wants the output from.
  10. Use software/system delivery lifecycle (SDLC) processes wisely. SDLC’s are meant to be enablers but can end up being disablers! Always customise an SDLC to fit the project not the other way around.

If all of this seems hard work that’s because it is. As Steven Pressfield says in his book The War of Art:

The essence of professionalism is the focus upon the work and its demands, while we are doing it, to the exclusion of all else.

Change Cases and the Limits of Testing

This recent blog entry from Seth Godin on “the culture of testing” set me thinking about software architecture and testing. Is it possible to ‘over test’ applications or systems? Is there a point at which you need to stop testing and let your software ‘go free’ so users can ‘complete’ testing themselves? Let’s be clear here:

  • Software needs testing very rigorously. No one wants to fly on an airplane where the on-board flight control software has not been fully tested.
  • I’m also not talking about the well established practice of releasing beta versions of software where you get together a bunch of early adopters to complete testing for you and iron out the “last few bugs”.
  • Testing software against known requirements is most definitely a good thing to do and I’m not advocating just running your system tests against a subset of those functional and non-functional requirements.

Where it gets interesting is when you don’t overly constrain the architecture so that it allows users to take resulting application or system and evolve it (test it) in new and interesting ways. When defining an architecture we usually talk about it having to address functional as well as non-functional requirements but there is a third, often overlooked, class of requirement referred to as change cases or future requirements. Change cases are used to describe new potential requirements for a system or modifications to existing requirements. Change cases usually come from the specifier of the system (e.g. “in three years time we want introduce a loyalty scheme for our regular guests”). Some change cases however are not envisaged in advance and it’s only when users of the application get hold of it that they explore and find new ways of using the application that may not have originally been thought of. Such applications need to be carefully architected, and tested, such that these change cases can be discovered without, of course, breaking the application altogether.

So, by all means ship systems or application that do what they say on the tin but also lets users do other, possibly more interesting things, with them that may lead to new and innovative uses that the original specifiers of the software had not thought about.