Architecture Work Products as Social Objects

As architects we clearly need to describe the architectures we are creating. Sometimes what we are trying to describe can be fairly abstract and it helps to have a common and well understood set of work products to describe and facilitate an understanding of the concepts we are trying to get across. I’ve previously talked about how architecture social objects can be used as a communication device that generates conversation and action around the object. This blog post by David Johnson suggests that all social objects have three things in common, which I think further reinforces the usefulness of a well thought out set of architecture social objects. According to Johnson the three things social objects have in common are:

  1. Conversational: people want to talk and have conversations with other people connected with the social object.
  2. Brings People Together: people want to be around other people that are connected with the social object. They feel part of a community, that they belong with each other.
  3. Talk Worthy: people feel the desire to tell other people, who may not know about the social object, so that they, in turn, become part of the community.

Over time I’ve discovered there is a core set of work products that form the social objects which bring people together and can act as conversational pieces for architects to talk about. This is especially important when you have to quickly create a (partial) architecture that shows how a number of software components (both bespoke and off-the-shelf) can work together to address a set of requirements.

It’s especially useful to do this when there is little time to create such solutions, as when you may be responding to an invitation to tender (ITT) from a client for example. At such times requirements are often even more ill defined than usual and time is of the essence if you are to get in your bid by the deadline. There is usually no time for the niceties of ‘proper’ work products; the end game is to create physical view of the architecture which can be used for financial costing and other sizings. However you still need some level of discipline and process if you are to create something that can be understood by everyone. In other words you quickly need to come up with a set of architecture social objects.

Here then is a seven-step approach that is useful for creating social objects that can be discussed amongst architects and stakeholders. All of the social objects in what follows are stripped down versions of the work products we define in The Process of Software Architecting.

  1. Use what requirements you have and create a System Context identifying key actors (especially other systems).
  2. Try to break down what requirements into functional areas, ‘Customer Management’, ‘Facilities Management’, ‘Diary and Calendar Management’ etc.
  3. Draw an Architecture Overview showing the functional areas from 2) as subsystems/large-grain components.
  4. Use the Architecture Overview and match against known patterns to derive the technical subsystems/components. In particular this step should account for any non-functional requirements to qualify your choice of technical components.
  5. Capture Architecture Decisions as to why you chose which components you did.
  6. Create Change Cases that you can play back to the client as a way of validating the approach and further establishing what is out of scope but which the architecture can support.
  7. Validate the whole thing in an Architecture Assessment. This will be used to identify issues and risks associated with the architecture and recommend actions and mitigation strategies to address them.

Whilst I’m not suggesting the above is in any way a substitute for a full and rigorous architecture process it’s definitely better than not following any process and will at least create a basic set of those architecture social objects that can be used to have the right conversations.

On Being an Effective Architect

In a previous blog entry I talked about the key skills architects needed to develop to perform their craft which I termed the essence of being an architect. Developing this theme slightly I also think there is a process that architects need to follow if they are to be effective. Stripped down to its bare essentials this process is as shown below.

This is not meant to be a process in the strict SDLC sense of the word but more of a meta-process that applies across any SDLC. Here’s what each of the steps means and some of the artefacts that would typically be created in each step (shown in italics).

  • Envision – Above all else the architect needs to define and maintain the vision of what the system is to be. This can be in the form of one or more, free-format Architecture Overview diagrams and would certainly include having an overall System Context that defined the boundary around the system under development (SUD). The vision would also include capturing the key Architectural Decisions that have shaped the system to be the way it is and also refer to some of the key Architecture Principles that are being followed.
  • Realise -As well as having a vision of how the system will be the architect must know how to realise her vision otherwise the architecture will remain as mere paper or bits and bytes stored in a modeling tool. Realisation is about making choices of which technology will be used to implement the system. Choices considered may be captured as Architectural Decisions and issues or risks associated with making such decisions captured in a RAID Log (Risks-Assumptions-Issues-Dependencies). Technical Prototypes may also be built to prove some of the technologies being selected.
  • Influence -Above all else architects need to be able to exert the required influence to carry through their vision and the realisation of that vision. Some people would refer to this as governance however for me influence is a more subtle, background task that, if done well, is almost unnoticed but nonetheless has the desired effect. Influencing is about having and following a Stakeholder Management Plan and communicating your architecture frequently to your key stakeholders.

Each of these steps are performed many times, or even continuously, on a project. Influencing means listening as well and this may lead to changes in your vision thus starting the whole cycle again.

Adopting this approach is not guaranteed to give you perfect systems as their are lots of other factors, as well as people, that come into play during a typical project. What it will do is give you a framework that allows you to bring some level of rigor to the way you develop your architectures and work with stakeholders on a project.

A Service Based Development Process – Part 3

In Part 2 I discussed how activities in a delivery process could be created from capability patterns that are comprised of tasks and/or activities. Capability patterns are reusable patterns which can be applied across a range of delivery processes. Capability patterns have input/output work products which together define the ‘contractual boundary’ to which performers of the capability pattern must conform. In this post I’ll take a look at what these work products are and how they form part of a contract between capability patterns.

The table below shows the set of work products that are used in the process I have been describing.

Work Product Description Notation
Service Catalogue Catalogue of all services owned or used by the enterprise. May be technical or business, legacy or new/to be provided. Usually implemented as a service registry. WSDL
Service Certificate Identifies a service that meets certain quality criteria. Text
Service Portfolio Plan Describes what services will be provided, when and by whom. Text
Service Portfolio Describes the entire collection of business services that an enterprise uses or plans to use. Text
Service Model Defines the complete hierarchy of business and technical services used or planned to be used by the enterprise. Can exist at a logical and physical level. Contains one or more Service Specifications. UML
Service Specification Defines a detailed specification for each service. Can exist at a logical and physical level. Text/WSDL
Deployed Service A service deployed in the service platform. Jave, MQ etc
Deployed Service Environment A deployed service platform. N/A

This diagram shows the relationships (finish to finish dependencies) between the above work products.

Work Product Dependency Diagram

In this next diagram we can see how  what I’m referring to as ‘contractual boundaries’ can be realised using these work products which are passed between two of the capability patterns I discussed last time.

Contractual Boundaries

Activities produce one or more work products which are consumed by other activities, either in the same capability pattern, or in a different one. This means that provided the input and output work products are agreed the capability patterns can run in parallel or, if needed, at different times. So, one capability pattern, the ‘provide capability’ one in the above example can contribute to the Service Model work product which another capability pattern, the ‘manage service portfolio’ one can contribute to at a later time if need be. Having a common set of agreed work products, which are shared in a repository, becomes the key to having an effective delivery process.

A Service Based Development Process – Part 1

I have been toying around with this for quite a while. What prompted me to return to it was:

  1. With the cloud hype curve reaching its peak I feel compelled to join in, in one of the ways I know best, method, process and architecture.
  2. It allows me to further discuss the power of using an open, method framework based approach to building delivery processes in a plug-and-play kind of way.

I originally thought of calling this “A Cloud Service Based Development Process” however I think the word ‘Cloud’ is redundant. Whilst this process could be used for developing services “in the cloud” it is actually a generic process that can be used for developing services wherever they may actually be deployed. The process is based on three major components, all of which are in the public domain. All I’m doing is what architects are supposed to do, namely assemble components in new and interesting ways.

  1. Service Based Disciplines (from the CBDI), see here.
  2.  The Open Unified Process (from the OMG’s OpenUP), see here.
  3. The concept of activities and capability patterns from the Software and Systems Process Engineering  Metamodel (also from the OMG), see here.

The process emphasizes the organisational and contractual boundaries for a service-oriented enterprise (SOE) by a separation of concerns into a number of disciplines as follows: 

  • The service consumer and service provider are clearly separated as these require different skills and can be done by different teams. 
  • Enabling of SOA via the SO environment is a concern of its own (for example an ESB supports the implementation of services but does not affect their business content). 
  • Governance (manage) is integrated into everyday work. Example: negotiating the interface for a business service. 

These disciplines (concerns) are shown below.

Process phases are from OpenUP and are: 

  • Inception: Establish scope and define acceptance criteria. Identify key requirements, define candidate architecture and estimate the overall cost and schedule with detailed estimates for the elaboration phase. Identify risks and prepare the supporting environment. 
  • Elaboration: Establish a baselined architecture, produce an evolutionary prototype of production-quality components, as well as possibly one or more exploratory, throw-away prototypes to mitigate specific risks. Demonstrate that the baselined architecture will support the requirements and establish the supporting environment. 
  • Construction: Complete the analysis, design, development and testing of all required functionality. Iteratively and incrementally develop a complete product that is ready to transition to its user community. Establish that users are ready for the application to be deployed. 
  • Transition: Perform user and acceptance testing to validate the new system against user expectations. Convert operational databases. Ensure users and maintainers and ready to use the system. Perform deployment-specific engineering such as cut-over, commercial packaging and production, sales roll-out, field personnel training. Achieve stakeholder concurrence that deployment baselines are complete and user self-supportability. 

In the diagram below I show how the OpenUP phases combined with the previously mentioned disciplines can be overlayed with a deliver process for creating and deploying services.

The process is shown at the level of activities. An activity groups together one or more tasks and tasks deliver artefacts created by roles. I’ll detail more what each of these activities consist of in the next blog.
The other aspect to this diagram is that of iterations. As I’ve mentioned elsewhere I  believe the agile versus waterfall debate is essentially dead. We should deploy whatever processes make most sense for a particular project that can be accomplished in the most time efficient way possible. Using iterations usually makes sense so any process needs to allow for this as does this one. Each phase is iterative meaning that each iteration has elements of each discipline. You can view iterations as a wave which flows through the end-to-end process, early on the emphasis is on business modelling but also includes elements from the other disciplines whereas later on the emphasis shifts to operations and management even though the business process may still be being refined. 

Like I say, next time I’ll look at what each of the activities in this service based development process consists of and what artefacts they create.

Applying Architectural Tactics

The use of architectural tactics, as proposed by the Software Engineering Institute, provides a systematic way of dealing with a systems non-functional requirements (sometime referred to as the systems quality attributes or just qualities). These can be both runtime qualities such as performance, availability and security as well as non-runtime such as maintainability, portability and so on. In my experience, dealing with both functional and non-functional requirements, as well as capturing them using a suitable modeling tool is something that is not always handled very methodically. Here’s an approach that tries to enforce some architectural rigour using the Unified Modeling Language (UML) and any UML compliant modeling tool.

Architecturally, systems can be decomposed from an enterprise or system-wide view (i.e. meaning people, processes, data and IT systems), to an IT system view to a component view and finally to a sub-component view as shown going clock-wise in Figure 1. These diagrams show how an example hotel management system (something I’ve used before to illustrate some architectural principles) might eventually be decomposed into components and sub-components.

Figure 1: System Decomposition

This decomposition typically happens by considering what functionality needs to be associated with each of the system elements at different levels of decomposition. So, as shown in Figure 1 above, first we associate ‘large-grained’ functionality (e.g. we need a hotel system) at the system level and gradually break this down to finer and finer grained levels until we have attributed all functionality across all components (e.g. we need a user interface component that handles the customer management aspects of the system).

Crucially from the point of view of deployment of components we need to have decomposed the system to at least that of the sub-component level in Figure 1 so that we have a clear idea of each of the types of component (i.e. do they handle user input or manage data etc) and know how they collaborate with each other in satisfying use cases. There are a number of patterns which can be adopted for doing this. For example the model-view-controller pattern as shown in Figure 2 is a way of ascribing functionality to components in a standard way using rules for how these components collaborate. This pattern has been used for the sub-component view of Figure 1.

Figure 2: Model-View-Controller Pattern

So far we have shown how to decompose a system based on functional requirements and thinking about which components will realise those requirements. What about non-functional requirements though? Table 1 shows how non-functional requirements can be decomposed and assigned to architectural elements as they are identified. Initially non-functional requirements are stated at the whole system level but as we decompose into finer-grained architectural elements (AKA components) we can begin to think about how those elements support particular non-functional requirements also. In this way non-functional requirements get decomposed and associated with each level of system functionality. Non-functional requirements would ideally be assigned as attributes to each relevant component (preferably inside our chosen UML modelling tool) so they do not get lost or forgotten.

Table 1
System Element Non-Functional Requirement
Hotel System (i.e. including all actors and IT systems). The hotel system must allow customers to check-in 24 hours a day, 365 days a year. Note this is typically the accuracy non-functional requirements are stated at initially. Further analysis is usually needed to provide measurable values.
Hotel Management System (i.e. the hotel IT system). The hotel management system must allow the front-desk clerk to check-in a customer 24 hours a day, 365 days a year with a 99.99% availability value.
Customer Manager (i.e. a system element within the hotel’s IT system). The customer manager system element (component) must allow customer details to be created, read or updated (but not deleted) 24 hours a day, 365 days a year with a 99.99% availability value.
Customer Manager Interface (i.e. the user interface that belongs to the Customer Manager system element). The customer manager interface must allow customer details to be created, read or updated (but not deleted) 24 hours a day, 365 days a year with a 99.99% availability value.

Once it is understood what non-functional requirement each component needs to support we can apply the approach of architectural tactics proposed by the Software Engineering Institute (SEI) to determine how to handle those non-functional requirements.

An architectural tactic represents “codified knowledge” on how to satisfy non-functional requirements by applying one or more patterns or reasoning frameworks (for example queuing or scheduling theory) to the architecture. Tactics show how (the parameters of) a non-functional requirement (e.g. the required response time or availability) can be addressed through architectural decisions to achieve the desired capability.

In the example we are focusing on in Table 1 we need some tactics that allow the desired quality attribute of 99.99% availability (which corresponds to a downtime of 52 min, 34 sec per year) to be achieved by the customer manager interface. A detailed set of availability tactics can be found here but for the purposes of this example availability tactics can be categorized according to whether they address fault detection, recovery, or prevention. Here are some potential tactics for these:

  • Employing good software engineering practices for fault prevention such as code inspections, usability testing and so on to the design and implementation of the interface.
  • Deploying components on highly-available platforms which employ fault detection and recovery approaches such as system monitoring, active failover etc.
  • Developing a backup and recovery approach that allows the platform running the user interface to be replaced within the target availability times.

As this example shows, not all non-functional requirements can be realised suitably by a component alone; sometimes full-realisation can only be done when that component is placed (deployed) onto a suitable computer platform. Once we know what non-functional requirements need to be realised by what components we can then think about how to package these components together to be deployed onto the appropriate computer platform which supports those non-functional requirements (for example on a platform that will support 99.99% availability and so on). Figure 3 shows how this deployment can be modelled in UML adopting the Hot Standby Load Balancer pattern.

Figure 3: Deployment View

Here we have taken one component, the ‘Customer Manager’, and showed how it would be deployed with other components (a ‘Room Manager’ and a ‘Reservation Manager’’) that support the same non-functional requirements onto two application server nodes. A third UML element, an artefact, packages together like components via a UML «manifest» relationship. It is the artefact that actually gets placed onto the nodes. An artefact is a standard UML element that “embodies or manifests a number of model elements. The artefact owns the manifestations, each representing the utilization of a packageable element”.

So far all of this has been done at a logical level; that is there is no mention of technology. However moving from a logical level to a physical (technology dependent level) is a relatively simple step. The packaging notion of an artefact can equally be used for packaging physical components, for example in this case the three components shown in Figure 3 above could Enterprise Java components or .NET components.

This is a simple example to illustrate three main points:

  1. Architecting a system based on functional and non-functional requirements.
  2. Use of a standard notation (i.e. UML) and modelling tool.
  3. Adoption of tactics and patterns to show how a systems qualities can be achieved.

None of it rocket science but something you don’t see done much.

Is Agile Architecture an Oxymoron?

Much has been written by many great minds on agile development processes and how (or indeed whether) architecture fits into such processes. People like:

  • Scott Ambler (Architecture Envisioning: An Agile Best Practice). “Some people will tell you that you don’t need to do any initial architecture modeling at all.  However, my experience is that doing some initial architectural modeling in an agile manner offers several benefits”.
  • Mike Cohn (Succeeding with Agile). “On an architecturally complicated or risky project, the architect will need to work closely with the product owner to educate the product owner about the architectural implications of items on the product backlog”.
  • Walker Royce (Top 10 Management Principles of Agile Software Delivery) : “Reduce uncertainties by addressing architecturally significant decisions first”.
  • Andrew Johnston (Role of the Agile Architect): “In an agile development the architect has the main responsibility to consider change and complexity while the other developers focus on the next delivery”.
  • Martin Fowler (Is Design Dead?):  “XP does give us a new way to think about effective design because it has made evolutionary design a plausible strategy again”.

I’ve been contemplating how the discipline of architecture, as well as the role of the architect, fits with the agile approach to developing systems whilst reviewing the systems development lifecycle (SDLC) for a number of clients of late. In my experience most people have an “either-or” approach when it comes to SDLC’s. They either do waterfall or do agile and have some criteria which they use to decide between the two approaches on a project by project basis. Unfortunately these selection criteria are often biased toward the prejudices of the person writing the criteria and will push their favourite approach.

Rather than treating agile and waterfall as mutually exclusive I would prefer to adopt a layered approach to defining SDLC’s as shown here.

The three layers have the following characteristics:

  1. Basic Process. Assume all projects adopt the simplest approach to delivery as possible (but no simpler). For most software product development projects this will amount to an agile approach like Scrum which uses iterations (sprints) and a simplified set of roles, namely: Product Manager, Scrum Master and Team where Team is made up of people adopting multiple roles (architect, programmer, tester etc). On such projects decide up front which artefacts you want to adopt. These don’t need to be heavy-weight documents but could be contained in tools or as sketches. Here the team member performing the architect role needs to manage an “emergent architecture”. The role of architect maybe shared or not dedicated to a single individual.
  2. Complex Process. At the next level of complexity where multiple-products need to be built as part of a single program of work and all these have dependencies some level of governance usually needs to be in place that ensures everything comes together at the right time and is of a consistent level of quality. At a micro-level this can be done using a scrum-of-scrums approach where a twice or thrice weekly scrum that brings together all the Scrum Masters happens. Here the architect role is cross-product as it needs to ensure all products fit together (including maybe third party products and packages). This may still be a shared role but is more likely to be a dedicated individual. This may involve some program level checkpoints that at least ensure all iterations have created a shippable product that is ready to integrate or deploy. The architecture is not just emergent any more but may also needs to be “envisioned” up-front so everyone understands where their product fits with others.
  3. Complex Integration Process. The final level of complexity is caused when not only multiple products are being developed but also existing systems need to be incorporated which have complex (or poorly understood) interfaces. Here the role of the architect is not only cross-product but cross-system as she has to deal with the complexity of multiple systems, some of which may need to be changed by people other than the core development team. Here the level of ceremony as well as the number of artefacts to control and manage the decisions around the complexity will increase. The architecture is certainly not emergent any more but also needs to be “envisioned” up-front so everyone understands where their product fits and also what interfaces they need to work with. This is something Scott Ambler suggests happens during “iteration zero“.

Each of these layers is built on a common architectural as well as process “language” so everyone has a common understanding of terms used in the project. I’m much in agreement with Mike Cohn’s comment on his blog recently where, in reflecting on the tenth anniversary of the Agile Manifesto he says: “The next change I’d like to see (and predict will occur) over the next ten years also occurred in the OO world: We stop talking about agile” and goes on to say “Rather than “agile software development” it is just “software development”—and of course it’s agile”.

I would like to see an agile or lean based approach as the de-facto standard that only adds additional artefacts or project checkpoints as needed rather than thinking that every time we need to either adopt an agile approach or a waterfall approach.

Challenging What Architects Do

For most of this year I’ve been fortunate (sic) enough to travel around several countries (and indeed continents) delivering a series of architecture workshops to a client. These workshops aim to teach, through the use of instructor led case studies, best practice approaches to building solution architectures. The workshops take a holistic approach to developing architecture by ensuring both the application as well as infrastructure aspects are considered together and both the functional and non-functional requirements are taken into account.

An enjoyable side-effect of these workshops is I get to meet lots of different people who like to challenge the approach and test my understanding (I sometimes think I’m learning more than the workshop participants). Here’s one such challenge (paraphrased) I received recently: “The problem with architects and the work products they produce (for example models built using the Unified Modeling Language) is that the work products become an end in their own right and we can easily lose track of the fact that actually what we want to build is a system with running code on real hardware”. The clear implications behind this challenge could be described via three architecture anti-patterns:

  • Antipattern: Architecture Overkill. Symptoms: Architects create work that no one understands or cares about. Multiple artefacts with overlapping content get created that no one actually reads. They are too large, too complex and may even be delivered too late.
  • Antipattern: Ivory Tower Architecture. Symptoms: Architects are out of touch with real-world technical aspects. Instead they just sit in their ivory-towers making their pronouncements (which most of the time no one listens to).
  • Antipattern: Work Creation. Symptoms: The role of the architect becomes self-fulfilling. They create work products which only they understand and therefore need more architects to help create these work products.

Here’s my response:

  • Any architectural work product needs to have either an up-stream (e.g. to the business) or down-stream (e.g. to the implementers) justification. Each of these groups of stakeholders needs to be able to recognise how the architecture, as encompassed in the work products, addresses their particular concerns otherwise the work product really is only for its own sake. There are other stakeholders who will have interests or concerns in the architecture but I just focus on these to illustrate my point.
  • The role of the architect needs to be clearly defined and work produced around that role. The role may change as business needs etc evolve but the work created should be in tune with that role and ultimately fulfil the needs of the business. This does not, of course, mean that part of what architects  do is research new technology or try out new ideas. Indeed if they don’t do this the next point will be tricky to fulfil.
  • One of the difficulties with the role of the architect is that it can mean the architect easily becomes out of touch with the latest trends etc in programming. This can be caused by a number of things including: architects are paid too much to do programming, architects are too senior to do programming, architects are too busy to do programming etc. This particular problem is particularly endemic in many services organisations where architects may be charged out at premium rates and where the role of programming is seen as almost too “menial”. What to do? It’s vital that this is not the case. There must always be time allowed in any software architects work day to stay in touch with the stuff that gives him his reason for being.