Introducing our agile dashboard

The Accelerated Delivery Platform’s (ADP) Agile Dashboard is a pragmatic and publicly available tool (free) for managing project progress online. The Agile Dashboard was originally intended to manage progress for our agile projects, but these day it is used in a much broader perspective.

As the ADP Core Team receives a lot of questions about it, it’s time to present a small introduction here, including future directions.

Administrating projects
The Agile Dashboard simply administrates projects, project teams, work packages and work items. Furthermore, the Agile Dashboard allows you to define statusses your work items go through. In a typical Scrum project you will likely have ToDo, Working and Done. In a typical Smart project this will likely be New, InIteration, Working, Testing, Rework, Done, Dropped.

Managing work items
Work items such as user storie or features can either be added manually, or via a straightforward import feature that allows UML models to be imported in the Dashboard (from XMI). As Smart projects evolve around implemeting smart use cases, use cases can be marked as work items automatically. For each work item the complexity of the item needs to be established. Whem importing a UML model using smart use case stereotypes, establishing complexity is automatically handled.

The actual Dashboard
The main page in the Dashboard shows the progress of your project’s work items, or a the work items in a single work package, as shown below. This page will give you an immediate overview of the project.

When a work item changes status, e.g. when development is done, the user can drag-and-drop the work item to te next status, in this case for instance Testing. This pop-ups a next page where the user can add comments, and give an estimate for the amount of work still needed on the work item. Note that this is typically an action that is done during stand-up meetings.

Real-time burn-down
This information is used by the Agile Dashboard to recalculate the burn-down chart for the project real-time.

The freely available Agile Dashboard offers very basic functionality and does not excel in abundant charting and progress reporting. This is done to make managing progress in projects as easy as possible – albeit the Dashboard could do with some minor useability improvements.

Using the Agile Dashboard thus presents real-time project progress. It’s online availibility makes it suitable for distributed scenarios, e.g. in offshore – we e.g. manage projects with it that are executed on several locations simultaneuously in the Netherlands and India.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

P.S. There's an additional part to this post on my personal blog, that elaborates on the new features of the dashboard, to be implemented as soon as possible. Check it out at: Introducing our agile dashboard.

Posted by Sander Hoogendoorn on January 23, 2010

Beyond agile testing. Or: how to become a pro-active tester

Agile – in all it’s variations – becomes an increasingly popular process for realizing software. The roles testers and testing plays in these projects is challenging and new. Testers are no longer considered code-killers, but can play – and are expected to play – a very pro-active role in agile projects.

Although all agile process agree on the importance of testing, unfortunately many of these agile processes do not elaborate much on the specific role testers have. Testing is often referred to in unit testing, which is really a developer technique. Much less attention is given to true functional testing, performance testing, integration testing and acceptance.

Tester and developer closely collaborating

To investigate where testing and testers can play this vital new role you will need to see what it means for projects to be agile – beyond the scope of the processes available, such as XP, Scrum, Lean, Kanban and Smart. It’s about close collaboration of all different roles, finding the right unit of work (that also needs to be testable), defining short iterations, celebrating feedback and delivering fast and early.
Testing from day one

When introducting agile and testing to each orter, in short a number of questions pop up:

• What is this agile stuff anyway? To traditionally organized projects and organizations it is not self-explinatory that testing and testers will be in the project from day one.
• How does an (a)typical agile team look like? Many agile processs do not include the roles of testers, but rather stick to the traditional agile Bermuda triangle of coach-customer-developer.
• What can be the role of testers in agile projects? In my personal experience, that comes from doing projects in all kinds of processes, testers do play a vital role in almost all activities in a project. Testers can influence projects positively during software estimation (test effort is always underestimated), requirements gathering (what scenarios apply to a use case?), (test) design, and even development in agile projects. There are very good reasons for have an explicit test role in agile projects, as the somewhat more structured agile process such as Smart and DSDM emphasize.
• What’s a good unit of work in agile project when it comes to functional testing? Do we apply user stories, features or smart use cases? In my opinion, adding some structure is not too bad, especially when it comes to larger, more complicated projects such as service oriented or cloud oriented projects. User stories will be a to adhoc techniques to deal with in such projects. Features, but especially smart use cases add this little bit of structure and reuse that make life easier.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

P.S. There's an additional part to this post on my personal blog, that elaborates on a new talk I created to this subject, aimed at introduction agile to testers, and agile testing to (agile) project managers and project teams. Check it out at: Beyond agile testing

Posted by Sander Hoogendoorn on January 8, 2010

Avatar, reuse and model driven software development

Last week my daughter Sam (13), her friend Joey and I went to see the movie Avatar in 3D at the IMAX theatre in Rotterdam. Following up on the hype and reading reviews I just got curious. Especially after reading an interview with director James Cameron, who previously directed Aliens and Titanic.

Being more than interested in model driven software development, this interview cracked me up. James Cameron states that there’s a model for every rock, every straw, every tree on the planet Pandora, and for every piece of skin and body of the inhabitants. That’s something for you. On the one hand you might say: hey, this Avatar project took over 7 years to build, and it costs around 400 million dollar. No way should we do software development projects like that.

James Cameron: “Use the models for sequels”

But it gets more interesting. In the interview James Cameron also says that the argument that sold Avatar to studio 20 Century Fox is that these expensive and extensive models could be used for any number of sequels. To sum up why the studio put up with the length and price of the project in one simple word: it’s reuse.

Seeing is believing
I have to admit, seeing is believeing. This movie has the most incredible visual appearance I ever witnessed. And it’s all generated from the model. And that’s where I got jealous, or even a bit depressed.

There we are: enthusiastic software engineers building business software from our BPMN or UML models or our textual or visual DSL’s. Whatever abbreviation you follow, be it MDA, MDD, FMDD, or MDSD, we think we’re doing great jobs raising the level of abstraction just a little bit from hand-written code. But even this very little step forward seems to be highly complex. The technology is hard, the meta-level thinking is harder, and convincing management and customers (the studios) that this is the way to go with software development is even more challenging.

Kindergarten stuff
But in comparison to the complexity and size of producing Avatar our whole MDx venue is kindergarten stuff. We’re not creating a highly detailed blasting 3D movie experience. We are only implementing a simple business process to change your address at your local bank. Pfff. Is that all?

Jake Sully: “Ha ha, changing your address at your local bank is difficult?“

Don’t get me wrong. We will simply have to move from hand-writing applications to modeling and generating applications. Raising the level of abstraction is the only way to deal with the rapidly increasing level of complexity of technology vendors present us with, and the ever increasing demand for new software, presented on new devices. There’s just no way we can keep with this demand just by hand-writing all the code, not with all the developers in the world, not with the best-of-class frameworks at hand, not with the best agile processes and techniques in place (let alone waterfall).

Even though model driven techniques are no silver bullet either it will play an increasingly important role in software development . Our work will be around simplifying customer demand, standardizing techniques and raising our levels of abstraction, writing less code and generating more code. And for those of you not yet convinced: go see Avatar or preferably one of its likely many sequels.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 28, 2009

Questions on smart use cases. Part III - Stereotypes and minimal use case specifications

Smart use cases have become a fairly straightforward requirements technique that we have introduced in many different types of projects, such as Java, .NET, BI, SOA, SAP projects. Often I receive questions on how to apply smart use cases in projects, this time from Ron Kersic (Capgemini).

Ron: If you apply smart use cases and the associated stereotypes, do smart use case specifications become unnecessary? If so, this is a massive unique selling point, and should get much more attention.

Smart use cases have a low and equal granularity. This low granularity (sub-function level) guarantees that use case specifications are never very long, and in general will not consist of more than pages of text, including alternative flows.

Example smart use case specification (in this case 2 pages)

Applying smart use case stereotypes
Additionally we standardize further by adding smart use case stereotypes. There is a growing list of smart use case stereotypes available. We occasionally add new ones, for instance I would like to add a Register stereotype soon.

Using these stereotypes, the scenario’s smart use cases run become very much standardized. Think of one of the most used stereotypes Manage, that is meant to maintain (add, update, delete) single instances of a particular domain object. It is more or less the same for any application, for any domain object in any domain. Why bother writing long use case descriptions for that? So we have templated it, both for documentation purposes, and also for code generation.

During analysis workshops, where we model out smart use cases together with our customers, we tend to model as many of the use cases we can towards using the well-known stereotypes. This saves an enormous amount of time and effort in projects – which yes is also a very good thing for our customers. They too will have less work on the project, and it gets delivered sooner, and better.

Model driven estimation
Moreover, as we apply the stereotypes during these workshops, and use code generation, we simply generate out an estimate (in an Excel spread sheet) for the project at the end of a workshop, using Tobago MDA and a simple template that runs over the use cases in the model. “Dear customer, this project consists of 298 smart use case points. It will take us 14 weeks to build it.”

So, to answer the question, yes smart use cases have some BIG unique selling points, and yes, we should give them more attention.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 22, 2009

Questions on smart use cases. Part II - The smart use case life cycle

As you might have heard from me before (endlessly), smart use cases are a fairly straightforward reqirements technique that we have introduced in many different types of projects, such as Java, .NET, BI projects apply smart use cases, but also in service oriented projects, where smart use cases where used to model the front end and also the services and workflow.

At times I receive question on smart use cases in projects, this time from Belgium, by Erwin Bauwens. See Erwin’s original Dutch version below.

Erwin: In the smart use case life cycle all smart use cases go through the same stages (“New”, “In iteration”, “Working”, “Testing”, “Rework”, “Accepted”) from left to right. But, I take it, from “Rework” use cases go back to “Testing”?

And, additionally: it’s only the sub-function level use cases (or the user-goal level use cases without sub-function level use cases) that go through the cycle.

The smart use case life cycle identifies a number of stages smart use cases CAN go through. The stages proposed in Smart (the agile methodology the life cycle comes from) are not mandatory, but represent the most used stages. Some projects add stages such as “Design”, or split “Testing” into “Testing Team” and “Testing Customer”.

Having this simple staged smart use case life cycle allows projects to put up an nice dashboard – either on the wall or white board, or online using a dashboarding tool such as our Agile Dashboard, Mingle or VersionOne.

Example smart use case dashboard

In fact, ALL smart use cases go through this life cycle, not just the sub-function level ones or the user-goal level ones without additional sub-function level use cases. And indeed, coming from “Rework”, it is likely that the use case needs to go through “Testing” again.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 17, 2009

Does Agile mean no Processes and no Governance?

There are two generally held perceptions about agile. One, that agile is a kind of anarchist model with no processes, and second, that agile means no governance. Let’s look at the first perception that agile means no processes. We can understand how agile looks at need of processes by applying a well known principle called Occam's razor which states that ” one should not increase, beyond what is necessary, the number of entities required to explain anything”. The principle helps us to remove those concepts, variables or constructs that are not needed to explain a phenomenon. By doing this, it becomes easier to create a model which resembles reality much better and redundancy and inconsistency is removed. In essence, Occam's razor stipulates that when multiple theories are available to explain a problem, the simplest one is preferred. Nature likes simplicity. Ocaam's razor when applied to project management methods would imply that a Project should have only those processes which are required and not more. This is exactly what agile paradigm means with its use of low ceremony, just enough processes. Agile teams chose processes which are most relevant for them to meet their goals in their context. It means you do not pick set of pre-defined, rigid processes in a bunch and deploy them on a project. A process is good as long as it adds value to project; or it should be discarded or amended. So, why is to so hard to agree with this!

It is possibly due to the fact that traditional management methods have relied heavily on process standardization and execution efficiency. But it has not helped achieve predictability of results which is clear from number of IT project failures. Today's business scenario is vastly different with constant change and dynamic markets. Project teams like Organizations need to adapt quickly. Traditional methods, which are based on "execution as efficiency”, find this hard to achieve because they inherently oppose changes in system. In a Harvard business review paper published in 2008, Professor Amy Edmondson identifies a different approach to execution, called "execution as learning" which looks uncannily similar to agile thinking of software development. She calls execution as learning “a radically different organizational mind-set, one that focuses not so much on making sure a process is carried out as on helping it evolve". Agile practitioners would easily relate to her thoughts and “execution as learning” looks very similar to agile manifesto.

Second perception about agile is that there is no or very little Governance in agile methods. But the fact is that agile makes governance more effective as it puts onus on participation of all stakeholders in project. Traditional command and control governance models usually end up creating bureaucratic hurdles and illusion of control. Agile governance model on the other hand is about enabling the team and facilitating an environment where team can meet its objective without avoidable hindrances. Whole team principle is about shared vision and goals and that there are no finger pointing and putting the blame when and if things go wrong. Should project sponsor remain a distant illusive figure or rather become part of the team! Perception of agile being low on governance perhaps comes from principle of "self-organizing teams". Scott Ambler explains this principle when he says that "self organizing doesn’t mean that you are out of control and doing your own thing. It means that team members are themselves deciding how to meet the goals. But the goals they have to meet, the resources used and timeframe, are governed by organization". Agile governance he says is about keeping the baby and throwing the bathwater. And that might well summarize processes and governance perspective in agile.

Posted by Gaurav Sharma on November 16, 2009

The perfect business application

Cost is becoming an ever increasing player on the market. Total cost, not just project cost. Time-to-market is good, but time-in-market is better, so to say

In the end, there are questions that need to be answered. 70% of those questions are asked after the go-live of a project, not during
A project might be successful but maintenance might turn out to be a disaster (or anything in that grey area in between). Fill in the blanks for a project and run, a build and test, a design and build
For the 30% of the questions that get asked during the project, the same principle applies: the farther down the chain these get asked, the more costly they are

So, how can costs be minimized? Well, just by making an existing application perfect, or by creating a perfect application
What makes an application perfect? Great ease-of-use, splendid user interaction, and availability

That is achieved by a humane (sic) approach, a fault-proof approach, and a resilient approach. The word approach is used, because there are several phases in which the timeline between beginning and end of an application can be divided. Design, build, test, run, use, maintain. The perfect application should be perfect during all those phases

• The humane approach is achieved by approaching the application as if it were our own, used only by ourselves
• The fault-proof approach is achieved by approaching the application as if it were everyone else's
• The resilient approach is achieved by approaching the application as if it were everyone's application, used everywhere

A perfect application can be measured from the end, backwards to the beginning

• Is it hard to maintain? Then the technical documentation is not complete enough, or the code is not legible enough. It is not fault-proof for the maintenance phase


• Is it hard to use? Then the presentation is not intuitive enough, at least not for the users that have to work with it. It is not humane for the use phase


• Is it hard to run? Then the technical operation is not easy enough, or too costly. It is not humane nor fault-proof nor resilient for the run phase


• Is it hard to test? Then the functional design is not clear enough, or the technical design is way off to the functional design. It is not fault-proof for the test phase


• Is it hard to build? Then the technical design is not clear enough, or the functional design not clear enough in order to draw a technical design. It is not fault-proof for the build phase


• Is it hard to design? Then the business specifications are not clear enough. It is not fault-proof for the design phase

Everything starts with the business specifications. They draw the lines for the business application that serves a business goal

It seems that the overall answer is easy: every question needs an elaborate and documented answer as soon as possible, and it all starts with the business design

• For every business rule, there is a business exception. Embed that into the business specifications, so the scope is clear


• In the functional design: for every business rule, there is a functional exception, and an appropriate user interaction message. Embed that into the functional design


• In the technical design: for every functional rule, there is a technical exception, and an appropriate user interaction message. Embed that into the technical design


• In the build: for every technical rule, there is a technical exception, and an appropriate user interaction message. Embed that into the build

Is this rocket science? Certainly not. Are details of this overlooked every now and then? They sure are
Is this a handy checklist? I sure hope so.

Martijn Linssen is Enterprise Integration Architect within Capgemini. You can follow him via Twitter or join him on LinkedIn

Posted by Martijn Linssen on September 15, 2009

Writing better software faster

Looking back on twenty years of software development, I think I must have spent most of that time on trying to improve both the quality and productivity of software development. Ever since I started to write small civil engineering calculation programs in Turbo Pascal way back in 1988 I got interested in writing better software faster. So right after I finished the second of fourteen programs in total, I started working on my first framework. It took some effort, but writing the other twelve only costed me less time than bulding the first two.

Darkening perspectives
There are good reasons for wanting to increase both productivity and quality of software development. First of all, if we may believe Gartner, we will have to increase productivity by a hundred-fold to meet up with current demands for software. That’s not going to be easy. Furthermore, technology is becoming increasingly complex. Only ten years ago, I was able to work both in projects using Microsoft technlogy as well as projects that were build on Java technology. Nowadays, I’ve let go of the idea that I still have an overview of either one of them. It is hard enough being able to work with some of the frameworks. A Silverlight expert is not likely knowledgable on NHibernate. Being able to work with Spring does not guarantee foxy user interfaces.

Despite this darkening perspectice, there’s techniques and technologies that might keep you from drowning, by simply raising the level of abstraction of software development. Good frameworks do this. And, of course Smart use cases do that. Modeling in UML might even do so. And then there’s model driven development, or model driven architecture, or model driven engineering or domain specific languages. Different terms that all try to use the same means to the try to raise the level of abstraction in software development: generating code and sometimes even other deliverables from a model. What model? Well, any model might do the trick to be honest. Business process models, use case models, user interface models, domain models, and last but not least data models.

DIY
I think it was about eight years ago that I was helping a customer – a chain of DIY shops - set up a domain model for a help desk application, and after that build the application in .NET. Having modeled all classes, their properties and their services, I was typing over the same classes, properties and services in C#. So, as any decent developer would do :-) I wrote a code generator. That helped.

Although empirical, I suspect that most code generation efforts have a similar DIY start up history. That’s probably why there’s so many approaches to generating code. Some of them really high-brow approaches that try to squeeze everything and the kitchen sink into the model and then try to generate the whole application, from front end to back end. Others really at the low and of the spectrum, making an effort re-engineering the database and generation create-read-update-delete screen from it – probably in a really flat two-tier architecture.

Where’s the value?
Last week I was invited to to be on a panel on model driven development, including leading academic and commercial experts. Each of the members of the panel had to introduce their position and approach briefly to the audience. This noteworthy event learned me two things. One: never to allow four enthusiastic innovators talk freely on their innovations. The word briefly isn’t in their vocabulary. And two: coming from a more or less commercial background, you don’t have a clue to what model driven development does in the academic world. A whole new spectrum of approaches met my eye.

So where’s the value in all this? Well, following a good old Scrum adagium, I would have to say: that depends. To put it bluntly: generating code from a model can be very rewarding for projects, but I’ve also seen projects that make quite a mess out of it. Being successful at model driven development, in my honest opinion, requires some prerequisites to be met:

• Embed approach. Successful model driven development projects I’ve seen all share a single principle: it’s all in a days work. Model driven development should never be made a goal as such. It’s a means to make better software faster. Thus, modeling and code generation should be embedded as normal work in the project. Nog guru’s, no heroes.


• Allow changes. Projects go through changes. That’s a fact of life. Model driven development should not be the blocking technology, but rather should stimulate creativity, by allowing project to quickly show what effect new or changing requirements have on the software being produced.


• Stabilize architecture. Knowing where to generate what code is key for your projects. What aspects do I need to model and what code comes out of it. I stronlgy recommend to set up a stable software architecture model, with accompanying frameworks, and try this out, before elaborating on code generation.


• Be pragmatic. My general advice in life also holds for model driven development. Be pragmatic. Don’t overdo it. Don’t try the squeeze the whole world in the model. It’s not necessarily a bad thing to still have to write some code if it is easier to write than to model.

Sceptisism?
Having that said, and having seen projects that have very successfully applied a form of model driven development, why is there still so much sceptisim about this? After all, it can certainly help to deliver high quality software at high productivity?

Well, witnessing both highly successful and extremely failing model driven development projects recently, I have given this some consideration. I would say that the vast majority of model driven development projects that are in the middle of the spectrum of approaches just run smoothly, being fairly practical. But it’s the failing projects at either end of the spectrum that draw all the attention. Highly complex, slow moving, rigor projects on the one end, and the oversimplified, bug-absorbing, underestimated projects on the other end. It’s just no fun at either end.

No Jedi knight
And this is where our white paper comes in. Being titled Delivering on the promise. Pragmatic model driven development with smart use cases and domain-driven design, it demonstrates a highly practical approach – right in the middle of the spectrum. We model requirements using smart use cases, model the domain in class diagrams, combine the two, run it through our code generators (called Tobago MDA) and we’re set to go. We can make better software faster.

Can life be this simple? It certainly can. We have applied this approach to all kinds of projects, and generated code in a diversity of languages, all contributing to reaching higher quality and higher productivity, from Visual Basic to C#, from PHP and Java to Cobol dialects. To leave you with some final words of wisdom (ahum). If there’s one thing I’ve learned from being successful at model driven development, it’s this: make modeling and generating code a part of everyday practice in your project. There’s no such thing as a “model driven development project”. There’s only software development projects. You don’t have to be a true Jedi knight to be able to steer the code generator. It’s just all in a days work.

Download the white paper from the Capgemini website at Pragmatic model driven development.

Sander Hoogendoorn
Principal Technology Officer
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on September 10, 2009

Creative plumbing. Or: rethinking the construction metaphor for software development

For as long as I’ve been in this business, I’ve heard a lot of manager, project managers, architects and other non-coders compare software development to construction. In this metaphor the architect creates the design and hands it over to the contractor, who does the work. In this metaphor the creative parts ends with the architect handing over the design to the builders. Now doesn’t that sound like a software development project to you?

City planning and blueprints
There is a lot of terminology in software development originating from this metaphor. Never heard an architect talking about the systems landscape, or that architecture is like city planning? And where did you think the term architecture itself originates from? Or the fact that we are building software? And aren’t we still discussing software construction and layered architectures? Did you know that in SAP projects people still refer to the design as a blueprint.

At first sight this metaphor makes sense. However, what has worried me about this metaphor over the past twenty years is that the creative part ends after the design is delivered by the architect. No pretty sight for developers eyes. In construction, this seems to be true. It’s the main reason construction is done by low educated workers against equally low wages.
Ignoring creativity

Converting this metaphor to software development, as a consequence, when the software architecture and design is done well, coding could be done elsewhere in the world by low educated people much cheaper than we are. This is where I highly dislike the construction metaphor. It considers coding as non-creative and repetitive work. It simply ignores creativity.

Moreover, it also surpasses the fact that, as agile projects prove, the quality of software improves at high pace when there’s regular contact with the customer during development, much like the idea that during construction of your new house you regularly visit the construction site and pass comments to the contractor.

So this is where it all comes down to. The idea that an architect sits at his desk and actually design the software at his desk, after having consulted the customer. This is were creativity is ends, and building the software begins, according to plan.

From this crippled metaphor traditional methodologists have fantasized the living daylights out of us poor developers. We were made to believe that software development is a craft rather than creative work. If the architecture is set out correct and presented nicely, building the software is low skilled, repetitive work. And hence, it can easily be outsourced, offshored and done by low educated third world software developers.

So if all this would be true, and construction is a good metaphor for software development, why is it not all projects are delivered successfully, on-time, and on-budget?

Watching plumbers
I’ll tell you why. Last week my new house was delivered by my contractor. Since then, all of the sudden, I’m surrounded with lots of activity in and around the house. This activity falls in the fore mentioned category of low skilled, non-creative work. The house is crowded with plumbers, carpenters, friends with DIY capacities, gardeners. Some of them build closets, others construct my bathroom, whilst others lay floors or grass. And me? I’m constructing a attic in the garage with a friend.

When I was watching our plumbers reconstruct my bathroom, it occurred to me. After over twenty years in this business, I finally saw the light. Having opposed the house building metaphor for software development for so long, it’s time to see things through. Time for a little re-construction.

The plumbers in my bathroom, both in their early fifties actually quite enjoy their work. As I enjoyed constructing an attic in my garage. They were singing and making (lousy) jokes. Why is it, I wondered, that these middle aged, low educated construction workers whistle and make jokes during this work, that they’ve probably been doing since they left high school. Why?

I’ll tell you. It’s creative work. They try to make the water taps fit in the wall of my bathroom. It’s a puzzle. Drill bits and holes in my wall, fitting the taps again. It’s even fun watching. It’s actually quite like I would address software development. Build it, try it out, test it, and possibly refactor the code. The plumbers actually work quite agile. During the process they ask me where I want the tap, where the shower should be, or they propose suggestions on the tiling plan. Although the bathroom isn’t finished yet, I’m quite sure it will work out fine.

Applying patterns
And moreover, the plumbers work is bound by the bathroom architecture we’ve laid out, and even more by patterns, just like developing software is bound by software architecture and by design patterns. The plumbers know how high the shower regularly sits on the wall, how many centimeters they will have to drill in the wall to make the tap fit, just like I will use some implementation of a model-view-controller to implement the user interface, and domain driven design to construct the domain layer of my application.

And you know what, to round up my argument, they make errors too, just like we software developers do. And yes, they make repairs, like we do. It’s a bit like refactoring. They try to create a solution, but if it doesn’t work, they’ll fix it, and try something else. Just like I would refactor my code, if it doesn’t perform.

But most of all, there’s one simple thing that binds us software developers to gardeners, plumbers, and carpenters. So much for the construction metaphor. No doubt about it. Just like software construction. It’s creative work. Don’t let anybody ever tell you otherwise.

Sander Hoogendoorn
Principal Technology Officer, Capgemini The Netherlands
www.sanderhoogendoorn.com
twitter.com/aahoogendoorn

Posted by Sander Hoogendoorn on July 6, 2009

Are user stories an alternative to (smart) use cases?

About a week ago, someone (from the UK) send me an email with the following question:

I like your article on Smart Use Cases. Before I throw my hat in the ring on this, could you tell me your feelings on User Stories as an alternative to Use Cases?

To be quite honest, I’m not much in favor of using user stories. In my honest opinion, user stories are too unstructured to serve as a good unit of work in larger projects – although some projects actually benefit from having unstructured requirements. However, in my opinion regular use cases are unsuited to, as they differ too much in both complexity and granularity. For example, I have seen projects where implementing a single use case took several months. Not really handy in projects that are based on two week iterations, such as the agile projects we run.

We have therefore proposed our smart use cases – and have been using them for the last ten years, with ever growing success. Smart use cases are smaller, equally granular and more structured, but do not require much more elaboration than user stories. When it comes to larger projects, including for instance business intelligence and service oriented projects, smart use cases much better reflect the mapping from business processes and workflows to software development.

Furthermore, to work on user stories in iterations in agile projects (or sprints if you prefer), the user stories are broken down into tasks. Unfortunately, since user stories can be very different in nature, so are the tasks. When we use smart use cases, the tasks are always similar, that is, we design, test design, build (and quite often generate), test, rework, accept or drop them. In most projects in an almost daily cycle (the Smart product cycle). The agile dashboard in our projects actually reflect these solid tasks, which makes planning them easier.

Another argument for preferring smart use cases over user stories, is that they can be estimated on an abstract scale rather than in hours. Important is that this estimation process is repeatable, and can be done by anyone (of course with some knowledge).

Over the years we have created a rich vocabulary of smart use case types, including types for maintenance, reporting, services, BI and file exchange. Applying this vocabulary (in stereotypes) allows for fast but structured requirements modeling.

Some links can help you further investigating this material:

• Smart use cases versus regular use cases. http://www.accelerateddeliveryplatform.com/SmartUseCase
• The agile Smart process. http://www.accelerateddeliveryplatform.com/SmartLifecycle
• Estimating smart use cases. http://www.accelerateddeliveryplatform.com/SmartEstimation
• Smart product life cycle. http://www.accelerateddeliveryplatform.com/SmartUseCaseCycle
• Smart use case stereotypes. http://www.accelerateddeliveryplatform.com/stereotypes

Love to hear your feedback and opinions.

Sander Hoogendoorn
Principal Technology Officer NL
twitter.com/aahoogendoorn

Posted by Sander Hoogendoorn on June 28, 2009

Delivering products in agile (Smart) projects

In most cases where a form of agile software development is applied, projects are challenged with difficult issues, such as a swaggering scope, unclear and incomplete requirements, unstable software architecture, are quickly approaching dead lines. Within these strict boundaries projects try to deliver high quality software at high productivity - or velocity. This is not an easy challenge.

Delivering just what is needed
Therefore, during agile projects there is high pressure on delivering just that what is needed. The emphasis of an agile project team is on creating just the right (amount of) software, and minimizing the amount of work spent on other products, such as documentation and work in the peripherals of the project. Although many project teams seem to call on this principle to actually not deliver any documentation, the fine art of being agile is delivering just-enough, just-right documentation.

In our experience, a lot of time is spent in agile projects on deciding what's enough, and moreover, what's right. Especially when projects are executed in complex organizations, and applying complex (service or cloud oriented) technology, finding the right level of documentation is actually a quite intriguing task, and goes beyond writing (structured or not) user stories.

Delivering products
In the vision of the agile process Smart (and other agile processes, including Scrum and feature driven development) projects should focus on delivering products, not just on performing activities. Please note that working software is not the only product a project delivers.

The agile process Smart suggests to deliver a number of products that will guide projects through these just-enough, just-right in a more standardized fashion. In essence, Smart projects are product driven, as are other agile processes such as Scrum, FDD and OpenUP. During each of the iterations in a Smart project, the team delivers a number of working products. These products fall into a few categories:

• Project deliverables. In each of the different types of iterations Smart suggests to deliver additional products that will help drive your project to success. These products might include well know deliverables such as a project proposal and non-functional requirements. Although none of these product are actually mandatory in a Smart project, they are considered good practice, and over the years have been produced in numerous Smart projects. Using these standard suggested products, organizations get an even better grip on the projects at hand. Note that emphasizing the existence of other project deliverables besides working software (in smart use cases) does not mean projects become less agile.
• Smart use cases. The body of products delivered in a project is formed by smart use cases, small discrete pieces of functionality that drive development. Smart use cases is a technique to rapidly model your functional requirements at a equal-granular level. Smart use case supply a small and very useful unit of work, and furthermore serve as the main unit of estimation, planning, but also drive realization, testing and even delivery.

All products, both project deliverables and smart use cases run through (a subset of) the product life cycle.

Project deliverables
All products that need to be delivered to run your projects and do not deal with implementing functionality directly (which moreover is partitioned in smart use cases) are referred to as project deliverables. Most of these project deliverables are one-off, although they might be maintained and updated during the project, such as the project's software architecture and the domain model. Any Smart project needs to decide which of the suggested project deliverables will be produced, and in which iterations.

An easy way to achieve the optimal set of project deliverables is to add the products suggested by Smart to the project's backlog during the different iterations defined in the Smart process. Or alternatively, add them to the backlog at the start of the project. This allows project teams to add them to the list of products to deliver in any of the iterations.

For example, during the first Propose iteration the smart use case model, a project estimate and a project proposal can be added to the iteration backlog. At the start of the Scope iteration, the software architecture and reusable services might be added. At the start of each of the iterations, the customer and the project team decide which of these products will be picked up and produced.

Examples of project deliverables
Some examples of project deliverables are:

• Project proposal. A first proposal covering the primary scope of the project and a first cut estimate is produced as a the final project deliverable for Propose iterations.
• Project plan. The project plan describes the project approach, the stakeholders and goals, business case, risks, resources, timelines and project estimate. The project plan is mostly delivered at the end of Scope iterations.
• Iteration plan. For each iteration a very brief iteration plan might be documented, sometimes not longer than a single page. This iteration plan describes the list of products (mostly smart use cases) to be implemented and resources needed. In most projects, iteration plans result from the planning session at the start of each iteration.
• Software architecture. Document describing the architecture for the project. Ideally, your software architecture is an instance of a reusable reference architecture that might be in place at a organization. Having a reference architecture will allow a project team to focus only on deviations. This saves valuable time and effort.
• Business process model. Most projects implement one or more business processes, and a number of additional functions that support these processes. Smart use cases can be derived from these business processes, independent of how these processes are modeled. The business process model is either obtained from the organization, or created during the project.
• Smart use case model. The smart use case model is leading in Smart project. It delivers software estimates, and presents an overview of the scope of the project.
• Domain model. Most custom software development projects will have a model describing the business domain and business services. Sometimes this model stems from previous version of the software, for instance modeled in a data model. In service oriented projects, the domain model in most cases overlays the services called in different back end systems, as forms the basis for a service consuming front end.

Again, in most cases these deliverables are one-off. They are created once during a project, but quite often maintained or updated later on, think of the domain model and a reusable services model.

Smart use cases
Most of the work in delivering working software in iterations that are of type Realize or 'Finalize'' relates to the realization of smart use cases. The smart use case has become our main unit of work. They are implemented following the product life cycle. The work on each individual smart use case includes analysis, design, test design, build, test and acceptance. This work is always visualized using an agile dashboard, either using post-its on a wall, or using an (online) automated dashboard.

Sander Hoogendoorn
Agile evangelist

Read more
You will find more information on these subjects at: www.accelerateddeliveryplatform and www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on May 11, 2009

Some tips on distributed agile projects

I started using Twitter – no, this is not another post about Twitter – to investigate if it could help me innovate. And guess what, it kind of helps. Since I started writing tweets around the agile SAP SOA project I’m coaching, people started showing interest, and asking questions.

Last week I received to following email:

Hi Sander,
I have been following you work during the last days, and mainly the SCRUM SAP stuff, and the work done is really great.
I am planning to start SCRUM and apply in the company. I will lead a group of 5-7 people but I have a limitation, my developers will be in Argentina, me and 1 domain expert are in Costa Rica, Another one in UK and in future 1 more in Asia.
In the company we have lot of tools for sharing and control so I might try to mitigate those limitations, but time zones might impact and I am concerned.
i have read many discussion talking about Agile is not for remote locations but also people saying it doesn’t apply always. I know daily scrums / stand up. We will lack of face-to-face, but I hope to take advantage of webcams and many other tools.
Hope you can provide input about this situation and provide some tips on how to start and apply.
I really want to make difference and remove the old-style waterfall from managers and peers minds.

So I answered back. Some remarks first:

• You have an extremely distributed project! This makes work very difficult, not only in waterfall style projects, but in agile projects to.
• Be sure that tools will help, but they will not totally solve the problem of distributed communication. Face-to-face as you might except always beats distribution.

• We have similar experiences as you find in the literature. Especially, in applying short iterations face-2-face is always important.
• Still, an agile approach is always preferable over waterfall.

Some tips:

• Create agile awareness. Make sure the whole team is very well aware of the fact that they are in an agile project, meaning being open, share work and responsibilities.
• Organize joint kick-off. A one or two day joint kick off is preferable. Show how and why agile works, what is expected in everybody’s roles, and establish the scope for the project. After the kick-off, people will know each other, and communicate easier afterward.
• Apply standardized and small unit of work. Apply a highly standardized type of functional work item. This will save valuable discussion. (User) stories do not apply, they allow for far too much discussion and do not allow for reuse or repeatable estimates. We use smart use cases in a lot of our projects (also in agile SAP (SOA) projects), which we have standardized on a lot of aspects, such as estimation, testing, code generation. For more information please check Smart use cases.
• Apply online dashboard. In our distributed projects, we love to apply an online agile dashboard. Never forget to use the simplest tool that still does the work. ThoughtWorks’ Mingle will do, but is fairly complex but is not for free. However, there are free-of-charge alternatives. You might even use our Agile Dashboard, which is not perfect, but free-of-charge. See Agile Dashboard and Agile Dashboard 1.3.

Sander Hoogendoorn
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on April 10, 2009

Agile rigidity

One of the characteristics of most traditional – linear, waterfall – styled organizations is the extremely rigid execution of their software development projects . “Our handbook says we need to fill in this form, so that’s what we do guys.” People is these projects live by the blind assumption that whoever invented or wrote down their software development process knew what he or she was doing. So we’ll just fill in the form, even though we’re not quite sure what it is for. And even better, all projects great and small, live under the same regime. This is where six weeks, three developer projects get stuck with a steering committee, an architecture board and a quality assurance work group. We need predictability in our projects more than anything else!

But enough laughter. One of the characteristics of agile software development projects is the flexibility and adaptability to only do what is necessary in the project. No matter which agile process we apply – be it Scrum, Smart, XP, FDD, or OpenUP – we try to delay the actual work to the latest possible moment in time that we actual need to do the work. And we delay decisions in the same manner. In both cases we do this so we have maximum knowledge over what we should and should not do. Thus, for example the design for a particular piece of functionality is done just before building the software.

We know better
These two approaches appear to be totally different or even orthogonal, and yes, they are. Still a lot of organizations cling to the rigidness of their waterfall processes. In most cases this is because they think it will make their projects more predictable and repeatable. Or sometimes just because they just don’t have a clue. In most of these poor organizations, we agile people laugh at this rigidness, because we think we know better. But do we?

I would say there’s also a dark side to our laughter. In fact, with the increasing popularity of agile software development a lot of newbie agilists entering our projects I predict that the freedom and flexibility we preach will slowly shift towards more rigidity and ceremony. And I don’t mean waterfall rigidity, but agile rigidity. Again, a lot of people will think that whoever invented or wrote down their software development process knew exactly what he or she was doing. A lot of people only have a vague clue to why he or she wrote down the process.

No modeling
This agile rigidity anti-pattern is easily recognizable in projects – or even better still: in blog discussions. These days there seem to be more blog and Twitter posts on agile than there are user stories in agile projects. Let me present you with a few examples of this new agile rigidity. In the agile process Smart (see www.smartusecase.com) it is suggested to add a number of work items right from the start of the project to the project backlog that deal with determining project scope, modeling requirements (in smart use cases), creating a sound estimate for the project’s complexity, and also maybe deliver a project proposal. Having a project proposal, is not a bad thing if you want to (sell and) run the project. In my day to day life this short list of suggested work items works quite well, and moreover, is totally compatible with processes such as Scrum and FDD. We have been using this approach in agile projects for about ten years successfully.

However, much to my surprise, when I recently elaborated on this approach – in a blog post – people comments on this post and accused me of actually promoted doing waterfall disguised as agile – just because I advised a few neat deliverables other than writing code during the first iterations or sprints of my projects. Come again? Since when does being agile mean not capturing requirements. I’m not promoting the big upfront design here. Who says I have to deliver code anyway? I’ve used agile principles, techniques and best practices in numerous projects, including projects that didn’t deliver any code at all (like code audits). A backlog contains work to be done, whether that’s “check software architecture”, “investigate namespace dependencies” or “write login screen”. My burn down is still valid, and so are my estimates.

Another example? I love to have my requirements modeled out in what we call smart use cases – small use cases of similar complexity and granularity that work pretty well in filling the project backlog and determine sprint backlogs (just to use the currently most popular terminology). But, standing at a coffee machine at a customers office a Java developer came up to me, claiming that I should write user stories on post-its instead of modeling smart use cases in a UML modeling tool. And why? “Because that’s what Scrum dictates.” Eh.. “We do not do use cases in agile projects,” said my developer, long hair, goatee and all. “And besides, we do not promote modeling tools.” Come again?

Trojan rigidity
That’s agile rigidity for you. In my honest opinion, with a huge crowd of people migrating into agile software development, agile rigidity will play an increasing role in our life. And it will definitely contribute to failure in near future agile projects. Now you should note that this anti-pattern is not a threat coming from the ever dangerous outside world. This threat will be coming at you from within your own project. It’s Trojan rigidity. But there’s hope for you: just follow the correct procedures.

www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on March 27, 2009