New smart use case stereotypes for service oriented architecture

Smart use cases are a great technique for specifying standardized requirements in many types of projects. Over the past few years we have smart use cases being modeled and written in projects using Java and .NET, as you might expect, but also in Sharepoint projects, business intelligence, service oriented projects and even SAP implementations.

Stereotypes drive standardization
Standardization of your requirements can be an important driver to simplify software development and software rejuvination projects. An important addition to the smart use case modeling technique is the use of smart use case stereotypes.

These stereotypes represent often occurring patterns of requirements. Well known our the stereotypes manage, that represents a maintenance use case on some domain object, or search, which helps users to find a particular domain object, guided by search criteria. Other known stereotypes include select, master-detail, view, and report. As you might expect, these stereotypes all deal with user interaction.

Over the years however, we have gathered quite a few stereotypes, also in other fields. We have described for instance stereotypes that deal with Extraction Transformation Load (ETL) processes in data warehousing projects, such as aggregate, calculate and load.

Stereotypes not only help to put focus in modeling workshops, they also facilitate more easy project estimates, and even allow for straightforward code generation and isolated testing. Even better, once you marked the stereotypes (and put them into a UML profile), it will allow you to create much richer use case diagrams, modeling in colour.

Modeling in colour using (profiled) smart use case stereotypes

Slicing service oriented projects
More recently we have applied smart use cases to service oriented projects. These projects tend to be more complex than regular software development projects, and focus around more elaborate architectures Service oriented projects mostly include:

• Flexible front-end features. These can be either web pages, mobile phone front ends, Windows native applications or even Flash or Silverlight applications.
• Process-oriented middleware. Service oriented processes always implement business processes. These business processes or workflows are run in middleware, more specifically in an enterprise service bus (ESB), although having such a bus is not mandatory. There are many vendors supplying busses, such as IBM, SAP, Oracle and Microsoft.
• Stable services. A very important part of service oriented projects consists of the discovery and realization of services. Most often these services are implemented on top of existing legacy or packaged systems.

These slices in service oriented project all serve particular means and responsibilities. The approach to modeling smart use cases in such projects offers quite a few benefits. Next to modeling the front-end use cases, smart use cases help to model the business processes, and discover the required services.

Stereotypes for service oriented projects
In recent projects, that involved such complex service oriented architectures, we have gained a lot of experience in identifying and modeling smart use cases. This week we have analized a number of such models from different projects. This results in a set of new smart use case stereotypes, specifically targeted at the processes that run in middleware, and the encapsulation and execution of services.

One of the smart use case diagrams we’ve investigated

Next to our well-known front end smart use case stereotypes, the new service oriented stereotypes include the following: process, aggregate, dispatch, validate, read, write and inform. I will elaborate on these stereotypes in the next sections.

Stereotype process
A use case of stereotype process executes a business process. In most cases it delegates the execution of services to other use cases, often of types aggregate and dispatch.

The process use case is in the lead, and uses the post-conditions from called use cases to steer the process. This type of smart use cases is generally implemented in the enterprise service bus, such as BizTalk or SAP Xi.

Stereotype aggregate
An aggregate typed smart use cases generally is used to create a single response to a request.

The stereotype is called aggregate as these use cases trigger other smart use cases, othen of types dispatch and read, and aggregates the results from these use cases into a singel response.
Stereotype dispatch

Smart use cases of type dispatch are intended to link the aggregate or process types use cases to services that either require some mapping, or typically run outside the organization.

Mapping is often required, as the available services do not always deliver what is required by the calling use cases. The dipatch smart use cases performs the mapping (hence and forth) and relays the work to the actual service, that is often of type read or write.

Stereotype validate
Executing business processes often include specific validation services, for instance to see if an customer is allowed to add a particular product to their subscription (as in one of the projects we investigated). Smart use cases labeled with the validate stereotypes return a single answer, that can be either a yes or no response, or a single value from a limited list of possible codes.

In cases of more complex validation, for instance when match-making between different back-end or external systems is required, the validate use case relays fetching the required information to dispatch, read or even aggregate smart use cases.

Stereotype read
Smart use cases with the read stereotype are the most basic services. Here one or more domain objects or entities are returned from a back-end system. This often includes compressing a whole hierarchy (such as with many SAP systems), such as a product hierarchy into a single returning message.

Stereotype write
In reverse, write typed smart use cases will pick up such a compressed hierarchy that is input to the use case, distribute the hierarchy into the back-end systems and persist it.

Stereotype inform
During the execution of a business process, the actual user of the whole process (being the actor of the user goal level use case initiating the process) needs to be informed of the status. The actor might get an email, such as in a subscription process to acknowledge successful subscribing, or be sent a letter. Other alternatives are of course also possible.

The inform use case is responsible for merging the text of the message being sent with the actual domain objects the use case model evolves around.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com
www.smartusecase.com

Posted by Sander Hoogendoorn on February 4, 2010

Introducing our agile dashboard

The Accelerated Delivery Platform’s (ADP) Agile Dashboard is a pragmatic and publicly available tool (free) for managing project progress online. The Agile Dashboard was originally intended to manage progress for our agile projects, but these day it is used in a much broader perspective.

As the ADP Core Team receives a lot of questions about it, it’s time to present a small introduction here, including future directions.

Administrating projects
The Agile Dashboard simply administrates projects, project teams, work packages and work items. Furthermore, the Agile Dashboard allows you to define statusses your work items go through. In a typical Scrum project you will likely have ToDo, Working and Done. In a typical Smart project this will likely be New, InIteration, Working, Testing, Rework, Done, Dropped.

Managing work items
Work items such as user storie or features can either be added manually, or via a straightforward import feature that allows UML models to be imported in the Dashboard (from XMI). As Smart projects evolve around implemeting smart use cases, use cases can be marked as work items automatically. For each work item the complexity of the item needs to be established. Whem importing a UML model using smart use case stereotypes, establishing complexity is automatically handled.

The actual Dashboard
The main page in the Dashboard shows the progress of your project’s work items, or a the work items in a single work package, as shown below. This page will give you an immediate overview of the project.

When a work item changes status, e.g. when development is done, the user can drag-and-drop the work item to te next status, in this case for instance Testing. This pop-ups a next page where the user can add comments, and give an estimate for the amount of work still needed on the work item. Note that this is typically an action that is done during stand-up meetings.

Real-time burn-down
This information is used by the Agile Dashboard to recalculate the burn-down chart for the project real-time.

The freely available Agile Dashboard offers very basic functionality and does not excel in abundant charting and progress reporting. This is done to make managing progress in projects as easy as possible – albeit the Dashboard could do with some minor useability improvements.

Using the Agile Dashboard thus presents real-time project progress. It’s online availibility makes it suitable for distributed scenarios, e.g. in offshore – we e.g. manage projects with it that are executed on several locations simultaneuously in the Netherlands and India.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

P.S. There's an additional part to this post on my personal blog, that elaborates on the new features of the dashboard, to be implemented as soon as possible. Check it out at: Introducing our agile dashboard.

Posted by Sander Hoogendoorn on January 23, 2010

Beyond agile testing. Or: how to become a pro-active tester

Agile – in all it’s variations – becomes an increasingly popular process for realizing software. The roles testers and testing plays in these projects is challenging and new. Testers are no longer considered code-killers, but can play – and are expected to play – a very pro-active role in agile projects.

Although all agile process agree on the importance of testing, unfortunately many of these agile processes do not elaborate much on the specific role testers have. Testing is often referred to in unit testing, which is really a developer technique. Much less attention is given to true functional testing, performance testing, integration testing and acceptance.

Tester and developer closely collaborating

To investigate where testing and testers can play this vital new role you will need to see what it means for projects to be agile – beyond the scope of the processes available, such as XP, Scrum, Lean, Kanban and Smart. It’s about close collaboration of all different roles, finding the right unit of work (that also needs to be testable), defining short iterations, celebrating feedback and delivering fast and early.
Testing from day one

When introducting agile and testing to each orter, in short a number of questions pop up:

• What is this agile stuff anyway? To traditionally organized projects and organizations it is not self-explinatory that testing and testers will be in the project from day one.
• How does an (a)typical agile team look like? Many agile processs do not include the roles of testers, but rather stick to the traditional agile Bermuda triangle of coach-customer-developer.
• What can be the role of testers in agile projects? In my personal experience, that comes from doing projects in all kinds of processes, testers do play a vital role in almost all activities in a project. Testers can influence projects positively during software estimation (test effort is always underestimated), requirements gathering (what scenarios apply to a use case?), (test) design, and even development in agile projects. There are very good reasons for have an explicit test role in agile projects, as the somewhat more structured agile process such as Smart and DSDM emphasize.
• What’s a good unit of work in agile project when it comes to functional testing? Do we apply user stories, features or smart use cases? In my opinion, adding some structure is not too bad, especially when it comes to larger, more complicated projects such as service oriented or cloud oriented projects. User stories will be a to adhoc techniques to deal with in such projects. Features, but especially smart use cases add this little bit of structure and reuse that make life easier.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

P.S. There's an additional part to this post on my personal blog, that elaborates on a new talk I created to this subject, aimed at introduction agile to testers, and agile testing to (agile) project managers and project teams. Check it out at: Beyond agile testing

Posted by Sander Hoogendoorn on January 8, 2010

Questions on smart use cases. Part III - Stereotypes and minimal use case specifications

Smart use cases have become a fairly straightforward requirements technique that we have introduced in many different types of projects, such as Java, .NET, BI, SOA, SAP projects. Often I receive questions on how to apply smart use cases in projects, this time from Ron Kersic (Capgemini).

Ron: If you apply smart use cases and the associated stereotypes, do smart use case specifications become unnecessary? If so, this is a massive unique selling point, and should get much more attention.

Smart use cases have a low and equal granularity. This low granularity (sub-function level) guarantees that use case specifications are never very long, and in general will not consist of more than pages of text, including alternative flows.

Example smart use case specification (in this case 2 pages)

Applying smart use case stereotypes
Additionally we standardize further by adding smart use case stereotypes. There is a growing list of smart use case stereotypes available. We occasionally add new ones, for instance I would like to add a Register stereotype soon.

Using these stereotypes, the scenario’s smart use cases run become very much standardized. Think of one of the most used stereotypes Manage, that is meant to maintain (add, update, delete) single instances of a particular domain object. It is more or less the same for any application, for any domain object in any domain. Why bother writing long use case descriptions for that? So we have templated it, both for documentation purposes, and also for code generation.

During analysis workshops, where we model out smart use cases together with our customers, we tend to model as many of the use cases we can towards using the well-known stereotypes. This saves an enormous amount of time and effort in projects – which yes is also a very good thing for our customers. They too will have less work on the project, and it gets delivered sooner, and better.

Model driven estimation
Moreover, as we apply the stereotypes during these workshops, and use code generation, we simply generate out an estimate (in an Excel spread sheet) for the project at the end of a workshop, using Tobago MDA and a simple template that runs over the use cases in the model. “Dear customer, this project consists of 298 smart use case points. It will take us 14 weeks to build it.”

So, to answer the question, yes smart use cases have some BIG unique selling points, and yes, we should give them more attention.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 22, 2009

Questions on smart use cases. Part II - The smart use case life cycle

As you might have heard from me before (endlessly), smart use cases are a fairly straightforward reqirements technique that we have introduced in many different types of projects, such as Java, .NET, BI projects apply smart use cases, but also in service oriented projects, where smart use cases where used to model the front end and also the services and workflow.

At times I receive question on smart use cases in projects, this time from Belgium, by Erwin Bauwens. See Erwin’s original Dutch version below.

Erwin: In the smart use case life cycle all smart use cases go through the same stages (“New”, “In iteration”, “Working”, “Testing”, “Rework”, “Accepted”) from left to right. But, I take it, from “Rework” use cases go back to “Testing”?

And, additionally: it’s only the sub-function level use cases (or the user-goal level use cases without sub-function level use cases) that go through the cycle.

The smart use case life cycle identifies a number of stages smart use cases CAN go through. The stages proposed in Smart (the agile methodology the life cycle comes from) are not mandatory, but represent the most used stages. Some projects add stages such as “Design”, or split “Testing” into “Testing Team” and “Testing Customer”.

Having this simple staged smart use case life cycle allows projects to put up an nice dashboard – either on the wall or white board, or online using a dashboarding tool such as our Agile Dashboard, Mingle or VersionOne.

Example smart use case dashboard

In fact, ALL smart use cases go through this life cycle, not just the sub-function level ones or the user-goal level ones without additional sub-function level use cases. And indeed, coming from “Rework”, it is likely that the use case needs to go through “Testing” again.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 17, 2009

Questions on smart use cases. Part I - Estimating user-goal and sub-function level use cases

As you might have heard from me before (endlessly), smart use cases are a fairly straightforward reqirements technique that we have introduced in many different types of projects.

Of course, Java, .NET, BI projects apply smart use cases, but this year I have also been involved in service oriented projects, where smart use cases where used to model not only the front end but also the services and the workflow. Works smoothly. The developed system has been in production since December 1 and is running fine.

At times I receive questions around the use of smart use cases in projects, this time from Belgium, by Erwin Bauwens. I’ll try to translate. See Erwin’s original Dutch version of the questions below.

Erwin: Now there’s an opportunity to apply smart use cases in a project. This brings forward some questions.

When estimating the complexity of smart use cases, is an estimate only produced based on the sub-function level use cases (at lower granularity), and is the complexity of the user-goal level use case merely the aggregate of these points?

Extracting sub-function level use cases from a user-goal level use case is NOT a functional decomposition. The single user-goal level use case always performs some part of the works. At least it’s responsible for co-ordinating the whole process it implements, but in most cases it also will own part of the user interface, like in the example below where user-goal level use case Site Overview show all information on the managed site.

Thus, it has the same level of granularity as the underlying sub-function level use cases. And it is also estimated using the same scale (1..5, 8, 10) as the sub-function level use cases. The total complexity of the diagram then is the aggregate of the weights of ALL use cases in it.

To follow the presented example, use cases Site Overview is probably rated as a 3 – Average on the smart use case scale. I would estimate the total diagram at 22 smart use case points, including the 3 from the Site Overview use case.

Sander Hoogendoorn
Principal Technology Officer Capgemini
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on December 16, 2009

On identiyfing services we might need in the future but don't now right now

Earlier this week I attended the Landelijk Architectuur Congres in Nieuwegein. Besides the noteworthy percentage of attendees with mustaches, grey hair and ties, a pleasant and friendly event.

In the afternoon of the first day of the event I did a lively talk on shaping service oriented projects using smart use cases. During the talk I recevied some very peculiar feedback! Bare with me.

I addressed the possibilities smart use cases offer to model and implement new and existing business processes in smart use case diagrams. Using this approach front end, workflow and services become first-level citizens in the smart use case model, allowing not only to estimate and plan service oriented project more precise, but also to build up a solid repositry of services that are needed (and present) at a organizational (or domain) level – as proven in recent projects. Good stuff I would say. It adds to simplying such projects, instead of adding complexity

Colored smart use cases
For example, the diagram below resulted from a recent project. In this case, it combines both front end (orange use cases), workflow (purple use cases, realized in SAP XI) and services from different back end systems (green, yellow and blue use cases).

Defining unknown services?
Much to my surprise an architect in audience started shaking his head heavily during my presentation. Obviously, he didn’t agree. When I got curious and asked him about his misdemeanour he stated: “this approach doesn’t work because it only identifies services that are needed now.”

Although I think the technique allows for much more, I responded with a quick “So?”. “This is not service orientation,” the architect followed up. “This is only enterprise application integration.” It still didn’t worry me. After all, what’s in a name? But then it suddenly came out. “This approach does not allow you to defining services that you might need in the future, say in five years, but that you don’t know of right now.”

YAGNI
WTF! Was this guy serious? Guess he was. Unbelievable! This architect obviously has never of the YAGNI principle. Let me clearify what I mean by that in this particular case:

• Over-generic. Many software projects have gone under trying it make the software so generic that it will fit any future purpose. This makes writing software – service oriented or otherwise – hard to write and impossible to test. In reality, research has proven that of this generic software only 10% is actually used in future extensions of the code. That’s 90% wasted effort.
• The world changes. Moreover, trying to identify stuff that you will need in five years is a waste of time anyway. There’s no decent way to come up with requirements for software over that period. Simply because the world changes at high pace. Just imagine that you would have designed a piece software five years ago, and now need to build that. You would have missed the world wide financial crises, banks bankrupting, social media, Twitter, cloud computing, mobile possibilities, and likely even service orientation. Let alone changed legislation and government policies.
• Endless sessions. And it gets worse. Just think of the endless workshops and sessions organizations will start to organize to obtain these future requirements you don’t know of right now. I can see the PowerPoints coming out of those sessions, best printed at A0 format, full or requirements that will never be realised. But hey, everything works on a PowerPoint slide.
• YAGNI. Personally, I would prefer to design for business processes that organizations need now or in the near future, and that need to be automated now and not in five years. Thus, you focus on things that you can decide upon, and that can actually be designed. Think YAGNI.

Entering the twilight zone
So where does that leave us? To be honest, I kind of lost my cool during the presentation – although other people interpreted my response to the architect in the audience as pretty relaxed and quite polite.

But this head-shaking archtect perfectly stated what bugs me about (enterprise) architecture all together. Of course, I respect the idea that enterprise architecture focuses on the long term, and on strategy. But to actually think idea that you can endlessly embark on money-eating journey into the unknown future – easpecially given the current economy – is just not my cup of tea. It’s like entering the Twilight Zone. This is exactly why many large projects fail even before the first line of code is actually written.

It’s about adding unknown complexity to projects that are hard enough to run even without us having to investigate possible use in five years. In contrary, we should strive to simplifying our projects. Exactly what I meant to do with modeling services in smart use cases.

I know I’m generalizing this a bit, but please, dear architects, let’s focus on reality and costs. Come down from your architectural cloud and be welcome to our twilight zone!

Sander Hoogendoorn
Principal Technology Officer

Read this post (with images) at www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on November 27, 2009

Writing better software faster

Looking back on twenty years of software development, I think I must have spent most of that time on trying to improve both the quality and productivity of software development. Ever since I started to write small civil engineering calculation programs in Turbo Pascal way back in 1988 I got interested in writing better software faster. So right after I finished the second of fourteen programs in total, I started working on my first framework. It took some effort, but writing the other twelve only costed me less time than bulding the first two.

Darkening perspectives
There are good reasons for wanting to increase both productivity and quality of software development. First of all, if we may believe Gartner, we will have to increase productivity by a hundred-fold to meet up with current demands for software. That’s not going to be easy. Furthermore, technology is becoming increasingly complex. Only ten years ago, I was able to work both in projects using Microsoft technlogy as well as projects that were build on Java technology. Nowadays, I’ve let go of the idea that I still have an overview of either one of them. It is hard enough being able to work with some of the frameworks. A Silverlight expert is not likely knowledgable on NHibernate. Being able to work with Spring does not guarantee foxy user interfaces.

Despite this darkening perspectice, there’s techniques and technologies that might keep you from drowning, by simply raising the level of abstraction of software development. Good frameworks do this. And, of course Smart use cases do that. Modeling in UML might even do so. And then there’s model driven development, or model driven architecture, or model driven engineering or domain specific languages. Different terms that all try to use the same means to the try to raise the level of abstraction in software development: generating code and sometimes even other deliverables from a model. What model? Well, any model might do the trick to be honest. Business process models, use case models, user interface models, domain models, and last but not least data models.

DIY
I think it was about eight years ago that I was helping a customer – a chain of DIY shops - set up a domain model for a help desk application, and after that build the application in .NET. Having modeled all classes, their properties and their services, I was typing over the same classes, properties and services in C#. So, as any decent developer would do :-) I wrote a code generator. That helped.

Although empirical, I suspect that most code generation efforts have a similar DIY start up history. That’s probably why there’s so many approaches to generating code. Some of them really high-brow approaches that try to squeeze everything and the kitchen sink into the model and then try to generate the whole application, from front end to back end. Others really at the low and of the spectrum, making an effort re-engineering the database and generation create-read-update-delete screen from it – probably in a really flat two-tier architecture.

Where’s the value?
Last week I was invited to to be on a panel on model driven development, including leading academic and commercial experts. Each of the members of the panel had to introduce their position and approach briefly to the audience. This noteworthy event learned me two things. One: never to allow four enthusiastic innovators talk freely on their innovations. The word briefly isn’t in their vocabulary. And two: coming from a more or less commercial background, you don’t have a clue to what model driven development does in the academic world. A whole new spectrum of approaches met my eye.

So where’s the value in all this? Well, following a good old Scrum adagium, I would have to say: that depends. To put it bluntly: generating code from a model can be very rewarding for projects, but I’ve also seen projects that make quite a mess out of it. Being successful at model driven development, in my honest opinion, requires some prerequisites to be met:

• Embed approach. Successful model driven development projects I’ve seen all share a single principle: it’s all in a days work. Model driven development should never be made a goal as such. It’s a means to make better software faster. Thus, modeling and code generation should be embedded as normal work in the project. Nog guru’s, no heroes.


• Allow changes. Projects go through changes. That’s a fact of life. Model driven development should not be the blocking technology, but rather should stimulate creativity, by allowing project to quickly show what effect new or changing requirements have on the software being produced.


• Stabilize architecture. Knowing where to generate what code is key for your projects. What aspects do I need to model and what code comes out of it. I stronlgy recommend to set up a stable software architecture model, with accompanying frameworks, and try this out, before elaborating on code generation.


• Be pragmatic. My general advice in life also holds for model driven development. Be pragmatic. Don’t overdo it. Don’t try the squeeze the whole world in the model. It’s not necessarily a bad thing to still have to write some code if it is easier to write than to model.

Sceptisism?
Having that said, and having seen projects that have very successfully applied a form of model driven development, why is there still so much sceptisim about this? After all, it can certainly help to deliver high quality software at high productivity?

Well, witnessing both highly successful and extremely failing model driven development projects recently, I have given this some consideration. I would say that the vast majority of model driven development projects that are in the middle of the spectrum of approaches just run smoothly, being fairly practical. But it’s the failing projects at either end of the spectrum that draw all the attention. Highly complex, slow moving, rigor projects on the one end, and the oversimplified, bug-absorbing, underestimated projects on the other end. It’s just no fun at either end.

No Jedi knight
And this is where our white paper comes in. Being titled Delivering on the promise. Pragmatic model driven development with smart use cases and domain-driven design, it demonstrates a highly practical approach – right in the middle of the spectrum. We model requirements using smart use cases, model the domain in class diagrams, combine the two, run it through our code generators (called Tobago MDA) and we’re set to go. We can make better software faster.

Can life be this simple? It certainly can. We have applied this approach to all kinds of projects, and generated code in a diversity of languages, all contributing to reaching higher quality and higher productivity, from Visual Basic to C#, from PHP and Java to Cobol dialects. To leave you with some final words of wisdom (ahum). If there’s one thing I’ve learned from being successful at model driven development, it’s this: make modeling and generating code a part of everyday practice in your project. There’s no such thing as a “model driven development project”. There’s only software development projects. You don’t have to be a true Jedi knight to be able to steer the code generator. It’s just all in a days work.

Download the white paper from the Capgemini website at Pragmatic model driven development.

Sander Hoogendoorn
Principal Technology Officer
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on September 10, 2009

Are user stories an alternative to (smart) use cases?

About a week ago, someone (from the UK) send me an email with the following question:

I like your article on Smart Use Cases. Before I throw my hat in the ring on this, could you tell me your feelings on User Stories as an alternative to Use Cases?

To be quite honest, I’m not much in favor of using user stories. In my honest opinion, user stories are too unstructured to serve as a good unit of work in larger projects – although some projects actually benefit from having unstructured requirements. However, in my opinion regular use cases are unsuited to, as they differ too much in both complexity and granularity. For example, I have seen projects where implementing a single use case took several months. Not really handy in projects that are based on two week iterations, such as the agile projects we run.

We have therefore proposed our smart use cases – and have been using them for the last ten years, with ever growing success. Smart use cases are smaller, equally granular and more structured, but do not require much more elaboration than user stories. When it comes to larger projects, including for instance business intelligence and service oriented projects, smart use cases much better reflect the mapping from business processes and workflows to software development.

Furthermore, to work on user stories in iterations in agile projects (or sprints if you prefer), the user stories are broken down into tasks. Unfortunately, since user stories can be very different in nature, so are the tasks. When we use smart use cases, the tasks are always similar, that is, we design, test design, build (and quite often generate), test, rework, accept or drop them. In most projects in an almost daily cycle (the Smart product cycle). The agile dashboard in our projects actually reflect these solid tasks, which makes planning them easier.

Another argument for preferring smart use cases over user stories, is that they can be estimated on an abstract scale rather than in hours. Important is that this estimation process is repeatable, and can be done by anyone (of course with some knowledge).

Over the years we have created a rich vocabulary of smart use case types, including types for maintenance, reporting, services, BI and file exchange. Applying this vocabulary (in stereotypes) allows for fast but structured requirements modeling.

Some links can help you further investigating this material:

• Smart use cases versus regular use cases. http://www.accelerateddeliveryplatform.com/SmartUseCase
• The agile Smart process. http://www.accelerateddeliveryplatform.com/SmartLifecycle
• Estimating smart use cases. http://www.accelerateddeliveryplatform.com/SmartEstimation
• Smart product life cycle. http://www.accelerateddeliveryplatform.com/SmartUseCaseCycle
• Smart use case stereotypes. http://www.accelerateddeliveryplatform.com/stereotypes

Love to hear your feedback and opinions.

Sander Hoogendoorn
Principal Technology Officer NL
twitter.com/aahoogendoorn

Posted by Sander Hoogendoorn on June 28, 2009

Delivering products in agile (Smart) projects

In most cases where a form of agile software development is applied, projects are challenged with difficult issues, such as a swaggering scope, unclear and incomplete requirements, unstable software architecture, are quickly approaching dead lines. Within these strict boundaries projects try to deliver high quality software at high productivity - or velocity. This is not an easy challenge.

Delivering just what is needed
Therefore, during agile projects there is high pressure on delivering just that what is needed. The emphasis of an agile project team is on creating just the right (amount of) software, and minimizing the amount of work spent on other products, such as documentation and work in the peripherals of the project. Although many project teams seem to call on this principle to actually not deliver any documentation, the fine art of being agile is delivering just-enough, just-right documentation.

In our experience, a lot of time is spent in agile projects on deciding what's enough, and moreover, what's right. Especially when projects are executed in complex organizations, and applying complex (service or cloud oriented) technology, finding the right level of documentation is actually a quite intriguing task, and goes beyond writing (structured or not) user stories.

Delivering products
In the vision of the agile process Smart (and other agile processes, including Scrum and feature driven development) projects should focus on delivering products, not just on performing activities. Please note that working software is not the only product a project delivers.

The agile process Smart suggests to deliver a number of products that will guide projects through these just-enough, just-right in a more standardized fashion. In essence, Smart projects are product driven, as are other agile processes such as Scrum, FDD and OpenUP. During each of the iterations in a Smart project, the team delivers a number of working products. These products fall into a few categories:

• Project deliverables. In each of the different types of iterations Smart suggests to deliver additional products that will help drive your project to success. These products might include well know deliverables such as a project proposal and non-functional requirements. Although none of these product are actually mandatory in a Smart project, they are considered good practice, and over the years have been produced in numerous Smart projects. Using these standard suggested products, organizations get an even better grip on the projects at hand. Note that emphasizing the existence of other project deliverables besides working software (in smart use cases) does not mean projects become less agile.
• Smart use cases. The body of products delivered in a project is formed by smart use cases, small discrete pieces of functionality that drive development. Smart use cases is a technique to rapidly model your functional requirements at a equal-granular level. Smart use case supply a small and very useful unit of work, and furthermore serve as the main unit of estimation, planning, but also drive realization, testing and even delivery.

All products, both project deliverables and smart use cases run through (a subset of) the product life cycle.

Project deliverables
All products that need to be delivered to run your projects and do not deal with implementing functionality directly (which moreover is partitioned in smart use cases) are referred to as project deliverables. Most of these project deliverables are one-off, although they might be maintained and updated during the project, such as the project's software architecture and the domain model. Any Smart project needs to decide which of the suggested project deliverables will be produced, and in which iterations.

An easy way to achieve the optimal set of project deliverables is to add the products suggested by Smart to the project's backlog during the different iterations defined in the Smart process. Or alternatively, add them to the backlog at the start of the project. This allows project teams to add them to the list of products to deliver in any of the iterations.

For example, during the first Propose iteration the smart use case model, a project estimate and a project proposal can be added to the iteration backlog. At the start of the Scope iteration, the software architecture and reusable services might be added. At the start of each of the iterations, the customer and the project team decide which of these products will be picked up and produced.

Examples of project deliverables
Some examples of project deliverables are:

• Project proposal. A first proposal covering the primary scope of the project and a first cut estimate is produced as a the final project deliverable for Propose iterations.
• Project plan. The project plan describes the project approach, the stakeholders and goals, business case, risks, resources, timelines and project estimate. The project plan is mostly delivered at the end of Scope iterations.
• Iteration plan. For each iteration a very brief iteration plan might be documented, sometimes not longer than a single page. This iteration plan describes the list of products (mostly smart use cases) to be implemented and resources needed. In most projects, iteration plans result from the planning session at the start of each iteration.
• Software architecture. Document describing the architecture for the project. Ideally, your software architecture is an instance of a reusable reference architecture that might be in place at a organization. Having a reference architecture will allow a project team to focus only on deviations. This saves valuable time and effort.
• Business process model. Most projects implement one or more business processes, and a number of additional functions that support these processes. Smart use cases can be derived from these business processes, independent of how these processes are modeled. The business process model is either obtained from the organization, or created during the project.
• Smart use case model. The smart use case model is leading in Smart project. It delivers software estimates, and presents an overview of the scope of the project.
• Domain model. Most custom software development projects will have a model describing the business domain and business services. Sometimes this model stems from previous version of the software, for instance modeled in a data model. In service oriented projects, the domain model in most cases overlays the services called in different back end systems, as forms the basis for a service consuming front end.

Again, in most cases these deliverables are one-off. They are created once during a project, but quite often maintained or updated later on, think of the domain model and a reusable services model.

Smart use cases
Most of the work in delivering working software in iterations that are of type Realize or 'Finalize'' relates to the realization of smart use cases. The smart use case has become our main unit of work. They are implemented following the product life cycle. The work on each individual smart use case includes analysis, design, test design, build, test and acceptance. This work is always visualized using an agile dashboard, either using post-its on a wall, or using an (online) automated dashboard.

Sander Hoogendoorn
Agile evangelist

Read more
You will find more information on these subjects at: www.accelerateddeliveryplatform and www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on May 11, 2009

The iteration cycle of an agile Smart project

Smart is an easy to implement agile process, that is essentially smart use cases driven. Although Smart mixes very well with other agile processes and techniques, such as Scrum, XP and FDD, the process that takes a project through a little more ceremony than you might expect from a number of other agile processes. For instance, Smart suggests a number of products to add to the backlog, that deal with managing the project environment, such as project plans, estimates, or even setting up the development environment. Moreover, Smart is based on the use of smart use cases, where Scrum and XP on less standardized and less structured (user) stories.

The Smart iteration cycle
Each type of iteration in a Smart project is concerned with delivering a (number of) product(s), not necessarily all code. During early iterations, suggested products could for instance deal with setting up the project, determining the scope, creating a project estimate, or something trivial such as creating a straightforward project plan.

Further down the road, most of the products delivered in iterations will be (designed, built, and tested) accepted use cases. Depending on the type of products to be delivered, the iterations are branded as Propose, Scope, Realize, Finalize, and when the software is put into application maintenance iterations are usually typed as Manage.

There is one thing all these types of iteration have in common. All iterations in a Smart project follow the same procedure. This procedure is called the smart iteration cycle. The smart iteration cycle is split up into three different parts. These three parts in an iteration are known as Plan, Build and Evaluate.

Plan
Planning your iterations is essential. During the iteration kick-off the customer and the project team decide on which products will be delivered during the upcoming iteration. How many products can actually be delivered depend on the team velocity and the estimated effort that is required to realize the individual products.

A small but pragmatic iteration plan is usually set up, which also defines the acceptance criteria for each of the products to deliver - in other word defining done for each of the products.

Smart projects focus on implementing smart use cases ,delivering working software. One of the main reasons for applying these is that the approach to modeling smart use cases is standardized an thus it is easier to define done. Furthermore, it is also easier to estimate the effort required to realize these (quite often stereotyped) smart use cases.

Build
During the Build part of any iteration, the products will be realized to meet with the done criteria. To be able to monitor a project’s progress we realize products following a simple product life cycle, which is easy to visualize in a project dashboard. Although there are several possible variations of this product life cycle, it is always executed on a daily basis, starting with define what still needs to be done to finalize the product during the daily stand-up meeting.

When it comes to implementing smart use cases, the product life cycle defines brief analysis, design, coding, test and accepting the planned smart use cases. This is in high accordance with YAGNI principles, as elaboration of the individual smart use cases takes as little time as possible before a particular use case is actually built and tested.

Evaluate
Each iteration is rounded up and evaluated. This is quite an important moment in the iteration. Basically we evaluate the speed at which the project is travelling, the products that are delivered and how well the process that is applied suits the project.

Measuring in abstract points, it is quite easy to evaluate the current speed of a project, given the number of iteration spent and the number of points realized. This results in the iteration velocity, the average number of points realized during one iteration. Projecting the iteration velocity on the number of points still left in the project, will present you with an estimated end date. This data is used the steer the project.

Each of the products delivered is briefly considered during evaluation. The project’s progress can often easily be demonstrated with a short demo.

Although Smart projects are highly standardized, we see that it must fit in with the customer organization, or can be optimized to deliver a more precise to what is actually needed, or work even more efficiently. During evaluation adjustment to the process are discussed, and will be implemented during the upcoming iteration.

Sander Hoogendoorn
See for more information (and images): www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on April 22, 2009

Some tips on distributed agile projects

I started using Twitter – no, this is not another post about Twitter – to investigate if it could help me innovate. And guess what, it kind of helps. Since I started writing tweets around the agile SAP SOA project I’m coaching, people started showing interest, and asking questions.

Last week I received to following email:

Hi Sander,
I have been following you work during the last days, and mainly the SCRUM SAP stuff, and the work done is really great.
I am planning to start SCRUM and apply in the company. I will lead a group of 5-7 people but I have a limitation, my developers will be in Argentina, me and 1 domain expert are in Costa Rica, Another one in UK and in future 1 more in Asia.
In the company we have lot of tools for sharing and control so I might try to mitigate those limitations, but time zones might impact and I am concerned.
i have read many discussion talking about Agile is not for remote locations but also people saying it doesn’t apply always. I know daily scrums / stand up. We will lack of face-to-face, but I hope to take advantage of webcams and many other tools.
Hope you can provide input about this situation and provide some tips on how to start and apply.
I really want to make difference and remove the old-style waterfall from managers and peers minds.

So I answered back. Some remarks first:

• You have an extremely distributed project! This makes work very difficult, not only in waterfall style projects, but in agile projects to.
• Be sure that tools will help, but they will not totally solve the problem of distributed communication. Face-to-face as you might except always beats distribution.

• We have similar experiences as you find in the literature. Especially, in applying short iterations face-2-face is always important.
• Still, an agile approach is always preferable over waterfall.

Some tips:

• Create agile awareness. Make sure the whole team is very well aware of the fact that they are in an agile project, meaning being open, share work and responsibilities.
• Organize joint kick-off. A one or two day joint kick off is preferable. Show how and why agile works, what is expected in everybody’s roles, and establish the scope for the project. After the kick-off, people will know each other, and communicate easier afterward.
• Apply standardized and small unit of work. Apply a highly standardized type of functional work item. This will save valuable discussion. (User) stories do not apply, they allow for far too much discussion and do not allow for reuse or repeatable estimates. We use smart use cases in a lot of our projects (also in agile SAP (SOA) projects), which we have standardized on a lot of aspects, such as estimation, testing, code generation. For more information please check Smart use cases.
• Apply online dashboard. In our distributed projects, we love to apply an online agile dashboard. Never forget to use the simplest tool that still does the work. ThoughtWorks’ Mingle will do, but is fairly complex but is not for free. However, there are free-of-charge alternatives. You might even use our Agile Dashboard, which is not perfect, but free-of-charge. See Agile Dashboard and Agile Dashboard 1.3.

Sander Hoogendoorn
www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on April 10, 2009

Agile software estimation. Should I apply user stories or smart use cases?

The process of creating a sound estimate for your project will take some time in any agile project. You will have to define what work to do in the project. In general this work can be split up into two parts:

• Project related work. There's project related work, such as creating a project proposal or setting up your development environment. The agile process Smart suggests a number of such project related work items to be added to your backlog.
• Functionality. And then there's the functionality related work. In processes such as Scrum and XP this work is specified in (user) stories. In the Smart process we apply smart use cases, whereas in feature driven development (FDD) features are used.

Defining work
To create an estimate for your project, you will have to establish a number, in point, hours, pizza parts or whatever, for both types of work items. For the project related work items, this is fairly simple. How much time do your need to write the project plan? How much effort goes into setting up the development environment? The hard part here is to estimate the amount of functionality the project will realize.

Estimating this requires you to write down the stories, or in Smart to model the smart use cases. This will take some time, especially when you apply user stories as your main unit of work. The problem with estimating user stories is that each new user story is a new case. As user stories are (under normal circumstances) rather unstructured, there is no easy comparison possible with similar (previous described) types of user stories. Hence, the estimate will always be rather inaccurate.

Smart estimation
Using the agile process Smart we apply a specific way of modeling use cases, and apply these as our main unit of work. We model down use cases one level deeper than you would "normally" expect to do. Alistair Cockburn refers to this level of use case as sub-function level use cases. We refer to the collection of user goal level use cases plus sub-function level use cases as "smart use cases". They are equally granular and equally complex. And thus, they can be estimated by different teams in different locations, and in different types of projects in a similar way and on the same scale. Furthermore, we have built up a list of standardized types of smart use cases (stereotypes) from all kinds of projects. Smart estimates appear to be rather reliable (and repeatable).

Estimating agile SAP
For instance, I'm currently coaching an agile SAP project (not exactly the type of project you would expect to turn agile), and we've modeled and estimated the smart use cases in about three days with the team – more or less the time you would spend in writing stories for the project. This initial investment of three days, which in Smart project is normally one of the work items advised to put on your backlog anyway, pays back soon, as the smart use cases also make up the larger part of your product backlog.

www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on April 1, 2009

Agile rigidity

One of the characteristics of most traditional – linear, waterfall – styled organizations is the extremely rigid execution of their software development projects . “Our handbook says we need to fill in this form, so that’s what we do guys.” People is these projects live by the blind assumption that whoever invented or wrote down their software development process knew what he or she was doing. So we’ll just fill in the form, even though we’re not quite sure what it is for. And even better, all projects great and small, live under the same regime. This is where six weeks, three developer projects get stuck with a steering committee, an architecture board and a quality assurance work group. We need predictability in our projects more than anything else!

But enough laughter. One of the characteristics of agile software development projects is the flexibility and adaptability to only do what is necessary in the project. No matter which agile process we apply – be it Scrum, Smart, XP, FDD, or OpenUP – we try to delay the actual work to the latest possible moment in time that we actual need to do the work. And we delay decisions in the same manner. In both cases we do this so we have maximum knowledge over what we should and should not do. Thus, for example the design for a particular piece of functionality is done just before building the software.

We know better
These two approaches appear to be totally different or even orthogonal, and yes, they are. Still a lot of organizations cling to the rigidness of their waterfall processes. In most cases this is because they think it will make their projects more predictable and repeatable. Or sometimes just because they just don’t have a clue. In most of these poor organizations, we agile people laugh at this rigidness, because we think we know better. But do we?

I would say there’s also a dark side to our laughter. In fact, with the increasing popularity of agile software development a lot of newbie agilists entering our projects I predict that the freedom and flexibility we preach will slowly shift towards more rigidity and ceremony. And I don’t mean waterfall rigidity, but agile rigidity. Again, a lot of people will think that whoever invented or wrote down their software development process knew exactly what he or she was doing. A lot of people only have a vague clue to why he or she wrote down the process.

No modeling
This agile rigidity anti-pattern is easily recognizable in projects – or even better still: in blog discussions. These days there seem to be more blog and Twitter posts on agile than there are user stories in agile projects. Let me present you with a few examples of this new agile rigidity. In the agile process Smart (see www.smartusecase.com) it is suggested to add a number of work items right from the start of the project to the project backlog that deal with determining project scope, modeling requirements (in smart use cases), creating a sound estimate for the project’s complexity, and also maybe deliver a project proposal. Having a project proposal, is not a bad thing if you want to (sell and) run the project. In my day to day life this short list of suggested work items works quite well, and moreover, is totally compatible with processes such as Scrum and FDD. We have been using this approach in agile projects for about ten years successfully.

However, much to my surprise, when I recently elaborated on this approach – in a blog post – people comments on this post and accused me of actually promoted doing waterfall disguised as agile – just because I advised a few neat deliverables other than writing code during the first iterations or sprints of my projects. Come again? Since when does being agile mean not capturing requirements. I’m not promoting the big upfront design here. Who says I have to deliver code anyway? I’ve used agile principles, techniques and best practices in numerous projects, including projects that didn’t deliver any code at all (like code audits). A backlog contains work to be done, whether that’s “check software architecture”, “investigate namespace dependencies” or “write login screen”. My burn down is still valid, and so are my estimates.

Another example? I love to have my requirements modeled out in what we call smart use cases – small use cases of similar complexity and granularity that work pretty well in filling the project backlog and determine sprint backlogs (just to use the currently most popular terminology). But, standing at a coffee machine at a customers office a Java developer came up to me, claiming that I should write user stories on post-its instead of modeling smart use cases in a UML modeling tool. And why? “Because that’s what Scrum dictates.” Eh.. “We do not do use cases in agile projects,” said my developer, long hair, goatee and all. “And besides, we do not promote modeling tools.” Come again?

Trojan rigidity
That’s agile rigidity for you. In my honest opinion, with a huge crowd of people migrating into agile software development, agile rigidity will play an increasing role in our life. And it will definitely contribute to failure in near future agile projects. Now you should note that this anti-pattern is not a threat coming from the ever dangerous outside world. This threat will be coming at you from within your own project. It’s Trojan rigidity. But there’s hope for you: just follow the correct procedures.

www.sanderhoogendoorn.com

Posted by Sander Hoogendoorn on March 27, 2009