Capping IT Off

The invisible hand, the sequel

My previous post on The invisible hand created several interesting discussions (thanks Andy, Peter, John, Alex and Mark) and I would like to share some of it with the rest of you. Peter (Evans-Greenwood, CTO Capgemini Australia) brought up the interesting theory of swarm intelligence and I’ll quote Wikipedia to bring you quickly up to date:

“Swarm intelligence (SI) is artificial intelligence based on the collective behavior of decentralized, self-organized systems. … SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment. The agents follow very simple rules, and although there is no centralized control structure dictating how individual agents should behave, local interactions between such agents lead to the emergence of complex global behavior. Natural examples of SI include ant colonies, bird flocking, animal herding, bacterial growth, and fish schooling.”

To extend the list of “natural examples”, I would like to add “Mumbai drivers” as well. If I may. Andy (Mulholland, Global CTO Capgemini) replied on Peter’s comment: “I had not thought of swarm theory for Mumbai traffic but now that it is raised it makes perfect sense”. And it does make sense, really!

“The agents follow very simple rules”: Mumbai drivers honk in all situations to warn others
“no centralized control structure dictating how individual agents should behave”: very true, haven’t seen any speed camera or police controls
And the best one: “local interactions between such agents lead to the emergence of complex global behavior”: As John pointed out in his link that he supplied in the comments, it is the selfishness of the individual that drives a knowledge base, or applied to Mumbai traffic: “The Mumbai traffic participants are selfish in the sense that they do not want THEIR car to be damaged, thus resulting that other cars don't get damaged either.” This all leads to a situation where there are not that many accidents as you’d expect. The selfish drive for self preservation, benefits the whole system.

Alex added a link to a blog entry of the Austrian Economists that did an experiment with speed limits on a highway. On a highway where the speed limit was 50 MPH, everyone was driving 70 and the traffic was flowing rather well. As an experiment, four cars drove 50 MPH just next to each other and thus blocking the traffic, it created a huge traffic jam. If you apply this to Mumbai traffic, you can easily see that there is a much higher throughput possible in this seamlessly chaotic situation. Because they are not following the lanes, speed limits or anything else, but just take every millimeter that is possible and honk their way through traffic.

A question that I still have is whether this selfishness or self-preservation attitude from people always creates an optimum? Or better said, in what situations is this behavior desirable? Could you build your commercial software like this? What I think is that you have to distinct two main categories:

The team or group is more important than the individual: think about commercial software development and sports teams. Of course you need to have expert developers or star players, but in these two cases, the group aspect outweighs the individual aspect. A nice example is a soccer club that has only the best players of the world in the team. When you look then sometimes at their ranking and game play, you see that each individual is brilliant but there are too many egos walking on the field. 11 brilliant soccer players do not make a brilliant soccer team. The same applies to commercial software teams: if you have five superstar developers that can think of the most incredible algorithms, it is no guarantee that your project is finished in time and in budget.
The notion of “group” is actually just a side effect of similar behaving individuals, e.g. Mumbai traffic. No Mumbai trafficker wakes up and thinks “well let’s create a cozy group of mad-driving cars and rickshaws today”. The same applies to economics: a business man’s primary goal is not to create a perfect running economy, but to maximize its business’ profits. But in order to maximize its profits it has to participate in the market workings of supply and demand and thus aid in reaching an optimum. Not because he wants to, but because of selfishness. A good working economy creates a stable environment and thus a higher chance of profits.

To which category does an open source software project belong to? My first reaction is to say the second, but the more I think about it, I’m rather going for the first category. Because when you look at big successful open source projects, they are all managed by either a commercial company, a governing body, some strong community leaders or other way of centrally coordinated effort. True, you see a mixture with the second category in the sense that they often make use of these individuals that are in it out of selfishness. However selfishness in the broadest sense: pride or honor, improving own skills, community recognition or just because they really need that particular feature. Luckily the open source community is mainly driven by the last group that contribute code that was made because of something that THEY needed.

I did put the commercial software development in the first category, but would it be possible to have a commercial project that is driven by selfishness of the individual?

The invisible hand

Only a few people know that I actually love economics, more specifically macro-economics. I love the theories from economists like Pareto, Keynes, Friedman, Ricardo and of course Adam Smith. The latter one’s theory of the so-called “Invisible hand” is one of my favorites since I am quite liberal. Not as extreme as a good friend of mine who has very interesting ideas about a society where the government has almost no involvement at all, I am still a supporter of little government intervention.

Before you point out to me that societies where there is little government intervention are often far from ideal because it does not take care of the less fortunate members of the society, I acknowledge that in real life this does not work out quite as I would like to see. We do not live in a perfect world, where the market is fully transparent, where everyone is honest and is not only focused on money-making. So, often these economic theories do not seem to work quite well as expected.

Why do I tell you that? Not to change the direction of this blog into something like Freakonomics (must read by the way!), but because it relates a bit to my love for self-organizing chaos. Adam Smith’s “The wealth of nations” really focuses on the idea that free markets only appear to be chaos, but there is an “invisible hand” that guides the production and price setting to an optimum.

I am at the moment for three months in Mumbai to lead the RightShore operations from my practice and while observing the traffic, it appears to be utter chaos. People don’t use their reverse mirrors nor direction indicators, but it still “somehow magically” works. Most likely that there are a set of traffic rules, but nobody seem to follow them. How do they manage to drive without constant bumping? Honking and good faith! When you want to overtake someone from the left, you honk. When you overtake someone from the right, you honk. When you think the person before you is going too slow, you honk. When someone is getting too close, you honk. When you… anyway you get the picture.

So, without the need of an enforcing strong hand, each individual seems to take up the responsibility to warn others, by honking, of possible collisions and other dangerous situations. Isn’t this brilliant? When you take a step back from this economic blabla and look at the software industry, you see this idea of not having one central enforcing entity, but shifting responsibility to the participants coming back in similar or reduced forms. Think about distributed source code management tools like GIT or Mercurial or the open source community. The reason why it does seem to work is because they all have an interest in reaching an optimal situation where everything works out nicely, just as the drivers in Mumbai have an interest in not bumping to other cars. True, you often do have a strong “hand” in open source projects, look at Linus Torvalds at the Linux Kernel project, but there are plenty of open source examples where it is more a community-driven project without too many dominating people.

However, I do wonder to what extent this works? It seems to work for a city of 16 million people in Mumbai, but to what extend can you keep on doing this in (open source) software projects? A huge project as the Linux Kernel, would it be possible without having a leader person? And does a leader person necessarily needs to be of a strongly enforcing type? Could this actually work for a commercial software project as well, or is that doomed to fail? Please share your thoughts.

Security Access Permisisons Considered Harmful

Look at the way we set up access permissions today on, say, a windows file. We go into a form and state the exact users and files that will have permission for the file. If we want to set the same permissions on a different file, we have to go through the whole process again, manually.

At home, I have an XP PC, a Vista laptop, a Mac laptop and a network storage drive. My XP PC has 3,000,000 files on it, each file must have its permissions set correctly; my other computers are similar. I have many other applications that control access to my information, not to mention web sites. I have a Capgemini laptop, of course (actually, I’ve just remembered, I have two). Then I have my desktops at client sites; that’s without even starting to think about the clients’ infrastructure.

I have no idea whether my security setup meets the Data Protection Act, or whether it’s sufficient to meet the threats that are out there. In the unlikely event that, by some random chance, I’ve got everything right, if the threat changes, or the interpretation of the Data Protection Act changes, then I’m back to the beginning again.

This should give you some idea of the scale of the security problem we all face. The response we are forced to take, that is, setting access permissions, is rather less powerful than assembly programming. There must be a better way. Ideally, I would want the following:

The capacity to specify an access policy that applies to many different objects simultaneously.

The capacity to specify an access policy in a single place in a single style, which is then interpreted consistently by many different applications and locations.

The capacity to offload security policy setting into the cloud – security as a service.

The good news is, this is now a possibility. The new XACML v2 standard from OASIS describes how a security policy decision point can be separated from the policy enforcement points; along with protocols for them to talk to each other, and a language for specifying rich security policies.

The bad news is, support for XACML v2 in real-world applications is still pretty patchy.

I hope I have now explained why security access permissions, as currently implemented, cannot hope to solve the security problem, and justified the provocative title of this blog entry. XACML-style rich policies are, I believe, a crucial component for building a collaboration-oriented architecture and meeting the de-perimeterisation challenge.

iPodification unfuzzed

What does iPodification mean? I thought I knew, but it remains a fuzzy subject. I tried googling for definitions, and I only found these (including one of my own attempts of defining it):

the process of making technology simpler or easier to use by reducing the complexity of a device or its user interface (found on the wiktionary),
a noun to demonstrate the fact of using the whole iPod experience as a benchmark of sorts for any consumer-based products designer (source),
the almost evolutionary ongoing process of devices becoming smarter and especially smaller (and eventually leading to nothing at all),
and making "digital stuff" portable (the fuzziest one of them all...).

There does not yet exist a Wikipedia article on iPodification, making the term sort of unreal, doesn't it?. Like many others, I tend to trust Wikipedia. And that makes me tend to doubt terms that have no definition on WikiPedia (note my use of the verb 'tend'). Funny thing, really. I guess I should write the Wikipedia article myself then.

So again, what does iPodification mean? Based on the first three definitions above, I'd say that iPodifying something means that the thing gets simpler, smaller, smarter and a lot cooler than its predecessors. Also, this definition only applies to devices that do not have an Apple iPod in their ancestry. Because iPodification can also mean "creating an iPod-like device" or something that is at least as cool as one, but possibly cooler (and smaller, simpler and smarter). See?

Some time ago, I thought that iPodification is about using the iPod and its descendants as a medium for publishing content, and as such making the tails of the web longer. Oh, what the hell, the English language is already full of ambiguities, so why not add another one. As far as I am concerned, both meanings are right. The first is applied to hardware and the second to the content that you carry around on those things.

Case closed: iPodification unfuzzed.

A day in the life of a Web 2.0 citizen

I usually start the day with firing off that foxy browser to explore the internet. A good starting point to get some vibes is the Capgemini Web 2.0 overview page which gives an one-eye overview of what’s happening in the Capgemini universe. Then I usually consume some RSS feeds that gives me some food for thought. And then the day starts. While reflecting on the latest news and sipping on my cappuccino in that delicious coffee bar around the corner, I moblog an article to my personal weblog or take a picture with my cellphone and Flickr it online. Usually I’m so taken up by my mobile twittering that I forget to notice the name of the gorgeous coffee girl. Too bad, could have facebooked her and flirt with her through status updates. However I noticed that in the Netherlands that they are more into hyving, which made me create an account there as well. It’s quite hard to keep track of all your friends that are feeding you with their interesting lifes.

One of my latest gadgets to enrich my life as a techno-sexual is an iPod that made me realize that touching is so much better than scrolling. After connecting to the local wifi, I check the Youtubed adventures of my friends and total strangers, till my secretary SMSes me to ask why I didn’t RSVP to the company's beach party invitation yet. But SMS is so 2000, I actually prefer to be blackberry’d since the company pays then when I am replying.

So after all the social networking, I often check my second life where I meet up with Tim or our Spanish SL experts. Quite awesome how they managed to build a new virtual Capgemini office. It almost makes me forget to get back to my first life in the real office. Unfortunately not in sunny Span but in rainy Netherlands.

Oh, it’s time to go home. Just need to Cap IT off to our corporate technology blog…

Your mobile phone, serving the internet

Just recovered from my "gosh, I thought I've seen it all" moment when I discovered about the Aptana Jaxer server, Jonathan Mulholland (a Twitter buddy, and read here his excellent blog) pointed me to the fact that Nokia launched the Nokia Mobile Webserver initiative. Djeez, it was one of those stupid things you read over the internet when someone said "wouldn't it be cool if you could run a webserver on your mobile phone?" and now.... it's reality!

That does make me think btw that you better listen to newbies, noobs and the simple-minded people that yell things like mobile webservers and running JavaScript on the server instead of the expensive reports from "certain reputed analysts". It's much cheaper and has a high probability of becoming true apparently.

Anyway, Nokia managed to tweak the Apache httpd server (one of world's most popular webservers) to run on their Symbian operating system. Since it already has some kind of POSIX layer, they could port the httpd server to their mobile phone. Since they already ported Python to Symbian, they reused that effort and now you can run your Python Server Pages on your phone.

But the most interesting part is not the fact that they managed to do this, but more what tremendous opportunities this could bring us. So basically your device becomes an active participant of the internet. Nokia envisions a future where users can browse to your phone and for instance watch your pictures and leave messages. But think also about the following:

Stream your phone's video feed live to the world. (Could also be used by jealous husbands and wives, and when I think about it, the porn industry could be a major driver of this feature.)
Extreme live blogging/twittering: I don't think how much more live and extreme this kind of blogging can be? If you would have a temperature sensor (and I don't see why you couldn't), your blog could adapt on the fly to the weather circumstances :-)
Dating: when you enter a room or club, some wicked Personal Area Network detects all the participants and feel free to browse the profile pages from everyone around you.

But, and yes there are some "but's", think about the following:

IPv6: if all those millions of phones start hitting the internet, I think we'll run into troubles with the amount of available IPv4 addresses, so we need to switch to IPv6
Performance: when you're John Doe that wants to show off his cats to the occasional visitor, no problem. If you are some kind of porn star or just... popular, what if hundreds of people start hitting your mobile site? Can't imagine that your phone can handle that.
Security and privacy: well I don't think I really need to explain you what kind of security and privacy nightmares this can cause?
Bandwidth and costs: will UMTS be enough? And how much money will these huge data transfers will cost you (in case you want to live stream your video feed ;-) )

But let's not be so negative and let's focus on what kind of added value this could bring to a business! You can leave a message on my phone...

How reliable is your Information?

Whether you work in a managerial or operational environment, you are likely to be bombarded with information. The challenge with this is typically described as that of having access to the right facts and figures at the right time. However, I believe there is a more fundamental challenge which is sometimes overlooked: this is the challenge of interpretation.

For example, you are presented with information which, you are assured by your team, is correct and can use confidently in a certain way. This, as I hope to show, is not always the case. Worryingly, interpreting data & information is increasingly being pushed as a science rather than an art. Depending on the environment you work in, the apparent veracity of the information can be very misleading. Let me describe this using three examples:

The Good
Take the retail environment. Perhaps you own, or franchise, stores and all the organisation uses the same EPOS tills and stock codes. Everything is precise, barring the odd cross-stock mistakes. Depending on your control mechanisms, errors introduced by this type of mistake are likely to be quite small. As a manager, you can probably trust your reports. This doesn’t necessarily mean other large challenges don’t exist but, in this instance, perhaps you have an error rate of 1-2%!

The Bad
Taking the example of Local to Central Government reporting structures, it is easy to see that demands for additional statistics create issues. Typically, the information asked for is unusual; it may not be routinely collected locally in the manner required. Also, it is usually required in a timescale that demands the use of existing information rather than having the luxury to collect it properly. So the information is gathered locally, and massaged into the format demanded. The challenge here is whether the information from each source is semantically the same. Can you rely on a report detailing school absenteeism when families move between councils/schools and when figures are demanded per term and the information is collected per annum. What is the error rate then? 20-30%?

The Ugly
One police force reports an incident and identifies a person using structured reporting that uniquely identifies the role of that person as a witness. In another force, the same type of incident is captured in unstructured form. For example, witnesses may be identified in the text using a description such as "John SMITH identified the accused because of his..." If this information needs to be shared nationally in structured form, is John Smith the accused because his name is within 3 words of "the accused", or is he the witness? Will Mr Smith get a tap on the shoulder from a force that does not read the original unstructured report. In this case, the accuracy of the information could be 100% wrong! Is this inevitable and is it even possible to fix this?

In my next articles I want to explore, in greater detail, the issues behind each of these examples.

Wikimania

Warning: this blog post has an extremely high link density. When I wrote this, there was no article on "Link density" on Wikipedia yet. Feel free to create that article yourself. As of version 2.0 of the Web, everyone is allowed and entitled to do just that. In 1995 (long before the term Web 2.0 was coined), Ward Cunningham made this possible with his simplest online database that could possibly work. He was recently interviewed on the FLOSS Weekly Podcast and his original WikiWikiWeb is still online.

Probably because of its inherent simplicity, the wiki has become immensely popular. To give you an idea of this popularity, here are some crude statistics:

Google yields almost 300,000,000 for "wiki",

WikiEngines lists 147 implementations of the simplest online database,

The most popular wiki engine MediaWiki has been downloaded 16,456 times in the last 2 months (also see the figure below),

Wikipedia (based on MediaWiki) currently has 2,301,479 articles in the English language,

Wet Paint, one of the most popular wiki hosting sites, has registered over 810.000 wiki sites.

MediaWiki Download Stats

Now you might think: "So, wikis are hot. Thanks for the stats, but what's in a wiki for me?" A wiki has proven to be a simple and effective tool for building a database of collective knowledge of a group of people. Fruits of a wiki range from solutions to problems to ideas for new products. The added value of a wiki lies in the possibilities that it creates for harvesting collective intelligence (crowd wisdom) and for collaborative innovation. Tim Hyer excellently explains this. I have much trust in the wiki model too. Some time ago, I even played with the idea of applying that model to a country's civil code. Thinking of adopting a wiki yourself, but not entirely convinced yet? Then watch this 21 episode movie series about wiki adoption (Blip TV).

The wiki certainly seems the easiest way to add collaboration functionality to your website. All you need to do is choose a wiki engine, style it to your company's style guide and your done, right?. Well...not exactly. You will first need to choose a wiki engine. There are quite a lot of wiki engines that we can choose from, but fortunately, there is the Wiki Engines wiki that provides us with a nice top 10 of wiki engines. You could start by picking a wiki that you can deploy in your back office. There are, of course, more selection criteria. Selecting a wiki engine is just like selecting any other software package. The Wiki Choice tree might ease the selection process.

Once you have launched your wiki, how do you make it work? We all expect to see a flourishing community, but how are you going to convince people that they should submit their knowledge and ideas? In order to grow a wiki needs fertilization (generating interest) and cultivation (monitoring and moderation), so point out a Champion. A wiki, like any other social site, requires substantial investment before you can reap its fruits. Will there be any return on your investment? What does a wiki cost? Maybe you will find answers to these questions at the International Symposium on Wikis (WikiSym) (the WikiSym website is, of course, a wiki).

Finally, here's a list of 12 wiki's that actually work. How is your wiki doing? Are you reaping fruits or are you still investing?

BI and Financial Risk Management

I'm sure we're all acutely aware of the current financial market problems as a result of sub-prime lending and the the down turn in the economies in many countries resulting in bad debt.

We could look to better intelligence to predict the market down turns, to look for the early triggers in the economy to help spot the warning signs and to help manage the situation. Complex mathematical models that give varying levels of probablistic outcome would have shown the shifting nature of the risk. We've also had increasing levels of legislation and regulation to identify individual risk and fraud and general improvements in risk determination at the Business to Consumer level. And yet after all these things we've seen a financial service market in near melt down.

What has surprised me most in this current period of crisis is the seeming lack of understanding of corporate exposure to risk through institution to institution trading. It appears that financial institutions may have been deeply concerned with the individual risk of a customer transaction, they've not had a suitable handle on the bigger and as events have transpired, potentially organisation crippling decisions they've been taking.

How many companies are looking back at their balanced scorecards and their array of metrics and rethinking which ones are really important to survival? I hope most of them.

Social networks, RSS, email... aaargh! Information overload!

I have the feeling that I get more and more overloaded with information from my social network memberships, subscribed RSS feeds and email accounts. Sometimes I can’t even catch up anymore and prefer to just delete the whole list of feeds instead of saving them for another time, since it just won’t get any better…

There are a couple of web applications that try to find a solution for it:

Yahoo! Pipes: Yahoo! offers a solution where you can "aggregate, mainpulate and mashup content from around the web". Honestly, it's one of those cool kick-ass online tools where a genuine "wow" comes out of your mouth. Pipes offers you a solution where you can graphically put feeds together and apply filters to it. You can say that you want to have feeds from CNN that only matches your regular expression or rule and for the result, it has to fetch pictures from flickr.com. Have a look at this movie to get an idea.
Friendfeed: Friendfeed solves the problem of having to keep track of all your social networks and feeds in order to be updated about your friends' activities. Pretty cool web application that is backed by some top venture capitalists. If we apply Yahoo! Pipes to Friendfeed.... ouch! It brings us already one step closer to what we want...
NewsGator: NewsGator's approach is not unique, there are more web-based RSS feed readers out there, but I have to say that NewsGator's is pretty well-developed. But imagine that for each post that you can say whether you like it or not (a bit like what Facebook offers on the activity list) and perhaps also the reason and that your RSS feed reader learns from it.

So basically we want something that is a mix of all of the above. Here's what I would want to see: some wicked abracadabra AI engine that perfectly knows what you want and do not want to read. But there are some problems. For instance I am not interested in chickens, but I do like to be updated on initiatives like www.adopteereenkip.nl where you can adopt a chicken (I am about to adopt one) to support a good cause. But how does the rule engine know that I do want to be updated on chicken adoption initiatives and not on some weird mad chicken disease stuff?

Someone that can help me out? I'm pretty sure the chickens would really appreciate it...

*update 6.35 pm @ mumbai: "Influenced" by the comments I finally joined Twitter as well. I could resist for quite some time, but now my resistance has been broken. You can follow me on the account leeprovoost. In the meanwhile let's republish the link to Capgemini's Netvibes page with their Twitter stuff: http://tinyurl.com/34h5oq. Happy Twittering! (and now my social life is really ruined... thanks...)