2008 has been quite a year for Business Intelligence. We have seen large platform vendors buying well known BI companies. The result is that the BI market will be dominated by three maybe four players. This market, some predict, will grow with another 10% this coming year. Some analysts have stated that small independent BI software vendors will profit most from this all. They will grow faster and will be more innovative than their large competitors. It can be expected that this will give way to a renewed interest in Open Source Business Intelligence (OSBI). OSBI vendors are able to present themselves as an alternative. One of their strong arguments is that they can so help prevent a vendor lock in. Another aspect is status. Early adaptors can make good cheer with their choice for OSBI. They consider themselves winners as they have chosen to step outside the trampled down roads. Finally, this also fits in the increased interest for open source software. But are the OSBI vendors ready for the fight? Can they compete with the four mega vendors? To do so they need to have a complete BI solution (breadth) or a terrific niche solution (depth). One of the competing differentiators is the claim that OSBI is cheaper. By making the code public the vendor can lower their cost for research and development. At the same time it creates a form of shared ownership. Another advantage is that OSBI vendors do not charge you for the software (licenses). Their profit model is based on delivery of services and support. The bottom line is that the total cost of ownership (TCO) for OSBI is often lower. But the game is not won on costs alone. The fit between the user requirements and the BI solution will be just as much a deciding factor. The general impression in the market is that the functionality of OSBI runs behind. They offer some nice (additional) solutions for reporting and analysis but that’s it. They are no real alternative for their commercial counterparts. But is this true? Let’s take a short look at what the OSBI vendors came up with last year. Pentaho brought BI to the iPhone, added a meta data layer to their BI suite v1.6 and made their product compatible with Sun Solaris 10 (two OS products, 1 solution, low TCO),  Also Pentaho and Ingres have agreed on a strategic partnership for OSBI. This means shared sales and marketing activities but also an integration of their products (Pentaho is certified for the Ingres Dbase).  Pentaho also agreed on a partnership with Infobright for data integration and ETL for all MySQL customers. They have also been actively expanding in Europe. Jaspersoft came with v3.0 which includes interactive web 2.0 interfaces, drag and drop technology, a new metadata layer and improved security. They have also been cooperating with Microsoft to optimize their BI suite for windows and office. MsExcel can now fully be used as a front end tool for their JasperAnalysis data analysis server. Ingres – as mentioned before – closely work with Pentaho but also with salesforce.com offering a CRM SAAS product (Icebreaker). It is safe to conclude that OSBI vendors are strengthening their position with partnerships within the OS community and by adding new functionality or technology to their BI proposition. They are also rapidly closing the gap with the commercial BI vendors (who are often halting as they face product integration). If they can keep this pace they will have their BI products at the same level as the commercial vendors within three or four years. OSBI has the potential to become a serious alternative for the commercial solutions. (This article has been written together with Leo Cardinaals. Leo also works for Capgemini BI and is an active member of the open source community.).

Just imagine that your local hardware store gives you a hammer for each and every project you want to undertake. Paint the house? Here have a hammer. Fix the light? Here have a hammer. Build a shed? Here have a hammer. Actually, that last one made sense. In order to select the right tool you need to know a couple of basic things. One of the most important questions would be: where do you want to use it for? The same thing off course applies for BI tools. Giving a standard reporting tool to all user across the organization does not make sense. Some people might need a dashboard and other statistical data mining tools. Therefore the same question must be asked: where do you want to use it for? But who to ask this question? Do you know your Business Intelligence (BI) End-users? Are you wondering why your BI-applications are barely used? Do you think technology is enough? Do you want to want to get maximum benefit from your BI investments? During the next six months a scientific research project is conducted by a graduate student of Utrecht University to get an answer to these questions. The main goal of this research is to obtain insight into the factors influencing the fit between BI Solutions and BI End-users. Derived from the research goal(s), the following main question is formulated: “What are the major factors influencing the fit between end-users and Business Intelligence Solutions?” Results will be posted regulary at
www.intelligent-business.nl. Are you interested in helping him with his project? To validate the research project several semi-structured interviews will be conducted with BI-Practitioners (in English or Dutch). Please do not hesitate to contact him directly. Send an e-mail to Rick Tijsen or leave a comment in this post.

I was asked by an IT magazine in the Netherlands to come up with a couple of practical tips on BI service delivery. In my opinion the most important element in service delivery is creating a optimal mix of all crucial elements. My submitted three practical tips cover design, build and implementation. However it is just a start. Please join me to create a top 10 in BI service delivery.  [1] When designing a Business Intelligence environment always consider target groups and type of usage.Target groups: Some users such as managers often have the need for a high level dashboards with little detail. However other users like analysts need lot of detail and powerful analysis tools. Every target group needs their own environment in form, function and frequency (of data refresh). Usage: There should be a mix of fast (ad hoc) and structural (more long term) BI solutions and products for the end user of a BI environment. [2] Excellent BI specialists are a scarce resource but are crucial for the success of the BI environment. Do not cut back on quality. Make sure you have a proper mix of ‘hardcore’ geeks as well as business architects or BI visionaries. In order to make the costs controllable consider outsourcing parts of your BI environment. A good starting point would be technical realization (build) and application maintenance.  [3] Don’t stop when it’s finished. After the technical realization of the BI environment the real work just starts. A critical success factor for this is the BI Competence Center. This center should mix IT and business components to ensure the optimal use of BI within the organization. It should be the BICC that pro actively supports the users in their daily work. In this phase it is no longer about high level sponsorship but more about working floor thought leadership.

Times they are changing. Last year has been tumultuous for BI at best. We have seen the BI market consolidate into a few remaining mega vendors. But there was also room for niche players especially the ones focusing on speed, visualization or collaboration. So what will 2008 bring? To quote from Star Wars: “Impossible to see the future is”. But let’s give a try anyway. There are five mega trends that I foresee. Some have been around for quite some time and some are new. The first one is called: (1) Increase: Business Intelligence must be able to cope with considerable increase in both data (size) and speed (real time). The second one is (2) Integration: Business Intelligence will continue to integrate with operational processes but will also be more aligned with strategy. The integration of (un)structured data will increase. Finally, Business Intelligence will integrate with other business applications or ERP systems. Third, and my personal favorite, is (3) Insight: Business Intelligence will continue to become more intuitive (less end user involvement) showing an increase in advanced visualizations but also more analytic (more end user involvement).The fourth is (4) Interactive: Business Intelligence will be more reciprocal by nature; creating a dialogue between system and end users as well as sharing information and collaborating within performance networks. Finally, number five will be (5) Industrialization: Business Intelligence development will be characterized by a more industrialized approach using standard development frameworks opening up the possibility to outsource.  Increase will be mostly characterized by the need for real time information (partly due to the fact that BI becomes more operational). This will be supported by tools like Netezza and Teradata. They can also support the growing process of data retrieval from vast amounts of data. Integration of unstructured data (text, search, documents, agents) for example in a document warehouse (using tags) sitting next to the data warehouse will be the wave of the future. I expect Google or Attunity to take the lead here but the original BI vendors (Cognos, BusinessObjects) have also been investing in this. Although the market is more or less divided among a few remaining vendors it is still possible that we will see some more integration. This will be partly triggered by the fact that BI will be used on all levels: strategic, tactical and operational. This means BI tooling must support strategy development as well as pervasive BI (it’s everywhere). The renewed interest in open source Business Intelligence (Pentaho, Jaspersoft) is a clear counter reaction against the market domination due to vendor consolidation. Creating Insight today will be the one mega trend where I expect the most innovation to come from. It is a hot topic and fun to watch evolve. Here you will find things like intuitive intelligence (Strategy companion) or decision engines supporting your daily work (Fair Isaac). Also advanced visualization will help the end user get more grip (fractal edge) on things. Triggered by the success of “competing on analytics” or “super crunchers” we will see an rise in the use of analytics, strongly combining informatics, statistics and business knowledge. Perhaps here lies the opportunity for the long awaited promise of data mining? A clear and much appraised trend is the sharing of information with other people either inside or outside your own company to create mutual benefit. But it is more than just sharing the information it is also about sharing the reasoning behind it. How have we come to this conclusion and do you agree? It is asking feedback or contributions to your problem solving by your stake holders. All the characteristics of the web 2.0 can also apply here. This can also be done by using a trusted third party to solve your BI wishes by providing BI as a service. As BI is getting more mature by the minute I expect that we will see a standardization of best practices resulting in an industrialized approach in (standard) BI development. BI factories, often based in typical outsourcing countries, can provide high quality, low cost ready-to-use BI solutions. Some people say that after all the consolidating this will be a boring year for Business Intelligence. But if I look at these five mega trends I can’t wait. Happy new Year.   

Almost all human beings, except for the blind or visually challenged, are by nature visually orientated. Our eyes constantly monitor our surroundings searching for things that are out of the ordinary. This real time interface between eyes and the visual cortex of the brain has helped the human race survive by quickly recognizing danger. The great thing about this is that we are also able to filter out all the noise. For example, if you want to mail a letter and during the trip from your home to the mailbox you see every little detail, you probably never get there. So we ignore the stuff that we already know or looks the same all of the time. Only the things that look different get our attention. At the present time this ability to quickly spot dangerous animals or other threats is most often used for other purposes: shopping! Store owners, website builders, they all try to grab our attention using visual stimuli. Recently I was at the new Starbucks coffee shop at Schiphol airport in the Netherlands. My eyes quickly spotted a new Ready To-Go coffee tumbler. It was different from the normal ones because it had some kind of holiday season appearance. Being a collector of Starbuck Coffee Tumbler (I know this is a bit geeky…) I bought it on the spot. For Business Intelligence it is not so much different. As we are confronted with an enormity of data we try to find ways to interpret it. We try to find the information or even knowledge in this overload of data. Therefore we use Business Intelligence. But this information is presented to us in more or less the same manner, shape or form. Therefore we are quickly bored with it. Knowing this and the fact that our eyes and brains have this great ability to recognize patterns and trends we need effective visual communication of data that changes its shape or form to grab our attention when needed and allows us to quickly asses if the information presented holds some kind of pattern of interest. We need visual business intelligence that makes use of our intuitive skills. Just take a look at the work of Stephen Few (http://www.perceptualedge.com/) one of the evangelists of visual Business Intelligence or tools like Gapminder (http://www.gapminder.org/) to see what I mean.

Alles is BI [Click to open pdf]

Everything is BI, but what is BI and what is happening in the BI world with all the vendor consilidation?

In this pdf file you can find an article (I was interviewed for this) published in channelworld about the subject.

ARTICLE IS IN DUTCH

It has taken much more time than expected – but finally IBM has announced that they are buying Cognos for 58 USD per share – making this a five billion dollar transaction. IBM and Cognos have had a strategic alliance dating back to early 2006. However it looked like IBM was slow in reacting to takeovers such as Business Objects by SAP. So slow indeed, that new names came to the surface: Microsoft and Informatica. But now it is final: IBM it stays. Therefore it is safe to say that this is not a big surprise. But is this a logical step by IBM? They themselves state that this fits perfectly with their information on demand strategy - opening up information to business users. IBM traditionally had a focus on hardware and data integration – the techy stuff of datawarehousing – lacking business interaction tools like Cognos. Therefore this acquisition of front end technology for business users - Business Intelligence and Corporate Performance Management - is very compatible with the hard- and software they already had. It has made their BI portfolio more complete without a doubt. So am I 100% enthusiastic? No, is the answer. The main reason for this is that the whole “Information on demand strategy” implies having data in the first place. Where other vendors like Oracle, Microsoft or SAP have transactional platforms or CRM solutions creating transactional data – IBM starts at the point of data integration – missing the essential first step of business applications. This way they can not get their claws into the much valued business processes. They will position themselves as being independent of any database, platform or application, which off course is true from an IT point of view. However, they will very often not be a first ‘automatic’ choice (like BO will be for SAP users) for business user, meaning that they have to put a lot of energy into marketing and sales. In summary: IBM & Cognos offer a powerful, complete and compatible Business Intelligence solution which will make the IT department very happy but will the business people give a hoot?

I get most of my creative ideas when I have the time to contemplate. Often this happens under the shower or in the gym. I always thought the main reason for this was that since I had nowhere to go physically my mind could wander instead. I turns out that I am right, but only partially.  Recent studies show that the emotional state of people highly influence the way they process information. If you feel happy you are more receptive for outside stimuli. You start to see the beauty in things. Also there is an increase in creativity when you are in a good mood. The other way around, depressed people tend to focus on themselves and their problems. So now you know: I like taking showers.  Design can make a product more attractive. Attractive things create positive energy and state of mind. When asked to choose between products that basically do the same - have the same performance, price and so on – people tend to choose the attractive one. The Apple I phone is probably one of the best examples. So if attractive things work better (or at least create the feeling that they do) and stimulate creativity why is it that a lot of Business Intelligence solutions still look like spreadsheets, matrices or boring lists? I always encourage my colleagues to make their solutions super fancy sexy. Meaning that it should not only hold the correct data but at the same time it has to look good too. By presenting the information in a creative, nice looking way you increase the acceptance and usage of the product.  For Business Intelligence solutions the presentation and visualization are just as important as the content. Do you agree?

Ask anyone in this industry for his or hers definition of Business Intelligence and you get a thousand different answers. Is that a bad thing? Not necessarily, but it does imply that we still have not agreed upon some of the basics. It is an interesting paradox that although we are apparently not 100% sure about what Business Intelligence is we do have an opinion on how it should be or where it is going.  Over the last couple of years a common understanding of Business Intelligence has been created holding some universal truths firm in its grasp. Frank Buytendijk and I came up with a Top 10 of those universal (so called) ‘truths’. (Frank is Vice President of EPM Strategy for Oracle’s Business Intelligence and Performance Management Global Business Unit. You can find his blog at: http://blogs.oracle.com/frankbuytendijk/). We decided - just for fun – to turn the BI world as we know it around and see what comes up if we take the opposite view. Here is our Top 10:

1. We need one version of the truth.
2. BI projects require a business driven approach.
3. BI development should be done incrementally.
4. BI projects needs high level sponsorship.
5. BI = decision making process.
6. BI is going to be real time (BI 2.0).
7. Information democracy: BI for everyone.
8. BI cannot be outsourced.
9. Merging : BI, CI, EI, EPM, CPM, BIM…
10. BI is the answer to everything. 

I will start with the bottom five (saving the top 5 for a future blog). Frank will react to this on his own blog. This way we hope to create a dialogue or diablogue (dialogue + blog). Please feel free to join the discussion and share your opinions with us and all other readers.    10. BI is the answer to everything. No, almost every answer creates a new question. Data is often limited, not captured or of low quality. Therefore one can say that BI is NOT the answer to everything. In the best case scenario it is helpful.   9. Merging: BI, CI, EI, EPM, CPM, BIM. There are so many acronyms and abbreviations that I am not even sure that we can agree on all the two- or three letter words at the beginning. And even if we agree that BI stands for Business Intelligence - I doubt that we can agree on a common definition - let alone talk about merging. To my opinion it is not a merger but a different manifestation where the core or main point may differ per area (focus on strategy, or client, or process…). In each case: it should be information centric.  8. BI cannot be outsourced. A lot of BI professionals tell me that BI cannot be outsourced because of its specific nature (mutual development with end users, incremental approach, and so on). One of our clients has already outsourced its BI environment as long as 8 years ago. The proof of the eating is in the pudding.  7. Information democracy: BI for everyone. Information is critical for almost each person, process or technology. We in the western world produce more data than products. But in order to transform the data into information, knowledge and action you need to add your personal experience, time and effort to the mix. Some people will and will benefit from this. Some people will not and can live without it. BI is not for everyone but at least for the smart people.  6. BI is going to be real time (BI 2.0). If you view BI as strategic there is no need for real time data. But if you view BI as more operational than it will come in handy. In my view BI supports better decisions by making a strategy, process or objective - accountable, adjustable or adaptive. For accountability you do not need real time, for the other two you do. Therefore my answer would be: it depends.  5.BI = decision making process. Ouch, tough one. As mentioned previously I think that BI supports better decisions. It is up to the end user if they make use of it. They can make a decision that is in full conflict with the number crunching. Therefore BI is not the same as the decision making process it is just a supporting tool. I look forward to reactions from Frank and all the other readers.

At first sex with robots might be considered geeky, “but once you have a story like ‘I had sex with a robot and it was great!’ appear some place like Cosmo magazine, I’d expect many people to jump on the bandwagon,” said David Levy, a researcher at the University of Maastricht in the Netherlands. He recently completed his Ph.D. work on the subject op human-robot relationships called: “Intimate relationship with artificial partners”. Don’t worry, this blog is still on Business Intelligence, but I have your attention now. There’s a trend of robots becoming more human-like in appearance and coming more in contact with humans,” Levy said. But it is not so much the ‘mechanics’ of human-robot sex that he is interested in. As software becomes more advanced the relationship between humans and robots becomes more personal.  In my own words, it is not so much about lust but about attraction. He is talking about symbiotic relationships where two unlike organisms live together for mutual advantage. In this relationship something like a dialogue starts to happen. Until now most of this dialogue is one-sided (like a guy talking to his car like she is his girlfriend – Not much of dialogue other than in his mind). However the advances in software programming allow us to make this dialogue more bilaterally (just Google: “ELIZA” + “psychotherapist” and you will see what I mean).  Business Intelligence has also been a one sided relationship. We have used technology to help end users by supplying them with the right information. We gave them cubes and reports and good luck with it. But the real intelligence does not start until the end user adds his personal skills to the mix. Slicing, dicing and drilling his way towards a decision. But what if BI exists in a bilateral way? What if it can offer the decision instead of only just the information? What if through a dialogue between the OLAP cube and the Marketing Manager the correct campaign decisions can be made? This means programming a framework based on the task and responsibilities of the end user. With use of business rules, correlation, regression techniques and all kind of other statistical analysis’s you can create a dialogue with suggestions and propositions.   It will no longer be man versus machine, but man with machine. To me this sounds like a match made in heaven.  

Kaplan & Me [Click to open]

Yesterday was the launch of Microsoft Performance Point Server in the Netherlands. Keynote speaker was Robert Kaplan (Business Balanced Score Card, Strategy Maps). In the afternoon we did a workshop with Kaplan and some of our clients. This gave me the opportunity to ask him some BI questions.

The first question was what role BI can play for the BSC? He answered that BI is very important. Not only for the reporting framework but especially when it comes down to analytics. (See also: http://www.randomhouse.com/bantamdell/supercrunchers/ and http://www.amazon.com/Competing-Analytics-New-Science-Winning/dp/1422103323). According to Kaplan analytics are crucial during the review of the strategy. Domain experts can crunch the numbers to come up with new and perhaps more effective measures to drive performance. In his opinion all organizations should have an analytics group.

I also asked him what he thinks about two trends in BI: real time (or 2.0) and performance networks or ecosystem BI  (sharing your information with your stakeholders).  Kaplan mentioned that real time BI often is only considered for operational dashboards. However if these operational dashboards are alligned with the strategy they can also be your early warning system for more strategic decisions.

He also strongly believes in sharing information to improve performance, but he underlined that a feedback loop is crucial for this. This must start within your own company, allowing your own employees to react (for example by e-mail) if they find deviations. He gave an example of a large petrol company. Their strategy was to improve the convenience for the customer. The truck drivers, delivering gas to the stations, were very sceptical: “guys who do strategy don’t drive trucks”. But after a while they started calling the head office informing them about rude personel, unclean rest rooms and so on. Sharing the strategy was enough to start a change even before an improvement program was set up.

In summary:

1. Analytics are crucial for reviewing strategy (regression is cool again).

2. Real time BI can be an early warning system for strategic decision making.

3. Information sharing implies a feedback loop.

4. Communicating about strategy is almost as important as excuting the strategy.

[Click on image to enlarge]

With great thanks to my collegue Ron Torrico, the outcome of a small internal poll. No big suprises here (IBM), although the number #2 Microsoft ? Time will tell.

My prediction? Although IBM and Microsoft are most likely a strategic partnership with Informatica sounds great. It plays the independence card and also completes the Cognos BI platform which lacks a robust ETL tool.

In order to adapt to faster changing circumstances, organizations are more and more in need of a (near) real time supply of information within the decision making process. This means gathering transactional data from inside the organizations processes (often structured data) or on the outside edge (often unstructured data).  The true business value of business intelligence (BI) can be found in the ability to efficiently gather useful information from a diversity of both internal as well as external information sources and by sharing or interacting the intelligence with the ecosystems partners and customers to improve mutual performance.  This information centric approach is also reflected in the BI or Corporate Performance Management (CPM) suite of various vendors: integrating both organizations processes such as planning, consolidation and forecasting and search / knowledge management solutions. This makes sense as the most productive fields within the BI/CPM market are at this point in time: financial consolidation, BI platform, budgeting, planning and forecasting. This makes BI/CPM suite vendors a tasty bite. BI/CPM vendors often have similar products build upon the same type of technology. Therefore there are little distinguishing characteristics between the various vendors. It is therefore more than likely that technology is not the driving force behind the recent storm of BI/CPM consolidation. Keeping or gaining market share is their main goal. From a technology standpoint there is a case to be made that the innovation will come from small companies focusing on niches, such as: strategy & implementation tools, meta- and master data, advanced visualization, text mining, search or business activity monitoring. These will be likely candidates for a content driven acquisition. A strong trend in the BI/CPM market is the integration between hard- and software. The main push for this is the focus on real time transactional data for the decision making process. This will result in a merge between (transactional) platforms/databases and (interactive) BI/CPM solutions. A dominant position for platform/database vendors such as SAP, Oracle, IBM and Microsoft is to be expected.  These BI/CPM platform vendors will end up with a large number of duplicate tools. They will have at least two or more tools fit for the same solution. They must choose if they want a technical integration (complex, time consuming) or to thin out or rationalize their stack (kill your own darlings). Over time this heated BI/CPM consolidation will result in new product stacks, making recent tool selections obsolete and future ones a challenge.  Recent BI/CPM acquisitions Some of the most striking BI/CPM acquisitions so far have been:

• SAP acquiring Pilot Software, Outlooksoft and Business Objects;
• Oracle acquiring Peopleosoft, Siebel and Hyperion;
• Microsoft acquiring Proclarity (and the release of their Performance Point Sever);
• SAS announced a strategic partnership with Teradata (just separated from NCR);
• Cognos acquiring Adaytum, Frango, Celequest, Applix.

SAP acquires Business Objects SAP has announced that it intends to acquire Business Objects in friendly takeover. They will pay 4.8 billion Euros for this deal. Business Objects will operate as a stand alone business but part of the SAP group. SAP is an established ERP vendor and market leader in Enterprise or business applications. They have a strong transactional platform which includes BI (Netweaver) and CPM (Outlooksoft) capability. SAP has had limited success with their own BI solutions. The main reasons for this are their weak front end capabilities and problems with unlocking data from non SAP sources.  Business Objects is a pure BI player with very strong reporting, analytics and CPM capabilities with many industry models available, focusing on the mid market. They have been the market leader in Business Intelligence. Business Objects has entered the CPM arena late in the game, some say maybe too late. They have made up for this by buying SRC, ALG and Cartesis, allowing them better access to the finance and CFO office. However as they are still busy with the XI integration this must be quite an extra burden. With SAP being strong in enterprise applications but falling behind in BI, this acquisition of the Business Objects makes up for this. Not only can Business Objects help them with their front end solutions or metadata, they can also give them the much wanted business user access and mid market expansion. Business Objects can also help with Non SAP data integration.  Does that mean that everything is like hand and glove between them? Some analysts question the robustness of the Business Objects ETL tool, especially in complex SAP environments. But the main challenge will be the integration of the all the (previously) acquired companies and their solutions. Having more than one solution for planning, reporting, consolidation, analysis or score carding will also create much confusion with their clients on which will survive the cut.  In summary, this acquisition allows SAP to have the transactional platform and the interactive Business Intelligence (Business Objects) and Corporate Performance Management (Outlooksoft) solutions. However they will face a large product overlap in the CPM area: planning, budgeting, financial consolidation. Having said this, technology probably has never been the main driver for this acquisition. This can purely be considered as a strategic reaction against the Oracle / Siebel /Hyperion merger.

Yesterday morning I was in the gym doing my weekly workout. There is little more boring than standing on s stationary machine, working up a sweat, and basicly not going anywhere. Therefore I try to make this task a little more attractive by watching TV (my cross trainer comes with a build in TV screen). I was watching a show called “How it’s made” on the Discovery Channel. This specific episode showed the production of artifical logs (for your fireplace), snowmachines and range hoods. Range hoods are those things you will find in a kitchen above the cooker or stove. The basic three functions of a range hood are: contain the smelly air, filter it and ventilate it somewhere else.

At first I thought of ETL (Extract the data, Transform it and Load it in the datawarehouse) and how to use this analogy in a presentation. But things got even more interesting. Because this TV programm shows how things are being made - the creation process of the range hood was shown in detail. You could see that the engineer had a screen in front of him showing all the important steps. With this electronic manual the company made sure that all steps were done in the right order. The engineer had to sign off on each step. Also the distribution of all materials (like a skirt, motor or fan) was monitored. Not only did this improve the quality of work - at the same time the management was provided with valuable information on product use (ideal for stock control!) - and the performance of their engineer (for planning & control).

Kimball talks about the 38 subsystems of ETL. He indicates that ETL is often complex and labor intensive.  Labor intensivity drives a strong case for industrialization. Making the process more standard could decrease costs and would enable system integrators to rightshore this solution. The complexity could be tackled by having the BI engineer use some kind of electronic manual for all the important steps in creating an ETL environment. 

Therefore: an industrialized approach to ETL makes sure that both the process and the products are of high quality and low cost.

PS: Yes, this doesn’t have to be limited to the backend, but can also be applied to reporting & analysis.

Yesterday SAP announced that they have agreed on a friendly take over with Business Objects. As they have already acquired Outlooksoft they can now work on integrating these seperate environments to create a complete DWH/BI/CPM platform. That means that SAS (privately owned), Teradata (just seperated from NCR) and Cognos are the last 3 men standing. So when will IBM annouce that they have bought Cognos? Or wil Microsoft swoop them away first? Fun times!See also my previous blogs: BOBJ for sale & kiss of death (vendor consolidation)

In the Volkskrant (a Dutch newspaper) I read two interesting articles The first one was titled: “Radiologist in India monitors Ethiopian baby”. The second one: “Drilling for oil in Iraq using a webcam”. If you take one step back and analyze these two articles you will find a very simple underlying model they have in common.

In both articles you have an expert (the radiologist or the oil driller) monitoring a remote process (baby development or oil drilling) and if needed taking action.

The main reasons for this remote monitoring are the lack of know-how, expertise or information in the vicinity of the process itself.

The main benefits of this remote monitoring and action taking were cost reduction and less time between the actual event and the appropriate measure.

Now apply this to the Business Intelligence domain.

The expert (Business Intelligence professional or knowledge worker) monitors the key business processes (for example: supply chain, time to market or product delivery), runs analysis’s and  if needed takes action by intervening in the process or contacting the process owner.

If I close my eyes I can see a mission control center – like the bridge in Star Trek – where the whole company and its surroundings is being monitored for better performance.

Make it so.

Last week Microsoft came in the news with two seemingly separate events. The Dutch government has decided to make more use of open source software. There are several reasons for this: lower costs, more innovative employees and so on. But the most dominant driver for this was limiting the position of power of Microsoft.

Also the European community won their fight with Microsoft, forcing them to give other companies more information on the ins and outs of Ms Windows. Giving these so called API’s to other companies allows them to better integrate their products with Ms Windows. In the extreme this can be considered as a first step towards making Microsoft more open(source).

The support for open source software among some IT professionals is powerful. For example there is a high correlation between pony tailed network administrators and (Linux) penguin T-shirts.

As many of you know Microsoft has been entering the BI arena with products like performance point server. Many argue that Ms Excel already is the dominant BI tool anyway. It has been common practice among IT professionals to bash Microsoft and for BI professionals to bad-talk Excel. But on the other side, the business people (especially those grayish people from finance) cannot live without their spreadsheets. So if IT doesn’t want it but the Business needs it – are we at a deadlock?

If this first step of forcing Microsoft to open up its secrets to other players continues - perhaps in the future IT and Business can work together in making Microsoft BI (2.0) a truly collaborative business intelligence experience.

Bookkeeper and BI professional – both wearing a “I LOVE BILL” T-shirt – working together on improving Microsoft BI. Shall we live to see this?

Day 1 in Mumbai. What an overwhelming city this is. After ‘surviving’ Mumbai rush hour I will never complain anymore when stuck in traffic (ok, maybe a little). But this is probably the only city in the world where you don’t get RSI from clicking your mouse but from honking your horn (if you have been here you know what I mean). After driving through the slums – overpopulated with people so poor they cannot afford to buy shoes - back in my hotel room I found two chocolate cupcakes with fresh whipped cream! So does this mean that we should consider the Indian IT solutions to be at the bottom of the value chain as well? Or at best be of mixed quality? Is India only a country for low cost IT resources – or is there more than meets the eye?

I have spent the day talking with people that know how to do business. And their knowledge of Business Intelligence (BI) far exceeds the ‘simple’ data warehousing and reporting capabilities. Ok, it’s confession time. I also thought of India as a quick fix for our resource problems. Let’s just dump our specs over the wall and let the Indians build it. But guess what? They have strong knowledge of consulting – in for example supply chain or customer value – and of BI integration, and development and even outsourcing and maintenance. Some customers have said to me: “India is a great concept but that does not work for BI. Also all our documentation is in Dutch. This will not work”. But can you believe that today I met some Indians who actually spoke some Dutch! They had worked on a project where originally all requirements were in Dutch but were translated (in India) into English. After they were checked again in the Netherlands they started building the solution in Dutch using an English/Dutch glossary as reference. By working together with the end customer in the Netherlands they quickly picked up some of the lingo. So I have seen the proof here. A lot of successful BI projects (91% of the customers valued it as good, very good or excellent). And although I have to admit that BI is not standard and will probably be different for each customer or even subject area, I am pretty sure these guys can solve anything thrown at them (or already have).

So here are my questions for you:
• How mature can a BI solution or project from India be?
• What are the limitations? What are the opportunities?
• Is it limited to offshore development based on onshore requirements? Or can they do more?
• Where will it end?

Please let me know what you think of this.

I hate to say I told you so - but I am going to do it anyway! I wrote in my blog about vendor consolidation that the innovation would come from small companies and that the larger (BI) vendors would acquire them. Well yesterday Cognos annouced they acquired Applix - a 200 person company specialized in performance Analytics.

See what cognos writes about this on their own website:

Cognos to Acquire Applix
Cognos has announced the execution of a definitive agreement to acquire Applix, Inc., a publicly held company based in Westborough, Massachusetts and an industry leader in analytics. With this acquisition, Cognos expects to further extend its position as a leading independent provider of financial performance management.

Applix will complement Cognos flagship products—Cognos 8 Planning, Cognos 8 Controller, and Cognos 8 Business Intelligence —specifically in the area of financial performance management. Applix performance analytics will give customers new capabilities to analyze and optimize financial performance. This will include improved control and leverage of large, complex financial performance data; strong finance self-service capabilities for “what-if” business modeling and scenario management; new solution areas including profitability analysis; and innovative technology, including a patented, 64-bit, in-memory multidimensional OLAP server

For as long as I can remember, the market for Business Intelligence tools has been more or less stable. Sure, sometimes a new player entered or another disappeared. But on the whole, the market was dominated by a couple of vendors. The most exiting news came from new versions, upgrades or a technological breakthroughs. To put it bluntly: you could choose either Cognos or BusinessObjects or if you were running SAP you could go with BW. Than things slowly started changing. Instead of Business Intelligence we suddenly found ourselves talking about things like Corporate Performance Management, financial consolidation, budgeting, planning, dashboards and scorecards. And we embraced the tools that came with it. Our work was no longer limited to reporting and analysis. We were proud that after so many years finally the strategic value of Business Intelligence was recognized. Thus changing the playing field completely. The focus in BI was shifting from a technical toward a business solution.

The BI and platform vendors quickly recognized this also and have been embracing these tools as well. They bought up companies that had tools specializing in this specific area and integrated them – as best as they could – in their own product portfolio. SAP for example acquired Outlooksoft (Consolidation) and Pilot Software (Strategy Management). Oracle acquired Hyperion (after they bought Peoplesoft and Siebel) – the market leader in planning, budgeting and financial consolidation. Microsoft acquired ProClarity (Analytics) and made a firm entry to the BI/CPM marketplace when they launched their Performance Point Server. Cognos – a pure BI player – acquired Adaytum and Frango and created a complete CPM solution. Finally BusinessObjects- still busy with integrating Crystal – bought Cartesis (consolidation) to complete their CPM offering that they already started building on when they acquired SRC and ALG Software. However is this buying up – this consolidation – not the kiss of death? Where is this going to end? Why this focus on applications rather than information?

Business Intelligence has come a long way. It is now closer tied with the business processes as ever before making the basic data (quality) that comes from this also more and more important. BI still can be an aggregated report of last month sales on a highly accumulated level but it can also be operational dashboards monitoring key business processes on a real time level supporting split second decisions. This makes a strong case for the platform vendors. They have the applications to support the business processes, capture the data and report on them. An integration of software and hardware therefore seems to be unavoidable. Cognos is one of the fist BI pure players to understand this. Their launch of Cognos Now! – which integrates a hardware server and dashboard for realtime monitoring – is proof of that. They might well be the first BI platform vendor.

But basically all BI/CPM vendors have more or less the same products and use the same technology. Even the corporate take over strategies are being copied resulting into a concentration of BI vendors. The real innovation still has to come from small companies that specialize in a niche of business intelligence such as: strategy implementation, meta- and masterdata, advanced visualization, text mining, search and business activity monitoring. These small companies will probably at some point in time be bought by one of the large BI vendors. After all, Cognos Now! was Celquest before. The real challenge for the large BI vendors will than be the technical integration of all these tools. It is easy to make a Ms Powerpoint presentation of where this new application fits in your product portfolio – but making it work with these other tools is a far more difficult thing to do.

Finally, will we end up with one or two BI vendors with a standard set of Business Intelligence tools? No, I don’t think so. The BI tool set will always be dynamic. Remember? After DDS (Decision Support Sytems) and EIS (Executive Information Systems) came Enterprise Reporting and OLAP (Analytics). Than came the consolidation, planning, dashboard and scorecarding tools. Nowadays, the streaming servers and realtime monitoring tools are being wheeled into the business. There will always be something new. The platform vendors will take a dominant position in the BI market but there will always be room for vendors with a focus on BI only. It is almost like the Olympics. Sometimes you need the decathlete to jump, run and throw the javelin and sometimes you just need a 100 meter specialist.

But I am not sure that the real innovation will come from these companies, though they may well acquire the innovators to embed them into their enterprise solutions. Like a thousand points of light, there are individuals and small companies out there already thinking in a really innovative manner about what Business Intelligence could be given new capabilities and technologies.

Who are they? I can’t wait to find out! Let me know what you have seen or are thinking, and that also means the big players too.