BI Consolidator FAQs

This past year, the Partner Intelligence team has kept busy converting legacy reporting applications into web-based BI products.

With the business intelligence market consolidation of the big mega-vendors (IBM, Microsoft, Oracle, and SAP) and an industry move to web-based reporting products, companies are replacing their legacy reporting applications with modern technologies. Many of these application conversions have also involved platform changes. 

Converting these large applications manually would take too long, cost too much, and be too risky.  Because of that, we automate as much of the process as possible.

I get asked lots of questions about how we accomplish this seemingly impossible task, so here are my responses to some common FAQs. 

What is DAPPER?

DAPPER is a toll-gate project methodology for addressing the complexity of converting legacy reporting applications into modern enterprise web-based BI products.  DAPPER is an acronym for our major conversion steps: Discovery, Analysis, Pilot Project, Project Plan, Execute Plan, and Retire Legacy Product.  

What is the BI Consolidator?

The BI Consolidator is a proprietary web-based software application that automates many of the tasks performed during a legacy BI conversion.  Written in C/C++, the application is quite small and fast -- the entire application takes less than 4 megabyte of disk space. Depending on the technology being converted, the software can process hundreds if not thousands of programs per minute.

What is the BI Consolidator Scanner?

During initial Discovery and Analysis steps, we use a feature of the BI Consolidator to scan legacy BI application and inventory the details within a data repository.  Using software to perform this activity dramatically reduces the manual effort, cutting down on the time and cost of performing upfront conversion analysis. This database provides details for scoping the work effort and performing the later conversion tasks.  Using scanned inventory data, we can automatically generate analysis and project-related documents. 

Does the BI Consolidator scan company data?

The BI Consolidator only scans application code such as program logic, metadata, web launch pages, batch jobs, and so forth.  It does not scan any of the underlying data. Partner Intelligence does not use client scanned application data for any purpose other than to perform the conversion engagement.

What is the BI Consolidator Converter?

When executing the conversion plan, we use a feature of the BI Consolidator application which can read legacy code and automatically translate the syntax to a web-based alternative.

From which reporting languages does the BI Consolidator currently convert?

We have used the BI Consolidator in client engagements to convert SQL, QMF/SQL, NOMAD 4GL, FOCUS 4GL, Oracle Portal, and Crystal Reports.  We are currently investigating parsing other languages such as IBM Cognos Impromptu and SAS.

To which BI products does the BI Consolidator currently convert?

Currently, we convert legacy reporting applications into WebFOCUS, the enterprise BI product from Information Builders. We are also investigating a new generator capable of producing IBM Cognos.

Is your conversion process "Point-to-Point" or do you have a "Canonical?"
Okay, this question came from a really smart person.  Rephrased, the question is: does your conversion tool only work with specific combinations (such as NOMAD-to-WebFOCUS) or does it translate the old language into a standard (a "canonical") and then re-translate that to another language (and is then theoretically anything-to-anything)?  The answer is that we started out with some point-to-point features but have since moved to a true canonical translation process. We have parsers that understand how to read a legacy language and convert it to the canonical.  We then have generators that can read the canonical and generate the web-based BI syntax. 

What languages can the BI Consolidator scan?

We can quickly create scanning capabilities for any text-based language.  In addition to the languages that the BI Consolidator can currently convert, we also have scanning capabilities for SAS, Brio SQR, RPG, COBOL, and others.  As an example, for the SAS product we can automatically determine the complexity of applications by scanning for statistical analysis procedures, screens, output file creation, reports, database updates, and so forth.  If you have a little-used reporting tool that is in a simple text format (let's say something like mainframe CA/EARL or DYL/280), then we can still put together a scanner feature within a day. 

What else can the BI Consolidator do?

We have also enhanced the conversion application to perform general mass-string conversions and specific conversion tasks.  For example, we automated the conversion of changing hundreds of userids from mainframe security to Active Directory. We translated hundreds of HTML web pages using CGI calls to instead use Java servlets.  We translated hundreds of mainframe batch jobs (MVS JCL) to call WebFOCUS instead of FOCUS.

Does it work?

Absolutely.  The predecessor of the BI Consolidator was first created and used successfully in the early 1990s for a large consumer goods product firm here in Cincinnati.  About ten years later, the concept was rewritten into a web-based application.  During one of the first engagements of the next generation, a Fortune 50 financial services firm won a ComputerWorld award for automating hundreds of legacy mainframe ad-hoc report user libraries to a web-based BI product.  Estimated to take years and cost millions of dollars, the project was instead done in three months at a fraction of the cost. Since then, we have successfully used the software at other well-known companies and government agencies.  If you would like, you can talk with our happy clients.

How much does the BI Consolidator cost?

Partner Intelligence does not sell the BI Consolidator as a formal software product. Instead, our consultants use it on our services engagements to speed the project.

Where is the BI Consolidator installed?

We do not install the BI Consolidator within the customer's environment.  It is only used on a stand-alone notebook belonging to Partner Intelligence. 

How can my company benefit from DAPPER and BI Consolidator?

If you have a legacy reporting application that needs to be converted to a web-based product as quickly, safely, and cost-effectively as possible, please contact Eric Bulmahn at Partner Intelligence.  We work directly with the company using the BI software or with their services partners who need assistance reducing the time, cost, and risk of performing legacy conversions. 

Disappearing Business Intelligence Products

A recent conversation with a Business Intelligence software vendor did not go well for me.  I told them that I knew of close to a dozen local companies who had either already eliminated their BI product or were planning to do so.  I saw it as a disturbing trend and asked how I could help.

They did not deny the facts.  Instead, they took my comment as a personal attack and never want to talk to me again.

I truly feel sorry for sales people working for BI software vendors.  It is a tough job even for the market leaders, much less the underdogs.  Sales people must often sell brand new innovations that nobody else has yet bought, recent technology that may or may not yet have any referenceable accounts, or legacy products that other companies are getting rid of.

By the nature of their jobs, software sales people are forced to ignore reality.  Their sole responsibility is to sell their company's product A, B, or C.  Just figure out how to make people buy it.  If they can't, they are put on the infamous "performance improvement plan" (which is BI vendorspeak for "You have three months to find another job"). 

Software sales reps have to pretend that their product is the right one for the prospect and keep pushing the decision maker for a purchase.  That is a tough job and I sympathize with what they go through.

My personal problem is that sales people who have to ignore facts hate people like me who point out the facts.

Which reminds me of a funny story.  Presenting at a seminar, a vendor rep claimed, "After implementing our software, company ABC saw a 300% decrease in project costs!"  A hand immediately went up in the crowd.  "I'm sorry, but if you eliminate all costs that is a 100% savings.  How is it possible to reduce costs by 300%?"  The perturbed sales rep countered with, "I'll send you a white paper."

Facts get in the way of a good sales pitch.

Sales people do not want to discuss problems customers are having with their software.  They do not want to know about bugs in their products. 

But that's not completely true.  More correctly, sales people do not want their prospects and customers to hear any negatives.  If I have the audacity to point out troubling facts to the BI vendor, they are worried I will pass that along to the market in general.

Sales reps should remember Sun-Tzu's famous saying, "Keep your friends close, and your enemies closer."  I have never heard anybody recommend "Take a hike, jerk" as a life mantra.

The BI software market is consolidating.  The mega-vendors (IBM, SAP, Microsoft, and Oracle) purchased the pure BI vendors that will survive into the future.  SAS leads the statistical BI market as a pure BI vendor, but the other BI firms face tough times.

In general, companies are replacing their existing BI products with those from the mega-vendor list.  Legacy reporting tools are being eliminated and replaced with modern technologies.  If a vendor's BI product did not make the jump to the web, it is probably dead.

If that BI tool is not prepared to move onto mobile tablet computing, it might also be headed for the scrap heap. Howard Dresner points out a predictive fact in a recent study on mobile BI.  A third of the people he surveyed said they would move to new technology that supported mobile BI.

Don't you just hate facts?

Apple in the BI Space?

Earlier today, Lizette Chapman at Fast Company wrote about Apple's $46 billion in cash reserves.

Investors are asking the company to use that to pay out dividends, but Steve Jobs wants to keep sitting on the dough.  Steve says:

We know if we need to acquire something--a piece of the puzzle to make something big and bold--we can write a check for it and not borrow a lot of money and put our whole company at risk.  The cash in the bank gives us tremendous security and flexibility.

If Steve wanted, he could take a small chunk of that money--not much, perhaps just a billion or two--and buy one of the few remaining pure Business Intelligence software vendors.  Apple could then leverage the acquired BI infrastructure and and build a nice BI front-end for the iPad.

That would classify as "big and bold."  Today, the BI software space is dominated by four mega-vendors: IBM, Microsoft, Oracle, and SAP.  All of these big companies acquired BI vendors.  Why not Apple?

My blog gets hit constantly by people looking for information about BI on the iPad and other mobile devices.  People want it.  Apple could buy it.  It would work.

Apple already has a Business Intelligence webpage showing other vendors' BI apps on the iPhone and iPad.  Using some of its cash, Apple could be a winner in the BI space.

Candidates for Steve to buy include Actuate, MicroStrategy, and Information Builders.  Personally, I would suggest that Apple acquire privately-held Information Builders' for their BI product WebFOCUS and their enterprise integration technology called iWay Software.

The WebFOCUS architecture works well as a powerful BI engine for Safari-based web applications.  So it already works today on the iPad.  Just to take things to that "big and bold" level, however, Apple could implement a beautiful touchscreen BI application to sit on top of WebFOCUS.

Of course, the product name WebFOCUS is right up there with "horseless carriage."  Apple will surely change that to something snappy like iBI (which is what most people call it today because of the vendor's initials), Apple Info, or Apple Knowledge. 

The iWay Software already has the Apple "i" at the beginning!  Still, I would make a name change recommendation--since enterprise integration is plumbing, the glue that holds things together, etc.--I would go with Apple Sauce.

With $46 billion in cash, Steve should have no problem becoming one of the BI software mega-vendors if he wants.

Why Amazon and Apple Will Not Kill Each Other

When Farhad Manjoo is not battling angry tweeters over his allegedly racist Slate article on "blacktags," he is writing interesting material about hot technology topics.

In the most recent September 2010 issue of Fast Company magazine, Farhad wrote a Tech Edge article called "The End of Winner-Take-All."  Farhad predicts that certain tech markets will result in stalemates instead of one company wiping out all other contenders.

As examples of two potential market dominators, Farhad points out the Amazon Kindle and Apple iPad. Rather than be enemies, Amazon and Apple might just have reasons to play nicely. 

I use Amazon's Kindle app and think it is the best e-reader for the iPad -- better than Kobo, Stanza, eReader, or even Apple's iBook.  Thanks to this app, my iPad becomes a front-end to the Amazon bookstore, which I prefer to those from competitors such as Borders or Barnes & Nobles.

Farhad writes:

Speaking to investors, [Amazon CEO Jeff] Bezos pointed out that because Kindle books are delivered across the Internet to a range of devices--including the iPad--Amazon might actually benefit from the Apple tablet's popularity.  The more iPads Apple sells, the more potential Amazon customers.  Bezos is wise to ignore the calls for Amazon to make the Kindle more like the iPad.  The Kindle will be a gadget whose only purpose is to read books.  Bezos concedes that such a device isn't for everyone, but 'serious readers' will always prefer a dedicated e-reader.

This cooperative thinking is foreign to the BI software market where each mega-vendor wants to supply the entire "technology stack" for a company's architecture.

Personally, I hope that Farhad is correct and Amazon and Apple will be buddies in the competitive e-reader market space (I love them both!).

IBM Continues Buying Spree

IBM just completed its purchase of web analytics company Coremetrics, right after buying Sterling Commerce, SPSS, and dozens of other software vendors.

Here is what KMWorld had to say about the deal:

IBM reports the Coremetrics acquisition will enhance its ability to help businesses rapidly gain intelligence into social networks and online media sources through a cloud-based delivery model, and use this insight to create smarter, more effective marketing campaigns.

Further, it explains, Coremetrics’ offerings can provide real-time intelligence on what consumers are saying about the products and services being offered to them and allow clients to make fact-based, accurate decisions on marketing expenditures. As a result, IBM claims, marketing teams can gain deeper insight about their consumers and present personalized recommendations, promotions and other sales incentives across a variety of channels where consumers interact with their brands. These channels span traditional outlets such as storefronts and catalogs and newer outlets including all forms of e-commerce and social media.

This is secret, so please do not repeat it, but my inside source at IBM says that next on their checklist of acquisitions are Smuckers, The Ohio State University Stadium, the Brent Spence Bridge in Cincinnati, and North Carolina.

Note: just a week later IBM acquired Datacap, a document capture  and management software vendor based in Tarryton, NY.