Data Warehouse Modernization: Vetting Forrester’s Return-on-Investment Calculations

By James Kobielus,

It’s well known that, in a prior life, I was an industry analyst focusing on the data warehousing (DW) market. So I think I have a good mental radar for identifying high-quality, data-driven DW research when I see it.

When you’re researching the potential return on investment (ROI) for a DW, you have to be rigorously quantitative, precise, and comprehensive in your approach. Enterprises often place the DW at the very heart of their big data and analytics strategies. Solid ROI metrics must support DW projects of any scope, and the range of competing alternatives demands a decision-support framework that facilitates apples-to-apples comparisons. Many DW projects involve modernizations at various levels, so ROI calculations must be adept at characterizing the potential bottom-line impact of new technologies, platforms, tools, and practices.

Sure, anybody can pull ROI estimates out of thin air, but finding metrics that can help you make DW investments with confidence can prove tricky. In that regard, I’ve long felt that Forrester Consulting’s Total Economic Impact (TEI) methodology is the best ROI calculation framework for information-technology (IT) investments of any sort. Grounded in Forrester’s extensive survey- and interview-based research, its TEI studies incorporate fine-grained benefit, cost, risk, and flexibility variables into an underlying spreadsheet-based model. The drivers, use cases, assumptions, formulas, and data intrinsic to Forrester’s ROI calculations are totally transparent, so you can vet them for yourself. On any specific TEI use case under scrutiny, Forrester projects the resulting ROI analysis over a risk-adjusted 5-year horizon from the point of view a typical “composite organization” that uses the technology in question.

Back in my Forrester days, I sweated these details when constructing a now-outdated TEI study of the DW appliance market. So naturally I was very curious when, over the holiday season, IBM made available a new Forrester TEI covering our entire Information Management (IM) solution portfolio, but with a core focus on DW.

On my first pass through the report, I noticed the sorts of high-level rollup numbers that usually figure into most marketing collateral or blogs on these kinds of studies. Specifically, Figure 1 states a 5-year risk-adjusted return of 148% and total benefits (present value) of $31.2 million for the typical composite organization. Still being an analyst at heart, I drilled more deeply into the study itself to determine what exactly it refers to.

The first thing you see, from Figure 2, is that, among the three use cases in this Forrester TEI, “DW modernization” accounts for around $5m of the benefits, with “security intelligence extension” a little over $3m and a whopping ~$23m from “enhanced 360-degree view of the customer.” Clearly, all of those are essentially DW-related returns.

When vetting a TEI, it’s best to single out the specific use case of interest. In my case, I focused on Forrester’s DW modernization use case, which estimates the quantitative bottom line from cost reductions and value enhancements due to more efficient storage and processing, speedier performance, and agile analytics. These are in line with the chief DW modernization drivers cited in Figure 6, which were derived from Forrester’s in-depth decision-maker interviews.

In terms of concrete decision support for DW professionals evaluating modernization initiatives, the real payoff from this study is on pages 27-30. These spell out the full assumptions for the use case, including scope of solutions included, size of the composite organization’s IT budget, percentage of that budget allocated to data and storage, number and growth of terabytes of DW storage, percent reductions in storage cost, number of staff using big data analytics, and so on.

Pay close attention to the solution scope under DW modernization. Forrester took the right approach by not limiting their analysis to DWs in the older, much more limited sense of premises-based analytic databases specializing only in structured, at-rest data for operational business intelligence. As they state on page 27, they included the broader sweep of big-data analytics, information integration, and governance solutions in IBM’s IM solution portfolio.

If they’d gone with a traditional DW scope, such as the one this former analyst included in his 2010 study, Forrester would have ignored the substantial evolution that this marketplace has experienced in this decade. If Forrester had stuck with that scope in this latest study, it would probably have limited its TEI to IBM PureData for Analytics, IBM DB2 with BLU Acceleration, and IBM Digital Analytics Accelerator for System Z. But it did the right thing this time around (reflecting what our customers are doing) by including our Hadoop, streaming, discovery, and InfoSphere IIG offerings in the scope of a hybridized, cloud-focused DW infrastructure.

To see how far mainstream DW solutions have advanced into cloud-centric hybrid architectures, check out this blog I published a few months ago on the new IBM dashDB. I’m assuming that Forrester’s exclusion of dashDB, as well as Watson Analytics and DataWorks, from this recent study was due principally to their need to lock down their project’s scope many months ago before these specific solutions were launched.

For enterprise analytics and IT professionals, the DW modernization ROI that you calculate for your own situation depends on the assumptions you make and how you adjust the Forrester TEI model’s parameters to align with those. The beauty of the Forrester TEI methodology is that its model can be easily customized and use cases easily extended to do justice to the complex range of technologies in DW modernization initiatives. Depending on the project and your requirements, DW modernization may include various blends of new technologies (e.g., Hadoop, in-memory), new topologies (e.g., hybrid, distributed, and zone architectures), new sources (e.g., machine, social, & mobile data), new form factors (e.g., cloud, appliance), new tooling (e.g., governance, curation, archiving), new development frameworks (e.g., MapReduce), and new scaling and performance approaches (e.g., consolidation, compression, scale-out).

If I have any quibble with the latest Forrester TEI, it’s with their apparent exclusion of traditional DW use cases, such as operational BI (the focus of our Cognos portfolio), from their scope. Also, Forrester doesn’t give the newer DW use cases, such as in-database analytics for statistical modeling and data science (the focus of our SPSS portfolio), as much emphasis as I’d wish.

But those are just scoping issues that can be easily addressed if Forrester ever chooses to take this TEI analysis in those directions in coming years.

About James, 

James Kobielus is IBM Senior Program Director, Product Marketing, Big Data Analytics solutions. He is an industry veteran, a popular speaker and social media participant, and a thought leader in big data, Hadoop, enterprise data warehousing, advanced analytics, business intelligence, data management, and next best action technologies. Follow James on Twitter : @jameskobielus


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s