Hybrid data warehouse architecture: many choices for full flexibility

by Matthias Funke

matthias blog pictureI admit, I love cars. And as a car enthusiast, I cannot imagine not having my own car. I use it every day, I rely on it. I feel at home when I enter it. But I accept that other people may be different. Some are not as attached, some can’t drive or don’t need a car often enough to justify the purchase. For them, using a cab or car service might be the better choice. And then there are people who need flexibility; they need a pick-up truck one day, a van the next day, and a sports car on the weekend. In short, they want full flexibility.

What does all of this have to do with IT and Data Warehousing? Well, at IBM, we think most of our clients have similar, diverse needs when it comes to their data warehouse environment. Depending on the use case at hand, one of several different data warehouse form factors may be better than the others for a particular analytics workload at that time.

Depending on the use case, one of several different data warehouse form factors may be better than the others for a particular analytics workload at that time.

Should it be hosted, vendor-managed, or do I want complete in-house control? Do I need full flexibility regarding the service levels I set for my clients, or is it sufficient to work within the distinct configurations that a service or vendor provides? Behind the scenes, all of this directly impacts the combination of compute versus storage resources I want to have to deliver the right level of  flexibility in the most cost-effective way.

No longer is data warehousing a one size fits all approach.  You need to weigh factors like the service level and the importance of meeting it, amount of flexibility you need, cost of the solution and the amount of control that each of your analytics workloads requires.  In line with this, we  see demand for three distinct data warehouse form factors:

  • Managed cloud service – A vendor-managed public cloud service is the most simple to use as it requires no system administrator on the client side. It is easiest to engage with because you can instantiate a service very quickly, paying for what you use at the moment (“pay-as-you-go”).
  • Predictable, high performance appliance – A client-managed data warehouse appliance offers the best predictability and performance due to its balanced, optimized software and system stack (including hardware), and the best price-performance when use cases require long-term, high utilization of a warehouse. Depending on client skills and effort, the appliance might offer the best simplicity and management, as well as lower TCO.
  •  Software-defined or private cloud software – A client-managed data warehouse service that would run on either your infrastructure, or a hosted IaaS (think Softlayer) is a third option. Use it when you want to increase utilization of existing infrastructure investments and when you need full long-term flexibility to adjust the service depending on the analytics use case and the LoB demand for the analytics. As I stated above, adjusting service levels means  you need the control and flexibility to adjust the combinations of compute  and storage resources to meet current needs. In this scenario, you have control and management of the infrastructure, and  you can enjoy appliance-like simplicity of the data warehouse while still being able to manage it yourself.

Now what if you want to use each of the above form factors in differing combinations to meet a variety of needs?  What if you could choose the best form factor for each workload at that moment in time? Integration across instances of each form factor could enable you to load or replicate data, or to abstract users and applications from the physical layout of your data stores. This becomes a critical success factor in building logical, hybrid data warehouse solutions that offer best flexibility and agility for the business at the lowest cost plus the ability to marry fit-for-purpose data stores, structured and unstructured, into the overall architecture.

No longer is data warehousing a one size fits all approach.  You need to weigh factors like the service level and the importance of meeting it, amount of flexibility you need, cost of the solution and the amount of control that each of your analytics workloads requires.

If you follow IBM, you know that we just launched a Preview Program for our IBM dashDB Local as a software-defined environment (SDE) data warehouse deployment option.  It addresses the needs of the software-defined / private cloud form factor above and it complements the IBM PureData System for Analytics appliance and the dashDB managed cloud datawarehouse service we already offer. Take this new preview for a test drive and tell us what you think so together we can shape the hybrid data warehouse architecture of the future.

About Matthias,

Matthias Funke_headshot Matthias is the worldwide leader of the IBM Data Warehouse product line and strategy. He is passionate about data as the “new currency” and looks for new ways to deliver insights from this data. Matthias brings many years of technology experience to his role including product management, software development and leading software development teams.

Follow Matthias on LinkedIn

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s