Turn Up The Power For Software-Defined Data Warehousing

by Mona Patel

Interview with Mukta Singh

As big data analytics technologies such as Spark and Hadoop continue their move into the mainstream, you might think that the traditional data warehouse is becoming less important.

Actually, nothing could be further from the truth.

To enable data of all types to be ingested, transformed, processed and analyzed efficiently, many companies are choosing to build hybrid analytics architectures that plug cloud and open source technologies such as Spark and Hadoop into on-premises environments. At the heart of these hybrid architectures lies the data warehouse – a highly reliable resource that provides a single source of truth for enterprise reporting and analytics.

This raises an important question: since the data warehouse is so central to the hybrid analytics architecture, how can we make sure it performs well and cost-effectively?

Traditional wisdom is that the infrastructure doesn’t matter – that running these vital systems of record on commodity hardware is perfectly adequate. But when you look at the numbers, you may begin to question that view.

To understand why the right hardware – in this case, IBM Power Systems – can make a real difference, I spoke with Mukta Singh, Director of Data Warehousing at IBM. In my conversation with Mukta, we take a deeper dive into why IBM’s software-defined data warehouse – IBM dashDB Local – on IBM Power Systems offers a better price/performance ratio compared to commodity hardware.

Mona_Blog

Mona Patel: Can you tell our readers a little bit about the Power Architecture? What is so unique about it?

Mukta Singh: IBM Power Systems is the dominant server platform in today’s Unix market, with over 50 percent market share. It has also become a leading platform for Linux systems, and we have seen tremendous growth in that area in recent years.

Unlike commodity servers, which typically use x86 processors, Power servers use IBM’s Power Architecture, a unique processor architecture that has been designed specifically for big data and analytics workloads.

Mona Patel: How does IBM dashDB Local integrate with Power Systems?

Mukta Singh: dashDB Local is a software-defined data warehouse offering that has been optimized for rapid deployment and ease of management. Essentially, the system runs in a Docker container, which means it can be flexibly deployed on different types of hardware either on-premises or in a private or public cloud environment.

One of the options today is to deploy your dashDB Local container on IBM Power Systems – it runs completely transparently, and it’s optimized to allow the dashDB engine to take advantage of the unique features of the Power Architecture.

If you want to move an existing dashDB Local environment from x86 to Power Systems, that’s easy too. The latest-generation POWER8 processors can operate in little-endian (LE) mode, which is the same byte order that x86 processors use. That means that you can move a dashDB container from one platform to the other without making any changes to your applications or data.

At a higher level, we have also ensured that running dashDB on Power Systems offers the same user experience as it does on x86, so the database and OS management, monitoring and integration aspects are exactly the same. The skills are completely transferable from one platform to another, so it’s a free choice and users don’t have to worry about being locked in.

Mona Patel: Can you tell us about the benefits that the Power Architecture provides for dashDB Local?

Mukta Singh: Well, for example, dashDB’s analytics engine is built on IBM BLU Acceleration – a columnar, in-memory technology that cuts query run-times from hours or minutes to just seconds.

BLU Acceleration is designed to take advantage of multi-threaded cores, and Power processors have more threads per core than most current x86 processors. In fact, if you compare an IBM POWER8 processor to an Intel Broadwell EX, it has four times as many threads per core. That means if you have a query that BLU can parallelize, you will get much better performance from Power Systems.

Similarly, because dashDB’s BLU Acceleration does all the processing in-memory, the bandwidth between the processor and the memory is very important. Again, Power Systems has a huge advantage here, with four times as much memory bandwidth as the x86 equivalent.

Finally, the processor’s cache size is important. BLU is engineered to do the majority of its processing in the CPU cache. That means it doesn’t need to repeatedly access data from RAM, which is usually a much slower process. Power processors offer four times as much cache than x86, which means they offer lower latency and reduce the need to access RAM even further. So they play to the strengths of dashDB’s query engine.

Mona Patel: So how do those numbers translate in terms of performance and cost-efficiency?

Mukta Singh: We’ve done a benchmark with dashDB Local of a 24-core POWER8 server versus a 44-core x86 server.

The Power server was 1.2 times faster in terms of throughput, despite having 45 percent fewer cores. Or to look at it another way, each POWER8 core offered 2.2 times more throughput than the x86 equivalent. Leadership performance and competitive pricing for Power scale-out servers deliver a very compelling price-performance-optimized solution with dashDB Local.

Mona Patel: How do you see the market for dashDB Local on Power Systems? Is this something that customers have been asking for?

Mukta Singh: Even when we started bringing dashDB Local to market last year, there were Power clients who were interested. As I mentioned earlier, Power has a dominant share of the Unix market, and there are thousands of companies whose businesses are built on DB2 or Oracle databases running on Power Systems. For companies that rely on Power Systems already, the idea of running dashDB Local on their existing infrastructure is very attractive.

But the results of our benchmark suggests that this isn’t just a good idea for existing Power clients – it’s also an opportunity for new clients to start out running dashDB on a hardware platform that is tailor-made for high-performance analytics.

And for any client who currently runs dashDB on x86 servers, the message we’d like to get across is that it’s easy to move to Power Systems. It’s faster, it’s more cost-effective, and you still get all the ease of use and ease of management that you’re used to with your existing dashDB environment.

Mona Patel: OK, last question: where can our readers go to learn more about dashDB Local on Power? Can they try out dashDB Local on Power Systems before they buy?

Mukta Singh: Yes, we offer a free trial with a Docker ID – please visit dashDB.com to learn more and access the trial.

About Mona,

mona_headshotMona Patel is currently the Portfolio Marketing Manager for IBM dashDB, the future of data warehousing.  With over 20 years of analyzing data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at IBM, a leader in data warehousing and analytics.  Mona received her Bachelor of Science degree in Electrical Engineering from UCLA.

Enterprise Data Warehouse Beyond SQL with Apache Spark

By Torsten Steinbach, Lead Architect for IBM Data Warehousing Advanced Analytics

Enterprise IT infrastructure is often based heavily on relational data warehouses, where all other applications communicate through the data warehouse for analytics. There is pressure from line of business departments to use open source analytics & big data technology, such as R, Python and Spark for analytical projects and to deploy them continuously without having to wait for IT provisioning. Not being able to serve these requests can lead to proliferation of analytic silos and lost control of data. For this reason, the new IBM dashDB Local for software-defined environments (SDEs) and private clouds has now integrated a complete Apache Spark stack, enabling you to continue to operate established data warehouses and leverage its proven operational quality of service as well as running Spark-based workloads out of the box on the same data.

This tightly embedded Apache Spark environment can use the entire set of resources of the dashDB system, which also applies to the MPP scale out. Each dashDB Local node, each with its own data partition, is overlaid with a local Apache Spark executor process. The existing data partitions of the dashDB cluster are implicitly derived for the data frames in Spark and thus for any distributed parallel processing in Spark on this data.

ts_screenshotone

 

 

 

 

 

 

 

The co-location of the Spark execution capabilities with the database engine minimizes latency in accessing the data and leverages optimized local IPC mechanisms for data transfer. The benefits of this architecture become apparent when we apply standard machine learning algorithms on Spark to data in dashDB Local. Comparing a remote Spark cluster setup with a co-located setup we found that those algorithms can get a significant increase in speed. This even includes optimization in remote access to read data in parallel tasks, one for each database partition in dashDB.So you can see that there is indeed a performance advantage provided by the integrated architecture.

In addition, the Spark-enabled data warehouse engine can do a lot of things out of the box that were not possible before:

1. Out of the box data exploration & visualization

ts_screenshotone

2. Interactive Machine Learning

ts_screenshottwo

3. One-click deployment – Turning interactive notebooks into deployed Spark applications

ts_screenshotthree

ts_screenshotfour

4. dashDB as hosting environment to run your Spark applications

Once as Spark application has been deployed to dashDB it can be invoked in three different ways.

Using spark-submit.sh from command line and scripts:

ts_screenshotfive

ts_screenshotsix

Using dashDB REST API:

ts_screenshotseven

Using SPARK_SUBMIT stored procedure:

ts_screenshoteight

5. Out of the box machine learning

ts_screenshotnine

In addition, this implementation of Spark capabilities in dashDB Local provides you with a high degree of flexibility in ELT and ETL activities and let you process and land data in motion in dashDB Local.

Let’s summarize the key benefits that dashDB with integrated Apache Spark provides:

  1. dashDB Local lets you dramatically modernize your data warehouse solutions with advanced analytics based on Spark.
  2. Spark applications processing relational data gain significant performance and operational QoS benefits from being deployed and running inside dashDB Local.
  3. dashDB Local enables analytic solution creation end-to-end, from interactive exploration and machine learning experiments, verification of analytic flows, easy operationalization by creating deployed Spark applications, up to hosting Spark applications in a multi-tenant enterprise warehouse system and integrating them with other applications via various invocation APIs.
  4. dashDB Local allows you to invoke Spark logic via SQL connections.
  5. dashDB Local can land streaming data directly into tables via deployed Spark applications.
  6. dashDB Local can run complex data transformations and feature extractions that cannot be expressed with SQL using integrated Spark.

Please also check out the tutorial playlist for dashDB with Spark here: ibm.biz/BdrLNG.  You can also download a free trial version of dashDB Local at ibm.biz/dashDBLocal to see these Spark features in action for yourself.

 

About Torsten,

torsten-steinbachTorsten has worked over many years as an IBM software architect for IBM’s database software offerings with particular focus on performance monitoring, application integration and workload management. Today, Torsten is the lead architect for advanced analytics in IBM’s data warehouse products and cloud services.

Why Are Customers Architecting Hybrid Data Warehouses?

By Mona Patel

As a leader in IT, you may be  incented or mandated to explore cloud and big data solutions to transform rigid data warehousing environments into agile ones to match how the business really wants to operate.  The following questions must come to mind:

  • How do I integrate new analytic capabilities and data sets to my current on-premises data warehouse environment?
  • How do I deliver self service solutions to accelerate the analytic process?
  • How do I leverage commodity hardware to lower costs?

For these questions, and more, organizations are architecting hybrid data warehouses.  In fact, these organizations moving towards hybrid are referred to as ‘Best In Class’ according to The Aberdeen Group’s latest research: “Best In Class focus on hybridity, both in their data infrastructure and with their analytical tools as well.  Given the substantial investments companies have made in their IT environment, a hybrid approach allows them to utilize these investments to the best of their ability while explore more flexible and scalable cloud-based solutions as well.”  To hear more about these ‘Best In Class’ organizations, watch the 45 minute webcast.

How do you get to this hybrid data warehouse architecture with the least risk and most reward?  IBM dashDB delivers the most flexible, cloud database services to extend and integrate with your current analytics and data warehouse environment, addressing all the challenges related to leveraging new sources of customer, product, and operational insights to build new applications, products, and business models.

To help our clients evaluate hybrid data warehouse solutions, Harvard Research Group (HRG) provides an assessment of IBM dashDB.  In this paper, HRG highlights product functionality, as well as 3 uses cases in Healthcare, Oil and Gas, and Financial Services.   Security, Performance, High Availability, In-Database Analytics, and more are covered in the paper to ensure future architecture enhancements optimize IT rather than adding new skills, complexities, and integration costs. After reading this paper, you will find that dashDB enables IT to respond rapidly to the needs of the business, keep systems running smoothly, and achieve faster ROI.

To know more on dashDB check out the video below:

 

About Mona,

mona_headshotMona Patel is currently the Portfolio Marketing Manager for IBM dashDB, the future of data warehousing.  With over 20 years of analyzing data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at IBM, a leader in data warehousing and analytics.  Mona received her Bachelor of Science degree in Electrical Engineering from UCLA.

IBM dashDB Local FAQ

When it comes to next-generation data warehousing and data management, the IBM dashDB family offers a range of options that all share a common technology.  dashDB Local is one of the new offerings and this blog provides a series of answers to your questions!  Please feel free to jump in at the bottom in the comments to add your own questions and we will respond.

      1.  What is the dashDB family?
        IBM dashDB is a family of next generation of database and data warehouse technologies that help you respond very quickly to application needs.  Originally in data warehousing, IT professionals assembled hardware software and storage to handle their large data sets for analytics needs.  This was risky, costly and time consuming.  This gave way to the data warehouse appliance that provided an optimized system for data warehousing and analytics.  The appliance was so successful that many consider it to be the backbone of their analytics architecture.But the world of analytics is expanding and new technologies are needed to handle more requests, more data sources and even self-service needs.  Hybrid data architectures are coming to the forefront to handle these increased needs. The dashDB family plays a key role here:

        • dashDB for analytics – as a fully managed cloud data warehouse
        • dashDB Local – as a configured data warehouse delivered via container technology to enable flexible deployment
        • dashDB for transactions – as a fully managed database as a service for transactional workloads.

        This family is designed to help you respond to new needs very quickly.  It also shares a common engine to help you leverage the same skills across different deployment models and application types. For more information on dashDB, visit dashDB.com.

      2. What is dashDB Local?
        dashDB Local is in-memory, columnar data warehousing software, supporting wide range of analytic workloads—from datamarts to enterprise data warehouses. It is deployed using Docker container technology, supporting a software defined environment such as private cloud, virtual private cloud or infrastructure of your choice, thus enabling hybrid Cloud configuration. dashDB Local can be deployed in minutes—making it fast and easy to deliver an auto configured data warehouse with built-in Netezza and Oracle compatibility.
      3. What is Docker?
        Docker is container technology that simplifies packaging and distribution of the software in a complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.
      4. What is the difference between Docker and VMware?
        VMWare is virtualization technology, while Docker is the container technology used for simplified packaging and distribution of software.   While Docker Container provides operating system-level process isolation, VMware virtualization lets you run multiple virtual machines (VMs) on a single physical server (thus providing H/W level abstraction). Unlike VMware, Docker does not create an entire virtual operating system (thus making it lightweight to deploy and faster to start up compared to VMs). Both technologies can be used together, for example, Docker containers can be created inside VMs to make a solution ultra-portable.
      5. Is dashDB Local generally available?
        Yes it is. You can find a free trial of it at ibm.biz/dashDBLocal.
      6. On which platforms is dashDB Local supported?
        Today, dashDB Local runs on any platform that Docker engine is supported such as Linux, Microsoft Windows, Apple Macintosh, and Cloud providers. More details can be found here and well as on the Docker site.
      7. On what platforms is dashDB Local supported?
        dashDB Local software is packaged and deployed using Docker container technology. Thus, dashDB Local can be installed on any platform where Docker engine client is supported. This includes Windows, Macintosh and variety of Linux platforms. Deploying IBM dashDB Local on Windows or Macintosh requires a Linux VM, in which you run Docker. By downloading the Docker Toolbox, which includes a GUI and a VM, you can accomplish this easily. Please refer to the dashDB Local knowledge center (documentation) for further details.
      8.  Is dashDB Local available on IBM Bluemix Local?
        Bluemix Local is managed by IBM on a clients’ infrastructure. There are no plans to offer dashdB Local on Bluemix Local at this time.
      9.  How long does it take to install dashDB Local?
        dashDB Local is based on Docker container technology, which allows setup and installation in less than 30 minutes. Today, the SMP version or MPP version of dashDB Local can be installed in less than 15 minutes.  This can be done on a range of servers from a simple laptop to a production-grade server. Since various components, such as LDAP security and DSM monitoring, are already bundled into a single container installation, there is a tremendous time savings and a more streamlined and efficient process.
      10. What  components are packaged and installed with  dashDB Local?
        dashDB Local is comprised of the following software components packaged in the Docker container, thus simplifying deployment and speeding up the overall setup. At the core is the dashDB analytics engine, tuned for columnar and in-memory workloads, built-in Netezza and Oracle compatibility, an LDAP server for user access management.  IBM Data Server Manager, which acts as the key monitoring component, is also included to provide key features such as query history monitoring, database performance monitoring and OS level monitoring.
      11. Is dashDB local available outside of Docker public hub?
        Soon, a standalone version of dashDB Local will be available on IBM Fix Central and will  leverage the Docker download command on the host OS. This will remove the dependency of pulling the dashDB Local image from the private, access-controlled repository on the public Docker hub. This will also alleviate challenges where a firewall port to access the public docker hub cannot be opened.
      12. What are the minimum prerequisites to install dashDB Local?
        You can find the documented prerequisites for dashDB Local in the Knowledge Center.  Some of the key requirements focus on Docker client and POSIX-compliant storage files systems, as documented. An additional key requirement revolves around opening access to the following network ports or on the firewall. Ensure that the following ports are opened and defined on all nodes in the cluster and defined in each node’s /etc/hosts file.60000-60024, for database FCM
        25000-25999, for Apache Spark
        50022, for SSH/container OS
        50001, for database connection with SSL
        50000, for database connection without SSL
        9929, for communication tests
        9300, for web console status
        8443, for web console HTTPS
        5000, for System Manager
        389, for LDAP
        22, for SSH/host OS
      13. How does Apache Spark fit into the dashDB Local architecture?
        dashDB Local lets you dramatically modernize your data warehouse solutions with advanced analytics based on Spark. It is installed and configured within the dashDB Local container, thus making it fully integrated, supporting a variety of use cases.Spark applications that process relational data can gain significant performance and operational QoS benefits from deploying and running inside dashDB Local. It enables end-to-end analytic solution creation, from interactive exploration and machine learning experiments; verification of analytic flows; easy operationalization of Spark applications through to hosting Spark applications in a multi-tenant enterprise warehouse system; and integration of Spark applications with other applications via various invocation APIs. It allows you to invoke Spark logic via SQL connections and can land streaming data directly into tables via deployed Spark applications. It can run complex data transformations and feature extractions that cannot be expressed with SQL using integrated Spark.
      14. I have a running dashdb local instance where I’ve set DISABLE_SPARK=‘YES’ in the options file. How can I tell if this option took affect and the actual memory the db is using?This can be confirmed during the startup of dashDB Local container. When Spark is disabled, you will see in the docker start output “Spark support is going to be disabled.” When Spark is enabled you will see”Current spark share : XX% of total memory. “
      15. Is the SPARK setting available as an install time switch, or can I enable and disable spark every time I start the database by changing this?
        The SPARK feature can be enabled/disabled any time. You can change the DISABLE_SPARK setting anytime again and just restart dashDB local in the container (docker exec -it dashdb stop/start).
      16. Is there currently a maximum number of nodes for dashDB Local?
        Currently, one can create up to 24 node MPP cluster in dashDB Local.
      17. Can I upgrade from the GA trial to the production version of dashDB Local?
        Yes, you can upgrade/update the post-GA trial version to a production grade version. Once the product is purchased, a permanent license can be applied to update the license.
      18. How can I get support for dashDB Local ?
        Upon purchase of dashDB Local, clients are entitled to support via email or phone. One can open a PMR with the IBMs support ticketing system. IBM will support the dashDB Local container and all of the components inside it. For Docker specific issues, clients should contact the Docker support team. The IBM support team will assist in identifying the problem and advise accordingly.
      19. If I run into docker issue, would IBM support handle it for me?
        IBM will support the “dashDB Local container”, but not the Docker engine itself. It is the client’s responsibility to subscribe to Docker support.  Customers can leverage Docker CS engine (from Docker) or Open source docker RPMs that come with the Linux distros (such as Red Hat). Docker will provide commercial support only for Docker CS engine and not for Docker RPMs and it is the customer choice of a relevant support path regarding docker components.
      20. How often are dashDB Local updates/fix packs made available?
        dashDB Local is based on an agile development cloud model and the intent is to roll out container updates frequently. This will not only make it easier to stay current with the latest bug fixes, and newest features. The updates are handled via container update process and will take less then 30 minutes, similar to the container setup.
      21. Can dashDB Local be installed on Amazon AWS or Azure/ AWS ?
        dashDB Local can be installed on the infrastructure of your choice, as long as that infrastructure supports Docker container technology. dashDB Local is client-managed and can also be installed in your data center and any virtual private cloud infrastructures such as Amazon AWS EC2 or Microsoft Azure platforms.
      22. What kind of storage is required for dashDB Local?
         dashDB Local requires a posix-compliant clustered file storage system. This is applicable for the MPP cluster only. For a standalone SMP installation, this is not a requirement. You can use standard local disks for a SMP dashDB local node setup.Cluster file system is a file system that is configured in a way to group servers and resources together to have concurrent access to a single file system. The key to a cluster file system is that the cluster appears as a single highly available system to all the end users. This increases the storage utilization rate and can result in high performance.
        Some common examples of clustered file storage system are :
        – VERITAS Cluster File System(VxFS) Sun Solaris, HP/UX
        – Generalized Parallel File System (GPFS) IBM AIX, Linux
        – GFS2 Red Hat only
      23. Is Oracle compatibility available in dashDB local?
        Yes, Oracle compatibility is supported in dashDB Local. You can enable applications that were written for an Oracle database to use dashDB™ Local without having to be rewritten. To use this capability, you must specify that dashDB Local is to run in Oracle compatibility mode prior to initial deployment.Before you begin, the /mnt/clusterfs/ directory must already be created. To perform this task, you need to have root authority on the host system OS. By default, Oracle compatibility mode is not enabled. To enable it, you explicitly make an entry in the /mnt/clusterfs/options file prior to deploying dashDB Local. Run the command below and then follow the normal steps around container deployment/initializationecho “ENABLE_ORACLE_COMPATIBILITY=’YES'” >> /mnt/clusterfs/options

        For more details:
        http://www.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.swg.im.dashdb.doc/admin/local_oracompat.html
        http://www.ibm.com/support/knowledgecenter/SS6NHC/com.ibm.swg.im.dashdb.doc/admin/local_setup.html#setup

 

Three session guides get you started with data warehousing at IBM Insight at World of Watson

Join us October 24 to 27, 2016 in Las Vegas!

by Cindy Russell, IBM Data Warehouse marketing

IBM Insight has been the premiere data management and analytics event for IBM analytics technologies, and 2016 is no exception.  This year, IBM Insight is being hosted along with World of Watson and runs from October 24 to 27, 2016 at the Mandalay Bay in Las Vegas, Nevada.  It includes 1,500 sessions across a range of technologies and features keynotes by IBM President and CEO, Ginni Rometty; Senior Vice President of IBM Analytics, Bob Picciano; and other IBM Analytics and industry leaders.  Every year, we include a little fun as well, and this year the band is Imagine Dragons.

IBM data warehousing sessions will be available across the event as well as in the PureData System for Analytics Enzee Universe (Sunday, October 23).  Below are product-specific quick reference guides that enable you to see at a glance key sessions and activities, then plan your schedule.  Print these guides and take them with you or put the links to them on your phone for reference during the conference.

This year, the Expo floor is called the Cognitive Concourse, and we are located in the Monetizing Data section, Cognitive Cuisine experience area.  We’ll take you on a tour across our data warehousing products and will have some fun as we do it, so please stop by.  There is also a demo room where you can see live demos and engage with our technical experts, as well as a series of hands-on labs that let you experience our products directly.

The IBM Insight at World of Watson main web page is located here.  You can register and then use the agenda builder to create your personalized schedule.

IBM PureData System for Analytics session reference guide

Please find the session quick reference guide for PureData System for Analytics here: ibm.biz/wow_enzee

Enzee Universe is a full day of dedicated PureData System for Analytics / Netezza sessions that is held on Sunday, October 23, 2016.  To register for Enzee Universe, select sessions 3459 and 3461 in the agenda builder tool.  This event is open to any full conference pass holder.

During the regular conference, there are also more than 35 PureData, Netezza, IBM DB2 Analytics Accelerator for z/OS (IDAA) technical sessions across all the conference tracks, as well as hands on labs.  There are several session being presented by IBM clients so you can see how they put PureData System for Analytics to use.  Click the link above to see the details.

IBM dashDB Family session reference guide

Please find the session quick reference guide for the dashDB family here: ibm.biz/wow_dashDB

There are a more than 40 sessions for dashDB, including a “Meet the Family” session that will help you become familiar with new products in this family of modern data management and data warehousing tools.  There is also a “Birds of a Feather” panel discussion on Hybrid Data Warehousing, and one that describes some key use cases for dashDB.  And, you can also see a demo, take in a short theatre session or try out a hands-on lab.

IBM BigInsights, Hadoop and Spark session reference guide

Please find the session quick reference guide for BigInsights, Hadoop and Spark topics here: ibm.biz/wow_biginsights

There are more than 65 sessions related to IBM BigInsights, Hadoop and Spark, with several hands on labs and theatre sessions. There is everything from an Introduction to Data Science to Using Spark for Customer Intelligence Analytics to hybrid cloud data lakes to client stories of how they use these technologies.

Overall, it is an exciting time to be in the data warehousing and analytics space.  This conference represents a great opportunity to build depth on IBM products you already use, learn new data warehousing products, and look across IBM to learn completely new ways to employ analytics—from Watson to Internet of Things and much more.  I hope to see you there.

Start Small and Move Fast: The Hybrid Data Warehouse

by Mona Patel

In the world of cutting edge big data analytics, the same obstacles in gaining meaningful insight still exists – ease of getting data in and getting data out.  To address these long standing issues, the utmost flexibility is needed, especially when layered with the agile needs of the business.

Why spend millions of dollars replacing your data and analytics environment with the latest technology promise to address these issues, when can you to leverage existing investments, resources, and skills to achieve the same, and sometimes better, insight?

Consider a hybrid data warehouse.  This approach allows you to start small and move fast. It provides the best of both worlds – flexibility and agility without breaking the bank.  You can RAPIDLY serve up quality data managed by your data warehouse, blended with newer data sources and data types in the cloud, and apply integrated analytics such as Spark or R – all without additional IT resources and expertise.  How is this possible?  IBM dashDB.

Read Aberdeen’s latest report on The Hybrid Data Warehouse.

mona's blog

 

Watch Aberdeen Group’s Webcast on The Hybrid Data Warehouse.

Let me give you an example.  We live in a digital world, with organizations now very interested in improving customer data capture across mobile, web, IoT, social media, and more for newer insights.  A telecommunications client was facing heavy competition and wanted to quickly deliver unique mobile services for an upcoming event in order to acquire new customers by collecting and analyzing mobile and social media data.  Taking a hybrid data warehouse approach, the client was able to start small and move fast, uncovering new mobile service options.

Customer information generated from these newer data sources were blended together with existing customer data managed in the data warehouse to deliver newer insights.  IBM dashDB provided a high performing, public cloud data warehouse service that was up and running in minutes.  Automatic transformation of unstructured geospatial data into structured data, in-memory columnar processing, in-database geospatial analytics, integration with Tableau, and pricing were some of the key reasons IBM dashDB was chosen.

This brings me back to my first point – you don’t have to spend millions of dollars to capitalize on getting data in and getting data out.  For example, clients like the one described above took advantage of Cloudant JSON document store integration, enabling them to rapidly get data into IBM dashDB with ease– no ETL processing required.  Automatic schema discovery loads and replicates unstructured JSON documents that capture IoT, Web and mobile-based data into a structured format.  Getting data or information out was simple, as IBM dashDB provides in-database analytics and the use of familiar, integrated SQL based tools such as Cognos, Watson Analytics, Tableau, and Microstrategy.  I can only conclude that IBM dashDB is a great example of how a highly compatible cloud database can extend or modernize your on-premises data warehouse into a hybrid one to meet time-sensitive business initiatives.

What exactly is a hybrid data warehouse?  A hybrid data warehouse introduces technologies that extend the traditional data warehouse to provide key functionality required to meet new combinations of data, analytics and location, while addressing the following IT challenges:

  • Deliver new analytic services and data sets to meet time-sensitive business initiatives
  • Manage escalating costs due to massive growth in new data sources, analytic capabilities, and users
  • Achieve data warehouse elasticity and agility for ALL business data

mona_dashDB

Still not convinced on the power of a hybrid data warehouse?  Hear what Aberdeen Group’s expert Michael Lock has to say in this 30 min webcast.

About Mona,

mona_headshot

Mona Patel is currently the Portfolio Marketing Manager for IBM dashDB, the future of data warehousing.  With over 20 years of analyzing data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at IBM, a leader in data warehousing and analytics.  Mona received her Bachelor of Science degree in Electrical Engineering from UCLA.

IBM dashDB Local preview to be featured at Cloud Expo East 2016 in New York

by Cindy Russell, IBM Data Warehouse Marketing

With an expected attendance of over 6,000 from June 7th through 9th, Cloud Expo New York is the one and only original event in which technology vendors can meet to experience and discuss the entire world of the Cloud.

On June 7, 2016, we are thrilled to be rolling out IBM dashDB Local™ as an open preview.  This exciting new IBM hybrid data warehouse offering is part of the dashDB family of data management solutions. dashDB Local delivers dashDB technology via container technology for implementations such as private and virtual private clouds. It is an ideal solution when you want cloud-like simplicity, yet need more control over applications and data. Participants in this preview program have been very enthusiastic about this technology, and you can read more in Mitesh Shah’s blog.

dashDB Local is:

·        Open: Leverage the power of an open warehouse platform

·        Flexible & Hybrid: Easily deploy the right workload to the right platform

·        Fast: Quickly realize business outcomes with advanced processing technologies

·        Simple: Automated management features help to lower the analytics cost model

 

dashDB Local will also be featured in this speaking session at Cloud Expo!

Building Your Hybrid Data Warehouse Solution with dashDB

Matthias Funke, Hybrid Data Warehouse Offering and Strategy Lead at IBM

June 7, 2016 from 11:40 AM – 12:15 PM

 

See the dashDB family in action at IBM Booth #201, the SoftLayer Booth!

cloud expo booth

Register to attend

IBM Booth #201

June 7, 2016

Cloud Expo East 2016

Javits Center, New York, NY

Note: First time attendees will need to create a new account.

As you respond to increasing requests for new analytics, you need fast and flexible technology. Learn how dash DB’s cloud-based managed service and new container-based edition gives you the speed and flexibility that you need.

Want to get started now with dashDB Local? Learn about and download our preview here: ibm.biz/dashDBLocal