By Rich Hughes,
Scanning the archives as far back as 2000 reveals articles speculating on the future of the DBA. With mounting operational costs attributed to the day-to-day maintenance of data warehouses, even 15 years ago, this was a fair question to ask. The overhead of creating indexes, tuning individual queries, on top of the necessary nurturing of the infrastructure had many organizations looking for more cost effective alternatives.
Motivated to fix the I/O bottleneck that traditionally handicapped data warehouses, and inspired by the design goals of reduced administration and easy data access for users, the data warehouse appliance was born. Netezza built the original data warehouse appliance that by brilliantly applying hardware and software combinations, brought the query request much closer to the data. This breakthrough paved the way for lower administrative costs and forced others in the data warehouse market to think of additional ways to solve the I/O problem.
To be sure, Netezza disruptive technology of no indexing, great performance, and ease of administration, left many DBAs feeling threatened. But what was really threatened was the frustrating and never ending search for data warehouse performance via indexing. Netezza DBAs got their nights and weekends back, and adjusted by making themselves more valuable to their organizations by using the time saved with no-indexing to get closer to the business. Higher level skills taken on by DBAs included data stewardship and data modeling, and in this freer development environment, advanced analytics took root. In the data warehouse appliance world, much more DBA emphasis was placed on the business applications because the infrastructure was designed to run for the most part, unassisted.
Fast forward to current day where the relentless pursuit of IT cost efficiencies while providing more business value continues. Disruptive technologies in the past decade have been invented to fill this demand, like the Hadoop Ecosystem and the maturing Cloud computing environment. Hardware advances have pushed in-memory computing, Solid State Drives are in the process of phasing out spinning disk storage, and 128 bit CPUs and operating systems are on the drawing boards. Databases like IBM’s dashDB have benefitted by incorporating several of these newer hardware and software advances
So 15 years into the new Millennium what’s a DBA to do? Embrace change and realize there is plenty of good news and much data to administer. While the Cloud’s Infrastructure and Platform services will decrease on-premise DBA work over time, the added complexity will demand new solutions for determining the right mixture of on, off, and hybrid premise platforms. Juggling the organizational data warehouse work load requires different approaches if the Cloud’s elasticity and cheaper off-hour rates are to be leveraged.
Capacity planning and data retention take on new meaning in a world where, while it is now possible to store and access everything, what is the return value of all that information? The DBA will be involved in cataloging the many new data sources as well as getting a handle on the unstructured data provided by the Internet of Things. Moving data, when to move data, to persist or not, how does this data interact with existing schemas are all good questions to be considered for the thoughtful DBA. And that is just on the ingest side of the ledger. Who gets access, what are the security levels, how can applications be rapidly developed, how does one re-use SQL in a NoSQL world, and how to best federate all this wonderful data are worthwhile areas for reasonable study.
In summary, the role of the Database Administrator has always been evolving, forced by technology advances and rising business demands. The DBA has and will continue to be one that requires general knowledge of several IT disciplines, with the opportunity to specialize. Historically the DBA, by keeping current, can go deeper in a particular technology– a move that benefits both their career and their organization’s needs. The DBA can logically move into an architecture or Data Scientist position, the higher skill sets for today’s world. What has not changed is the demand to deliver reliable, affordable, and valuable information.
About Rich Hughes,
Rich Hughes is an IBM Marketing Program Manager for Data Warehousing. Hughes has worked in a variety of Information Technology, Data Warehousing, and Big Data jobs, and has been with IBM since 2004. Hughes earned a Bachelor’s degree from Kansas University, and a Master’s degree in Computer Science from Kansas State University. Writing about the original Dream Team, Hughes authored a book on the 1936 US Olympic basketball team, a squad composed of oil refinery laborers and film industry stage hands. You can follow him on @rhughes134