My entire career has been devoted to data and database management. As is often the case in the world of technology, given enough time things are going to change sometimes significantly. And, indeed, things have changed in the realm of the DBMS… quite a lot, in fact.
In the early days when I started in IT, IMS was the primary database system used at most big enterprises and most of the computing was done on mainframe systems. And yes, IMS is still used, though not as pervasively as it was back then. And yes, the mainframe is still in use, too.
After a bit, IBM Db2 rose up and became the dominant mainframe database system, and it still is today. But enterprise computing has grown to encompass much more than the mainframe. Things are much more heterogeneous today. Due to the advent of client/server computing in the 1990s, then internet and web-connectivity driving development, and the more recent growth of NoSQL databases, we are now manag¬ing more types of data on more platforms than we ever have before. And that means that managing data is a lot more complex.
Indeed, if we just look at the database systems and platforms tracked by DB-Engines, an independent site that tracks the popularity of database systems, we see that there are over 400 different products that are currently being tracked! And these systems include many different models and types of DBMS including relational DBMS, key/value stores, document stores, time-series DBMS, graph DBMS, object-oriented DBMS, RDF stores, native XML DBMS, navigational DBMS, event stores, and more. This myriad of database systems introduced more diversity and better support for specific types of use cases, but it also increases the complexity of managing and administering data.
One of the most significant changes has been the widespread adoption of cloud-based DBMS. Cloud providers offer not only the database in the cloud, but also managed database services, which enables organizations to offload their infrastructure management and instead focus on their core business. This shift has led to greater scalability, flexibility, and ease of deployment for database systems.
But it has also brought with it added complexity, at least for larger organizations. You see, most large enterprises have not moved all of their computing and database systems to the cloud, instead adopting a hybrid multicloud approach. With such an approach, organizations choose what makes sense for each component, task, and project that is implemented. This often means supporting a combination of on-premises (mainframe, Linux/Unix, Windows) and cloud platforms; potentially multiple different cloud platforms, too. As is so often the case, new technology and ideas do not supplant the old, but add to the old. And that increases complexity!
So, DBAs today need to possess exper¬tise in cross-platform management and administra-tion. “Cross-platform” implies both the computing environment, which can span the spectrum from mainframe to Linux to Windows to the cloud but also the DBMS in use, which can be RDBMSs such as Oracle, Microsoft SQL Server, IBM Db2, and PostgreSQL, as well as NoSQL knowledge and skills for platforms such as MongoDB, Cassandra, Neo4j, and many others.
To do that successfully requires automated solutions for change management, performance, and data movement—and really all DBA tasks. Automation is the key as we move forward for many reasons. As the amount of data we manage expands—and it is growing by leaps and bounds—we must be able to expand our capability to administer that data. But organizations are not adding to their DBA staff. So, if organizations expect to manage more data with the same (or fewer) resources, then automation is required to avoid failure.
And speaking of automation, I’d be remiss if I did not men¬tion the impact of DevOps on data management and DBAs. The processes performed by DBAs are part of the Ops in DevOps. It is my observation that Dev gets emphasized more than Ops when DevOps is implemented, but you can’t omit the Ops … at least not if you want to have functional, performing applications.
When DevOps is embraced by an organization, it should mean that DBAs get aligned more closely with development and appli¬cations than in the past. By deploying agile development, with DBAs participating in teams along with the developers, you get increased cooperation and communication between the folks coding the application (that’s Dev) and the folks developing and managing the database (that’s Ops, or the DBAs).
So, another overarching change has been that DBAs are work¬ing in teams with developers instead of in teams of other DBAs, at least for periods of time when development projects are very active. For some applications, a permanent DBA, or team of DBAs, may be assigned. For others, the DBA may rotate back and forth between the development team and a centralized DBA team.
Regardless of the pattern, DBAs are becoming more appli¬cation-savvy. That’s a good thing because with improved application knowledge the DBA will be better able to administer the database for an application’s needs as the app moves from devel¬opment to test to production.
Data Security and Privacy
Another area that has become more of a focus is protecting data in the DBMS. As data breaches and privacy concerns have become more prominent, DBMS vendors have placed a greater emphasis on security features. Encryption, access controls, and auditing capabilities have been enhanced to protect sensitive data. Additionally, compliance requirements, such as GDPR (General Data Protection Regulation), have influenced the design and functionality of DBMS to ensure privacy and data protection.
And this means that DBAs need to know more about the data protection capabilities of the DBMS. Instead of focusing on just the ease-of-use and performance-related improvements in a new DBMS release, many times the most important new capabilities are in the area of better protecting data in the database.
The rise of containerization technologies, such as Docker and Kubernetes, has influenced the deployment and management of DBMS, too. Containers enable faster and more efficient deployment, scalability, and resource utilization. DBMS are being designed to run in containerized environments, facilitating integration with microservices architectures and improving overall system agility.
Tooling and Automation
With all of these changes comes the need for improved tooling and automation. It is vital for organizations to recognize that there is a management discipline called database administration. And it is required in order to plan, model, implement, and manage database systems. It comprises a long list of important and required tasks that must be done to ensure effective and efficient databases are made available, as well as secure.
If we think about DevOps for a moment, it works for developing code because of all the integrated DevOps tooling that auto¬mates and simplifies the development pipeline. By this I mean approaches such as Jenkins for coordinating and orchestrating the software development lifecycle (SDLC), Ansible for configuration and resource management, GitHub for source control, Docker to containerize the platform, and many more. DBAs need the same type of tools to automate and integrate data management tasks into the DevOps pipeline. And no, it isn’t just using Git for Data Definition Language! It is more complex than that.
DBAs are being asked to do more… with larger amounts and more types of data… being accessed more rapidly and from more sources… without any prolonged downtime permitted… using and supporting new database types and capabilities… and with fewer DBAs as a percentage of IT staff than ever before. Automation is imperative in this current scenario.
Summing It Up
To adapt to the changing database landscape, and ensure that we have modern data management practices and procedures, DBAs must be flexible, adaptable, and resilient as they work in an inte¬grated way with a team. Today’s data professionals have to move faster and more nimbly than ever before.
DBAs need to understand the organization’s databases inside-and-out, but they also need appli¬cation knowledge, management and communication skills, and a diverse set of tools to automate DBA and system administration processes and procedures. And organizations need to support their growing data and database infrastructure with the appropriate tools and culture to ensure success.