Becoming a data-driven organization is not something that can be done in a fortnight. It’s a long and possibly tedious process and certainly not just a matter of buying and installing certain tools. No, even though investments in new technology may be required, the organizational chart may also need to be changed, along with business processes, roles and responsibilities, and management styles. Becoming data-driven is all-encompassing, but the potential benefits can be worth it.
To become data-driven, organizations must ensure that as many data consumers as possible can benefit from all of the data collected over the years and that the data is used to improve and optimize both business and decision-making processes. This means that data consumers must get easier and faster access to data. This is not a performance issue, but a dedication to ensuring that if a need for data exists, it will be made available more quickly than before.
This requires a highly flexible data architecture that supports at least the following eight features:
1. Support for “ad-hoc reporting”: This may be an old-fashioned concept, but it is very relevant for the data-driven organization. Unexpectedly, management may need new reports to make quick decisions due to a sudden opportunity or urgency. Instead of making such a decision purely on a hunch, it should be based on the right data. This requires that data from multiple systems can be combined quickly and easily for analysis.
2. A consistent view of data: Wherever and whenever data is used by data consumers, and regardless of the individual consumer, it should represent the same state of the data. For example, sales data shown with a simple spreadsheet, an advanced dashboard, or a Java app should all be consistent.
3. 360 degree views of business objects: Whether the data consumer is a mobile app used by hundreds of online customers, or a spreadsheet used by top-level managers, full 360-degree views of business objects, such as customers, patients, trucks, or factories, should be available to all of them.
4. Centralized specification of data privacy rules: Data privacy rules must be implemented as centrally as possible, rather than being scattered across hundreds of systems, applications, databases, and reports. Organizations cannot afford data management and data availability blunders.
5. Zero latency data: The group of data consumers that requires low-latency data keeps increasing. This implies that a data architecture in which data is copied multiple times from one database to another is out of the question. A new data architecture needs to be able to present at least near-real-time data.
6. Support for new data processing technologies: Especially in the last ten years, many powerful, fast technologies for data storage and analytics have become available. When they offer benefits, a new data architecture should offer plug-and-play capabilities, enabling such technologies to be easily plugged in and used. New technologies can mean new business opportunities.
7. Access to master data: The use of master data should not be restricted to the happy few, but to every data consumer. Master data must be a fully integrated part of the data architecture.
8. Access to descriptive metadata: Metadata is crucial for understanding data. Storing and managing metadata is useful, but the challenge is to make it as easily available to data consumers as actual data. The easier it is to access metadata, the more transparent our systems will be to users, and trust in the reports will grow.
Becoming Data-Driven
Each of these features fit data virtualization like a glove. Data virtualization may not be the entire solution. By itself, it will not make an organization data-driven. But it can provide each of the critical features above, and that will be a great start. Data virtualization can act as the gateway of the data architecture to all the data hidden in transactional systems (cloud and on-premises), data warehouses, data lakes, or flat files. It can help to easily unlock all of the data, helping people to use it more consistently. It can also deliver metadata and master data, show near-real-time data, and exploit new technologies. It can help organizations in their process to use data more widely, efficiently, and effectively to improve and optimize their business and decision making processes.
- Metadata, the Neglected Stepchild of IT - December 8, 2022
- Zero-Copy Architectures Are the Future - October 27, 2022
- The Data Lakehouse: Blending Data Warehouses and Data Lakes - April 21, 2022