One constant trend of society is its continuous evolution into more intricate systems, with processes, markets, products and consumer needs becoming more and more complex over time. A common strategy for organizations to handle this increasing complexity has been specialization: dividing difficult problems into smaller tasks and solving them with domain-specific techniques that are optimal for those particular realms.
Moving away from “one-size-fits-all” technologies
This is no different in Enterprise IT. In the last few years we’ve seen how a whole new set of technologies has taken the field by storm: Big Data/Hadoop/Spark systems, document-oriented databases, graph databases, streaming data processing, search engines, in-memory grids, etc. By adopting these technologies, organizations are moving away from classic “one-size-fits-all” platforms, and embracing specialized technology to solve specific problems.
And when dealing with such complexity of needs, specialization is arguably the way to go. However, this approach can easily transform IT stacks in a patchwork of technologies that are very difficult to integrate, making it very challenging to consume data across the enterprise, and effectively creating multiple “new technology” silos.
When the solution becomes a problem
The adoption of specialized technologies for specific domain problems, along with different business units having different needs and moving in different directions, can disrupt organizations to the point where data integration becomes a critical issue:
1. Multiple data silos – Data is spread across many different stores, with many different formats and interfaces. There’s no way to realize value out of all of these data without having to replicate it over and over.
2. Need for highly specialized skills – As the diversity of technology grows, more and more very specific skillsets are required, driving up costs and making the organization very inflexible to change.
3. Lack of coherent governance and security – Very different systems provide very different capabilities around security, metadata management and policies, making it difficult to govern and secure data enterprise-wide.
The adoption of new technologies for specific domain problems and new data needs is necessary as part of the natural evolution of organizations, but the results are very complex.
It results in the creation of diverse ecosystems that traditional data integration technologies struggle to deal with.
Data virtualization offers a solution to this problem by implementing an abstraction layer that integrates all of these disparate systems and presents them in a unified manner, providing usable interfaces to access their data in real time.
Enabling best-of-breed strategies through data virtualization
Data virtualization solutions like the Denodo Platform, as a next generation integration technology, help address complex data integration needs, including the ones tied to the proliferation of specialized systems described above:
1. Unified data access layer – The virtualization layer connects to all the source systems and integrates them in virtual data models. Users can access these data models and transparently consume data across all sources, overcoming the data silo limitations.
2. No need for specialized skills – Denodo Platform works as an abstraction layer that hides the technicalities of the data sources to the consumers. Data can be accessed through this platform without any specialized knowledge, using common enterprise interfaces.
3. Centralized governance and security – Virtualization allows the enforcement of centralized security and governance model, regardless of the maturity and capabilities of the source systems.
Data virtualization allows organizations to overcome the data integration issues that arise from specialization as they grow larger and complex. However, virtualization should not be seen only as a cure for these symptoms, but as an enabler for specialization as a key enterprise strategy.
Platforms like Denodo promote technology specialization, empowering organizations to adopt the best solution available for every specific problem or domain, knowing that they will be able to easily integrate such solution as part of their ecosystem through data virtualization. And as many of our customers’ success stories can corroborate, the adoption of best-of-breed specialized technologies and their integration through data virtualization is a solid strategy for achieving better solutions, lower costs and faster response times.
- Making Sense Of A Patchwork Of Technologies - May 24, 2017