With the advent of digital transformation, the fluidity of data has evolved from slowly changing dimensions to frequently changing data sources and systems. Data pipes have not only become fat and fast, they have also become varied and diverse in terms of structures and interfaces, with a good amount of overlapping scope and synergy.
If you work within a progressively growing business, you most likely engage with multiple marketing agencies, multiple sales partnerships, numerous lead generation sources, several logistics partners, different partners that handle warranty and returns in each region, multiple supply chain partners, and a few manufacturing partners. If you are not there now, you might be moving in that direction.
This is a direct result of our networked economy, which lends itself to an increasing number of partnered and collaborative opportunities. It is also a result of changing business models, brought about once again by that same digital transformation that is sweeping every single domain.
Partnership, collaboration, outsourcing, and leveraging from the ecosystem are all part of the new norm. As new partnerships and new collaborations get formed, outsourcing and leveraging the fast, emerging ecosystem, are mutually beneficial. These partnerships or collaborations are based on specific goals, time-bound projects, or initiatives rather than everlasting enterprise commitments that are forged over the years or quarters. Accordingly, the underlying data platforms/systems could vary based on the requirements of each engagement. Existing data platforms could move from open source to proprietary platforms, or from one proprietary database to another, such as from PostgreSQL to Oracle, or from Oracle to Azure Cloud. As this progression happens, the data is invariably changing, in terms of the source, structure, and interface. The data will also change in terms of the underlying hosting infrastructure that provides it, such as cloud, SaaS, on-prem, or outsourced, and also in terms of timeframe, such as real-time vs near-real-time vs extended time vs historical.
This is the World of Fluid Data
In this world, sources and systems can rapidly change. New variations in data interfaces are the norm rather than the exception. The originator of a dataset could change from one entity to another (In the case of a new partner or a new service provider, say with the CRM of one business unit in Salesforce and another in Siebel or Zoho). The same data might be provided by multiple sources and streams (such as multiple business partners in a given functional area such as logistics, lead generation, contract manufacturing, etc.)
Data pipes have been growing more heterogeneous in structure and more varied in format, while offering a multiplicity of options for distribution. Because the overall quality of the data is only as high as the lowest-quality data in the chain, companies often find it challenging to derive value from this rich data diversity.
Welcome to the reality of fluid data. To leverage the power of this rich but fluid data, businesses need to turn this data complexity into a business advantage. Enterprises need to be able to receive, integrate, and assimilate these diverse, fluid, internal and external data streams, in as close to real time as possible. And businesses must be able to quickly respond to market realities with the appropriate actions, leveraging plug-and-play capabilities and cloud scaling advantages.
Robert Metcalfe’s Network Law, that “the value of a network grows by the square of the size of the network,” holds true for a data network as well. The richness of data non-linearly increases the data-advantage for businesses. For this reason, ignoring the reality of the world of fluid data, or ignoring the new normal, is simply not an option for a business that aspires to survive and grow.
This calls for some new ammunition in the enterprise arsenal. The enterprise needs to be data agile and shrink time-to-respond across the board, be it in terms of customer service responsiveness, component traceability in the manufacturing supply chain, sales team performance, or partner reliability.
The ability to manage and leverage fluid data systems is vital for business survival. The enterprise ability to deliver specific persona based data interfaces for deeper insights is necessary for business sustenance and growth. These data interfaces could include real-time API interfaces, message packets, and specific alerts, or they could be full-fledged dashboards with actionable insights.
Fast Provisioning
We term the ability to provide consumable data interfaces to heterogeneous data sources of various formats, across global locations, as the “fast provisioning” of data. The ability to “fast provision” data interfaces, with minimal change, while keeping the defined downstream interfaces functional and stable, is key to business survival and growth.
Building fast provisioning capabilities within the enterprise calls for a combination of tools, technologies, data modelling strategies, architectures and designs. Organizations and enterprises that enable fast provisioning will create a moat of competitive advantage around themselves as well as a huge information advantage. Enterprises that embrace the requisite tools that enable fast provisioning will gain an execution advantage.
COMPEGENCE, a Denodo partner, offers a compelling set of solutions and technologies that enable this fast provisioning moat for your enterprise, at the speed of your business. Reach out to us info at compegence dot com if you would like us to evaluate the fast provisioning readiness of your organization, or if you would like to learn how to increase the speed-of-response of your business.
His prior roles include over 25 years of rich experience with companies such as Hansa-Cequity (India), Intel technologies (India), Sun Microsystems (US and India), PricewaterhouseCoopers (US), Indigo technologies (US), TCS (India). He is an Alumni of IISc, Bangalore and UVCE Bangalore.