Using Data Virtualization to Simplify Microservices Architecture

Reading Time: 3 minutes

In a previous post, we reviewed the value of APIs and how the data virtualization layer can be used as a data service provider. Another powerful way to use data virtualization and data services is to use the data virtualization layer as an integration layer for microservices. In this post, I will explore the benefits of using microservices to build applications, and how data virtualization can simplify a microservices architecture.

The Benefits and Challenges of Microservices

Microservices enable you to build an application from small, modular components that are typically exposed as RESTful web services. Because such applications are built of several microservices, instead of a single component, each component can be easily replaced, scaled, or changed without affecting the others.

This can provide significant benefits. For example, using microservices greatly reduces the risk of upgrading or changing a component. When a change is deployed, a single component is affected instead of the entire application. Microservices also make it possible to easily scale up selected portions of an application. An online retailer may experience increased traffic on one part of the company’s website during a yearly sale. If the entire site is deployed as a single component, the retailer would need to scale up the entire application. If it is made up of a collection of microservices, only a portion of the application would need to be scaled up to meet the increased demand.

While this approach has clear benefits, it also adds significant complexity to the architecture.

To take advantage of microservices benefits, each microservice should be as independent as possible, so dependencies between microservices should be minimized. Typically, this means that each microservice should have an independent data source so that changing the schema of one microservice will not affect the behavior of the others.

While this makes sense theoretically, it can be challenging to implement in practice. Splitting databases may be logically difficult and can leave data silos.

After the data is split, you might need to join the data back together to provide a more complete view of the data. For example, if customer and order information is stored in different databases, it is easy to imagine a scenario where the data needs to be joined. This can result in the complicated practice of copying data from one database to another in order to form complete data sets. In addition to being complicated, it can also be time-consuming and lead to data inconsistency across the organization.

How Can Data Virtualization Help?

A key component of data virtualization is the ability to decouple your physical data from the logical representation. With the introduction of a data virtualization layer, each microservice can use a virtual view to expose a data API. This decouples the data API from the physical API behind the scenes.

As a result, microservices can share the same physical database, and still maintain their logical independence. If the underlying source changes, the logical representation defined in the data virtualization layer stays the same. Similarly, an API built in your logical layer can be changed without affecting other microservices sharing the same physical data.

On the other hand, if the data for each microservice is stored in different physical locations, there will be instances when the data needs to be combined together. The data virtualization layer allows you to easily combine data coming from different physical locations in real time, eliminating the need to physically move the data and create new repositories just for the purpose of joining data.

In either case, data virtualization will make it easier to create and reuse microservices that contain complete sets of data, while still maintaining their independence. Using this approach, developers will be able to take advantage of the benefits and flexibility of microservices, with simplified data access and combination.

A Robust Strategy

The use cases described in this and my colleague’s post, “Building Quick, Robust, and Flexible APIs with Data Virtualization” highlight two scenarios where data virtualization can be used in API initiatives.

Organizations using data virtualization as part of their API development strategy will be able to integrate real-time data from multiple locations and take advantage of easy-to-use data-service publishing options.

With the combination of these and many other features, data virtualization can play a critical role in data services and API management architectures.