BHP is a global resources company headquartered in Melbourne, Australia. It is among the world’s top producers of major commodities, including iron ore, metallurgical coal, and copper, and has substantial interests in oil and gas. BHP has operations and offices across Australia, Asia, the UK, Canada, the USA, and Central and South America. In this post, I will cover BHP’s experience with the Denodo Platform.
BHP’s New Data Strategy
BHP embarked on a new data strategy to transform itself into a data-centric organization and help the BHP management make better data-driven decisions. As Joshua Fletcher, Principal Solutions Architect at BHP, put it, “We recently put together a data strategy that could transform us. We wanted to improve the way data professionals accessed and consumed the information.”
As part of that goal, BHP created an asset-centric data architecture that provided data utility, plumbing, and wiring foundations for individual business units, accelerating the delivery of new digital solutions, and providing a data environment for data science and advanced analytics. BHP’s enterprise data spread across multiple continents, in both on-premises and cloud sources. The company had data teams running daily, weekly, and monthly reports for various business units like mining, oil and gas, human resources, finance, health and safety, etc.
Discovering Limitations
However, BHP’s global application landscape provided only limited and restricted reusability of existing data sources. This often led to repeated engineering efforts to access the same data sources for different solutions, as well as long lead times to ingest or load data before a data solution could be developed. In addition, the company created project-centric data repositories to provide a consolidated set of data for specific purposes, but this increased the total cost of ownership and complexity, while increasing the general variability of data interpretation. This approach also significantly slowed down the delivery of business value. BHP was also running advanced analytics projects in AWS, but many use cases required that the cloud data be merged with the data in on-premises sources. This was challenging because of the fractured state of BHP’s data architecture and the absence of a centralized data delivery layer.
A logical Data Fabric
BHP deployed a Denodo Platform cluster in every major geography where the company had a data center, including Australia, the United States, and Chile, effectively establishing a logical data fabric across these diverse regions. This multi-location deployment enabled BHP to create reusable global and regional data sets. “By using local Denodo Platform instances in each of our data centers,” said Fletcher, “any queries on a combination of multiple data sources there will be optimized, and the data set will be aggregated before it is returned to the user. This reduces the overall footprint of the data that we are moving. An additional benefit of data virtualization is that as we connect to more sources, we can now publish that data as a standalone dataset that other users can access.” Now, with the Denodo Platform, a BHP user sitting in Santiago, for example, can easily access the local files created in the Perth data center.
In addition, the Denodo Platform implementation was performed at the highest security level, and all connections to the Denodo Platform were tightly encrypted. “Security was a big focus for us in this project,” explains Fletcher. “Because we wanted the Denodo Platform to handle highly confidential data, we made sure that any data passing through it was encrypted.” BHP’s rapid virtualization of multiple data sources resulted in reduced project cost and fewer time delays in the project’s original solution design. It also led to faster data transformation and dashboard development.
