Changing business conditions, the rapid shift to renewables, and market pricing dynamics all require energy wholesalers to pivot strategies with agility and confidence. Making sure those pivots are the right ones, especially in a highly regulated industry, means having ready access to timely and relevant data.  

This need for accurate and real-time data is only increasing alongside escalating global energy demands. According to Enerdata’s World Energy and Climate Statistics – Yearbook 2024, “In 2023, global energy consumption grew by 2.2%, which was faster than the average growth rate of 2010–2019.”  The US Energy Information Administration (EIA) projects global energy consumption to increase “by nearly 50% by 2050.” 

Wholesaler VNG Handel & Vertrieb knows the value of data. It’s the fuel that drives the company’s tailored, cost-efficient energy services and innovative data insights that customers use for everything from prices to consumption. 

Keeping Pace with Huge Data Volumes

VNG Handel & Vertrieb’s internal data teams have a heavy daily workload. It starts with performing complex valuation calculations on overnight market prices, with continual dynamic ad-hoc analysis, and finally ends in the evening with forecast updates. These efforts can involve the processing of over 11 million datasets every single day.  

The company struggled to keep up and needed to move away from its existing tools and manual data integration workflows. “We were looking for an easy-to-use and flexible data management solution to deliver advanced analytics to our internal energy trading teams, “said Jens Junker, IT DevOps & Application Management at VNG Handel & Vertrieb GmbH. “With high data volumes and data sources and formats constantly changing, it was crucial that we could keep pace and provide consistent information and value-add services, such as customized reports, for our customers.”  

 VNG Handel & Vertrieb selected Pentaho Data Integration (PDI) to enable intelligent processing of its data across many different high-performance systems, including its central Energy Trading and Risk Management (ETRM) solution.
 
“We picked Pentaho Data Integration for its flexibility, its easy-to-use interface and its excellent value for money,” said Junker. “The visual, no-code drag-and-drop interface for modeling data pipelines and information flows makes a huge difference for our data team.” 

The Power of Automation and Connectivity 

Pentaho Data Integration has enabled VNG Handel & Vertrieb to benefit from a wide range of connectivity and data transformation capabilities like ETL Metadata injection to reduce repetitive workflows with automation, standardize data pipeline deployments, streamline user interactions, and accelerate business processes. 

 “With Pentaho, we can combine legacy protocols and cutting-edge APIs to create clean, standardized data pipelines for any source and target systems,” says Junker. “I haven’t seen another tool that offers the same flexibility and robustness. From simple transformations to more advanced Python scripts, Pentaho makes it easy to manage everything in a single place.” 
 
Pentaho Data Integration’s advanced integration options also add value to VNG Handel & Vertrieb’s existing technology investments. “Thanks to the built-in, REST-based Pentaho API, we can configure and run jobs easily. We’ve seamlessly integrated our data pipelines with business systems such as Allegro ETRM, our core trading industry solution from ION Group, and our Microsoft Dynamics 365 ERP application. And after Pentaho has processed the data, our analysts can use Microsoft Power BI to generate reports and dashboards to gain new insights.”  

Increasing Data’s Use, Value and Impact 

Thanks to the flexible Pentaho API, business users at VNG Handel & Vertrieb can get tailored metrics and reports at the push of a button, where previously, they had to specify and send examples of the code they needed to the IT team for tailored analysis. “By integrating our ETRM trading and business application with Pentaho, we’ve automated this step, and users can now directly request the data they need from inside the app they are using. That means they can gain insights within seconds, and we have also substantially reduced workloads for the IT team.” 

The company has also used Pentaho to archive trading data and massively cut storage costs. By easily and quickly creating a data pipeline that could manage large data volumes, the company reduced data volume and storage costs by 91 percent for six-figure annual savings. 

For VNG Handel & Vertrieb, Pentaho has become a trusted central and universal data handling tool. “Pentaho is like a Swiss Army Knife for data pipelines and transformations,” said Junker. “No matter whether we need a quick proof of concept to connect two applications, some ad-hoc data analysis for a specialized team, or a complex transformation pipeline – we can do it all quickly and easily with Pentaho.”  

The Path Forward with Pentaho  

Soon VNG Handel & Vertrieb plans to move its Pentaho solution into the cloud. “We’re modernizing our entire IT landscape with an agile DevOps mindset and moving everything to Microsoft Azure to increase flexibility and reduce costs,” said Junker. “Our goal is to run Pentaho in containers on Kubernetes with a higher level of automation and scalability. To accelerate insights for our traders and other business users, we’re also considering integrating with event streaming platforms for near real-time data analytics of energy trading and pricing data.”  

Learn more about how Pentaho Data Integration can help modernize your approach to data here or request a demo.