Home / Resources / Customer Story / Energy Service Provider, VNG Handel & Vertrieb, Boosts Productivity and Cuts Storage Costs by 91% with Pentaho
VNG Handel & Vertrieb automates integration and processing of 11 million datasets per day, streamlines the user experience, and generates six-figure annual cost savings.
- Jens Junker, IT DevOps & Application Management
Process huge volumes of energy market data to deliver rapid insights on rapidly changing energy prices, risk levels, and more.
Establish flexible, efficient DataOps processes with Pentaho Data Integration across a wide range of trading systems and business applications, with dynamic reporting capabilities.
Enables rapid development of data pipelines, handles large data volumes flawlessly, and integrates flexibly with key systems to streamline the user experience.
“With Pentaho, the sky’s the limit! We’re building data pipelines much faster than before and the intuitive user interface empowers our team to work more productively.”
Given the dynamic nature of global energy markets and the increasing rate of change as the energy industry focuses on improving sustainability, wholesaler VNG Handel & Vertrieb needs to be fast, nimble and flexible to survive and thrive. The company aims to offer its customers tailored, cost-efficient energy services combined with innovative data insights into everything from prices to consumption.
However, everyday data management presents a complex challenge for VNG Handel & Vertrieb. Internal data teams start by importing market prices overnight and perform complex valuation calculations and dynamic ad-hoc analysis during the trading day, before finally updating forecasts in the evening. Each day can involve the processing of up to 11 million datasets—and the company’s existing tools and manual data integration workflows were struggling to scale accordingly. To process huge amounts of data reliably every day, VNG Handel & Vertrieb looked for a new strategy.
Jens Junker, IT DevOps & Application Management at VNG Handel & Vertrieb GmbH, says: “We were looking for an easy-to-use and flexible data management solution to deliver advanced analytics to our internal energy trading teams. With high data volumes and data sources and formats constantly changing, it was crucial that we could keep pace and provide consistent information and value-add services, such as customized reports, for our customers.”
VNG Handel & Vertrieb selected Pentaho Data Integration to enable Intelligent DataOps processes across many different high-performance systems, including its central Energy Trading and Risk Management (ETRM) solution.
“We picked Pentaho Data Integration for its flexibility, its easy-to-use interface and its excellent value for money,” adds Jens Junker. “The visual, no-code drag-and-drop interface for modeling data pipelines and information flows makes a huge difference for our data team.”
Since deploying Pentaho Data Integration, VNG Handel & Vertrieb has taken advantage of its wide range of connectivity and data transformation capabilities. “With Pentaho, we can combine legacy protocols and cutting-edge APIs to create clean, standardized data pipelines for any source and target systems,” confirms Jens Junker. “I haven’t seen another tool that offers the same flexibility and robustness. From simple transformations to more advanced Python scripts, Pentaho makes it easy to manage everything in a single place.”
VNG Handel & Vertrieb also takes advantage of advanced integration options to streamline user interactions and accelerate business processes. “Thanks to the built-in, REST-based Pentaho API, we can configure and run jobs easily,” explains Jens Junker. “We’ve seamlessly integrated our data pipelines with business systems such as Allegro ETRM, our core trading industry solution from ION Group, and our Microsoft Dynamics 365 ERP application. And after Pentaho has processed the data, our analysts can use MicroStrategy to generate reports and dashboards to gain new insights.”
To increase flexibility and reduce the maintenance workload for similar data transformation jobs, VNG Handel & Vertrieb relies on Pentaho’s ETL Metadata Injection feature. “Thanks to the dynamic configuration capability, we’ve been able to automate repetitive workflows and reduced the overall number of data pipelines,” says Jens Junker.
VNG Handel & Vertrieb has a close relationship with its trusted implementation partner it-novum and is highly satisfied with the Pentaho support they provide. Jens Junker emphasizes: “We’ve been working with Pentaho and it-novum for over ten years now. Together, we’ve always found optimized solutions for all our data challenges.”
For VNG Handel & Vertrieb, Pentaho has become a trusted central and universal data handling tool. “Pentaho is like a Swiss Army Knife for data pipelines and transformations,” remarks Jens Junker. “No matter whether we need a quick proof of concept to connect two applications, some ad-hoc data analysis for a specialized team, or a complex transformation pipeline – we can do it all quickly and easily with Pentaho.”
Thanks to the flexible Pentaho API, business users at VNG Handel & Vertrieb can get tailored metrics and reports at the push of a button. “In the past, when teams needed specific analyses, they had to specify what they wanted and send example code such as SQL queries to the IT team,” explains Jens Junker. “By integrating our ETRM trading and business application with Pentaho, we’ve automated this step, and users can now directly request the data they need from inside the app they are using. That means they can gain insights within seconds, and we have also substantially reduced workloads for the IT team.”
VNG Handel & Vertrieb has also used Pentaho to archive trading data and massively cut storage costs. “I’ve tried to build a solution natively in the ETRM app in C#, but I ran into memory issues due to the large data volumes,” says Jens Junker. “I then quickly created a data pipeline in Pentaho, which reduced the data volume and storage costs by 91 percent. As a result, we’re now saving a six-figure sum annually.”
Another major benefit for VNG Handel & Vertrieb is the large user base and active community around Pentaho. Pentaho’s open architecture has enabled VNG Handel & Vertrieb and it-novum to extend the platform and add customized transformation steps.
Looking ahead, VNG Handel & Vertrieb is planning to move its Pentaho solution into the cloud. “We’re modernizing our entire IT landscape with an agile DevOps mindset and moving everything to Microsoft Azure to increase flexibility and reduce costs,” adds Jens Junker. “Our goal is to run Pentaho in modern cloud services with a higher level of automation and scalability. To accelerate insights for our traders and other business users, we’re also considering integrating with event streaming platforms for near real-time data analytics of energy trading and pricing data.”
Jens Junker concludes: “With Pentaho, the sky’s the limit! We’re building data pipelines much faster than before and the intuitive user interface empowers our team to work more productively.”
VNG Handel & Vertrieb GmbH, based in Leipzig, Germany, supplies natural gas reliably and flexibly to trading companies, redistributors, municipal utilities, power station operators and industrial customers in Germany and abroad. Innovative products, diverse services and individual concepts for an environmentally friendly energy supply offer comprehensive support in the implementation of the energy transition. With sales offices across Germany and in neighboring countries, investments and business contacts across much of Europe, and as part of the VNG AG group of companies, VNG Handel & Vertrieb GmbH is close to the customer and well positioned internationally.