Process and prepare data in real-time for Machine Learning
Discover how Lucy in the Cloud* experts created a new solution that enables near real-time processing of massive datasets, including load curves, and exposes them to analytics and Machine Learning applications for a major energy sector player.
Context
Our client approached Lucy*’s experts with a clear challenge to solve: they were processing very large datasets that had no proper update markers in proprietary ERP solutions.
Challenges
They therefore needed a solution capable of meeting the following requirements:
- A unified data platform and Data Lake, including real-time analytics capabilities.
- Reactive BI and self-service dashboard capabilities
- Platform ability to support more ML/AI-oriented use cases.
- Deployment agility in a controlled environment. And our experts wasted no time getting to work on a strategy and technical solution that would solve our client’s main challenges.
Method and Solution
The unified data platform we built for this key energy sector player is rapidly expanding and enables them to tackle all kinds of data-related use cases, including the use of machine learning and artificial intelligence capabilities.
- Data Lake solution with Redshift and Tableau for DataViz;
- Redshift provides a unified enterprise data model, with low synchronization time from source systems;
- Real-time streaming based on AWS MSK and Kafka connect;
And much more!
*Lucy in the Cloud* is part of Micropole, a Talan company. Micropole joined the Talan Group in October 2024.
Technology used
Contact our experts
