1. The case
2. The Data
This case study uses 5 months of actual data at 5 min resolution.
- Actual meter data are published by the transmission / ISO authority, with one-day delay, at a network of load nodes where the utility draws its energy. We fetch every load node meter data and aggregate them as the utility total load or demand. It should be noted that with intraday, we don’t have access to actual meter data.
- SCADA data is available in real-time. We use this as an intraday proxy for the utility’s real-time load. We fetch SCADA meter readings in real-time every 5 minutes, 24/7, via the API provided by the Utility. Similar to the meter data, SCADA is also published for the network of load nodes where the utility draws its energy, we fetch every load node and aggregate them as the utility total load.
Output data of the model: load forecast for every 5 min period of the day, revised every 15 min.
3. Data Exploration
QR Data Visualization toolbox is used to inspect the data and provide insights into its behavior:
- (a) There are dips in consumption at 12:30 pm for lunch, and 4:00 pm when people go home, therefore, the utility wants to capture these in the forecast.
- (b) The 30-min average is smoother than the 5-min data. (c) Observe the faint step-like behavior in the 5-min load average. It seems the meter reading equipment of this utility tends to accumulate load and publish it at the end of every 5-min period.
- (d) The behavior (c) is remedied in the QR AI Forecaster by changing the resolution of the load data during AI modeling, to get smoother data, while preserving the overall behavior or shape of the data.
QR Data Visualization is used to plot the daily pattern of the data across the days of the week:
- a) Weekends, particularly Sunday, have lower consumption.
- b) The load on Saturday is much higher when compared to the load on Sunday.
- c) Every day exhibits a dip at lunchtime, except on Sundays when people are at home or out and about.
- d) Peak periods are from 7 or 8 am to 7 pm. The rest are off-peak.
The following conclusions can be drawn by configuring the AI load forecast model for this data:
- Because of (a) and (b) we add Weekday as an internal predictor to our AI model.
- The Model Splitter feature of the AI modeling can be activated to create 3 different daily models for weekdays, Saturday and Sunday, and for each day to create a model for the peak and off-peak periods. This is a total of 6 virtual models trained and executed at run time.
4. Data Issues
This utility’s load and SCADA data present several challenges. QR Data Visualization is used to plot some of the behavior of the data:
Real-time SCADA data is unstable, at some periods, ranging from a few periods to days, some nodes publish erroneous or no data. This creates gaps or jumps. Mitigating these is addressed in a later section when we discuss the AI Forecast Model.
There are 3 obvious issues with these load data:
- DI1. The SCADA meter readers, plotted in orange, can jam and not publish data and create gaps at some periods.
- DI2. The SCADA meter readers, plotted in orange, can publish erroneous data in the form of jumps at some period.
- DI3. This is a 5-min market and meter, price and settlement data are in 5 min resolution. However, the SCADA and meter reading are not smoothly transitioning through the 5-min intervals and seem to accumulate readings and publish data incrementally at the end of each period. This creates little sharp indents in the load and SCADA data.
These issues will negatively impact the training of any AI model. Fortunately, they can and are remedied in the design of the AI load forecast model.
5. The Forecast Model
As explained above, the accurate intraday load forecast required by this utility is configured in QR AI Forecaster no-coding Auto-ML platform with the following features:
Intraday real-time load forecast model
2 Timescale modeling
We introduced a 30 min virtual version of the 5 min load data, to be calculated at a runtime for the training of the AI model. This is done with simple configurations.
There are 2 options to arrive at a 30 min data resolution.
- Option 1: We add the 5 minute load to get a 30 minutes load. The load is therefore 6 times bigger at the 30 min scale.
- Option 2: We just make a moving average of the 5 min data to make 30 min data points. In this case the scale of the load data remains the same as the 5 min load.
With either option (1) or (2) the virtual 30-min load data no longer represents the small indents of the 5 min data. This is good news.
- However, option (1), even though the natural option, still suffers from missing SCADA points. Such gaps lower the level of the 30 min aggregation. We will need to add an appropriate gap filling method.
But option (2) fills in automatically the missing SCADA points by moving average, and remedies both Data Issues (DI1) and (DI3).
QR AI Forecaster is configured with the following features:
- We configure a 30 min data resolution forecast using method (Option 2).
- We check the box that automatically produces both 30 min and 5 min forecasts. The 5 min load forecast is deduced from the 30 min forecasts by linear interpolation. This is the final design of the load forecast model for the client.
This method offers 2 great advantages:
- First: As explained below the accuracy of the 5 min forecasts obtained by using the actual 30 min forecast are very good. This is an indication that the above method works quite well in real-applications.
- Second: Forecasting at the 30 min resolution and deducing 6 forecasts at 5-min resolution by instantaneous interpolation uses a lot less computing resources than making 5 min forecasts.
The AI models
|Number of estimators
|Min child weight
|Max delta step
|l1 regularization coefficient
|l2 regularization coefficient
Data Pre-processing & Features selection are configured in their own tab in QR AI Forecaster dashboard:
- A calendar is attached to the AI model to determine working days, holidays and weekends, with the ability to switch a weekday holiday to the nearest Sunday.
- Outlier point removal, by standard deviation or min max methods, is applied to clean the SCADA data and remove the erroneous low or spiky readings when device reading fails.
- As a precautionary measure, we always activate the gap filling method via linear interpolation.
- QR AI Forecaster allows to juxtapose data for different periods to make longer time series for model training. More precisely, we use actual data until yesterday and SCADA data for today until the current period.
- The following predictors are also set:
Lagged Predictors: Previous Period, Previous Day, Previous Week
Builtins: Hour, Period, Weekday Enhanced, Workingday
Lagged Predictors: Previous Period, Previous Day, Previous Week
Builtins: Hour, Weekday
QR Forecaster Auto-ML
What about execution?
6. Forecast and accuracy analysis
The execution of the intraday load forecast, for 24 hours, at 5 min resolution, takes about 2 minutes:
Forecast error is computed by MAPE.
|MAPE: 2 months, May 1 - June 30
|5 min Forecast
|Around 1 %
|30 min Forecast
|Around 1 %
- MAPE for the 30 min forecast uses the 30 min forecast above as is, against the 30 min virtual load data obtained by averaging the 5 min load data.
- MAPE for the 5 min forecast uses the 5 min forecast data deduced from the actual 30 min forecast above, against the actual 5 min load data.
QR AI Forecaster offers a wide range of ready-for-use, advanced AI forecasting solutions, learn more about our forecasting solutions.