1. The case
This case study reviews a day-ahead load forecast model at a 5 min resolution, for a utility with a peak load of 900 MW in an urban area servicing mainly residential customers, big malls, business and shopping districts. The load forecast model is implemented using QR AI Forecaster.
This utility trades its sourcing portfolio of physical PPA and callable contracts in an hourly day-ahead market. Revisions on buy-sell orders for the next day can be made until noon of the previous day. The utility uses QR Trading Optimizer as its trading platform.
The utility’s total load forecast is a key input of trading optimization. Indeed, this is the global constraint of the optimal dispatch for each trading period forward: the sum of optimal dispatch / buy-sell for each trading period must be equal to this load forecast. Therefore, this utility requires accurate day-ahead load forecasts at 5 min resolution.
2. The Data
The forecast in this case study uses 5 months of data in hourly resolution.
Input data used by the model:
- Actual meter data are published by the transmission / ISO authority, with one-day delay, at a network of load nodes where the utility draws its energy. We fetch every load node meter data and aggregate them as the utility total load or demand. It should be noted that intraday we don’t have access to actual meter data.
- SCADA data is available in real-time. We use this as an intraday proxy for the utility’s real-time load. We fetch SCADA meter readings in real-time every 5 minutes, 24/7, via the API provided by the Utility. Similar to the meter data, SCADA is also published for the network of load nodes where the utility draws its energy, we fetch every load node and aggregate them as the utility total load.
Output data of the model:
3. Data Exploration
- Sunday load is lower than the rest of the week.
- Saturday and Monday consumptions are similar except for the low consumption at the beginning of the day in the former.
- Tuesdays until Fridays have very similar load profiles.
- Weekday as a predictor to the AI Model to indicate the days of the week
- Model Splitter feature in the AI modeling in which multiple models are trained. In the above observations, we train 4.
- The Model Splitter feature of the QR Forecaster should be activated to split data at run time and create and train 4 different Models to forecast different days of the week: Monday, Tuesday – Friday, Saturday, Sunday.
4. Data Issues
This utility load and SCADA data present several challenges.
As with most utilities, fetching data from the SCADA metering system often creates additional difficulties. We also mentioned above that this utility uses hourly data to place its buy-sell order for the next day. The next section will discuss the conversion from 5 Minute to Hourly resolution and how it coincidentally helped us with the SCADA issue.
- The SCADA meter is sometimes unable to publish data in real-time and leaves gaps in the data.
- SCADA publishes erroneous data; sharp increase and decrease.
- SCADA accumulates readings from previous intervals, and releases data at the end of a period, resulting in zigzag shape in the data.
However, we still have an issue when several 5 min data in the same hour have very low values which causes the drop in period 12 above. The same is true on the other extreme side where load level increases too high.
5. The Forecast Model
|Number of estimators
|Min child weight
|Max delta step
|l1 regularization coefficient
|l2 regularization coefficient
- Gap Filling. Gaps in data are prevalent and occur at random. QR AI Forecaster offers various Gap Filling Methods, e.g., Linear Interpolation, Weekly Pattern, Daily Pattern, etc. Typical option for load forecasting can be the Weekly Pattern. This takes the average of the previous weeks of the same weekday and period as the gap.
- Outlier Detection. QR AI Forecaster Offers Two-step outlier detection method. First we use the min-max method to catch the unreasonable highs in the data. These values would typically shoot up more than 10 times of the ordinary data. Then, we use a standard deviation method to clean up the rest of the erroneous data.
- The following predictors are also set:
|Previous Period, Previous Day, Previous Week, Previous 2 Weeks
|Hour, Period, Weekday Enhanced, Workingday
|6-Hour Moving Average
QR AI Forecaster allows offline training. A scheduler can be set for training the model, maybe once a day, and it can be used to forecast throughout the day.
6. Forecast and accuracy analysis
|Execution Time of DA Forecast
|Average Weekly MAPE
Recall that this is a day-ahead market and that bilateral contract nomination and buy-sell are submitted for the next day. However we forecast between Hr +1 and D + 2 since SCADA is available throughout the day. SCADA helps the model adjust its levels. This is evident in the MAPE table above: MAPE gets better as the forecast execution time gets closer to the next day. E.g. The 5:00 DA forecast has the highest error and the 17:00 forecast has the smallest error.
QR AI Forecaster offers a wide range of ready-for-use, advanced AI forecasting solutions, learn more about our forecasting solutions.