1. The case
- The PJM RTO, organizes prices by Locational Marginal Prices (LMP) incorporating the cost of producing and delivering energy from individual nodes, or locations on the grid, where the transmission lines and generation interconnect. PJM also publishes the System Marginal Price (SMP) which is LMPs minus system loss and system congestion at a given node.
- PJM allows trading at trading hubs: Chicago Hub, Dominion Hub, Illinois Hub, New Jersey Hub, Eastern Hub, Western Hub, Ohio Hub, AEP GEN HUB and AEP Dayton Hub, in an intraday 5-minute and Hourly market, as well as an Hourly DA market.
- This case study reviews a day-ahead (DA) price forecast model for PJM System price, at an hourly resolution. The price forecast model is implemented using QR AI Forecaster .
- The DA market opens for bids and schedules seven days before and closes the day prior to the trade date at around noon. We therefore execute and publish our DA forecast at 8:00 a.m. market time for the next day, to allow ample time for traders to plan their DA trading strategies. QR AI Forecaster serves as the key input of QR Trading Optimizer to optimize the bidding of your assets.
The data for this case study comes from QuantRisk free and publicly available forecast webportal. You can access the performance of our price forecast models for multiple markets here.
2. The Data
The hourly time series of the PJM system price is the main data we use to train the AI model. Feature engineering is key to increasing the accuracy and stability of price forecasts. For these we use various demand and supply side data published by PJM.
Input Data. The AI forecast model in this case study uses 6 months of hourly data for training, from February 1st to August 1st:
- Actual data: PJM system price is published by the PJM RTO at 1:30 PM for the next day. This is the price we are going to forecast.
- Demand side data: This is our own DA System demand forecast for PJM system demand.
- Supply side data: Different DA solar and wind generation forecasts, and outages (planned, maintenance, forced, total) data all published by PJM RTO.
Output data. The output of this AI forecast model is hourly DA price forecast for PJM system price executed and published at 8:00 a.m. New York time for the next day.
3. Data Exploration
The first step in designing the AI model is to take a deeper look into the data and identify quantifiable behavioral patterns that may not be visible at first glance.
In order to do that, the QR data science team uses different data analysis and visualization tools to extract information from raw data. You can see below the 6 months hourly PJM system price DA used for training.
In order to do that, the QR data science team uses different data analysis and visualization tools to extract information from raw data. You can see below the 6 months hourly PJM system price DA used for training.
The above price data shows seasonality change in the trend. These were due to weather factors leading to increased demand and therefore prices. We need more granular analysis to define the right features for this model.
The plot below displays the hourly PJM system price averages across the 6 months training data ranging from February 1st to August 1st. This data can help us see the daily pattern and daily Peak and off Peak.
Conclusions:
- The PJM system price peaks at 7:00 a.m. and 5:00 p.m., and bottoms at 3:00 a.m.The forecast model is required to capture these.
- This plot suggests to split the day in 3 periods and train the model on each data set, midnight to 7 a.m., 7 a.m. to 5 p.m., and 5 p.m. to midnight. This can be automatically accomplished by checking the Model Splitter feature of QR AI Forecaster.
- The price is clearly hour dependent. We therefore must use the built-in feature Hour-of-the-day .
We can further refine the above analysis by drilling down to days of the week. The next plot displays the average hourly price for each day of the week, computed for the 6-month training data.
The above plot suggests:
- a) Saturday and Sunday average PJM system price are lower than the rest of the days of the week and even the first peak pattern is different for Saturday and Sunday.
- b) Average PJM system price for Fridays and Thursdays is higher than the remaining weekdays.
- c) Monday, Tuesday and Wednesday have the same price levels and pattern.
- Weekday as a built-in feature or predictor to the AI Model to indicate the days of the week.
- The Model Splitter feature of the QR Forecaster should be activated to split data at run time and create and train 4 different Models to forecast different days of the week: Monday-Wednesday, Thursday-Friday, Saturday, Sunday.
4. Feature & Correlation Analysis
To select the best external time series as features or predictors, we need to analyze their correlation with the main time series PJM system price we are forecasting. A good or useful feature must be closely correlated to and shed light on a particular behavior of the main time series.
The PJM DA price data, including PJM system price, present several challenges in the 6 months period considered in this Case Study. Our data science team resolves these by configuring the right AI model so as to produce accurate price forecasts for our clients.
The PJM DA price data, including PJM system price, present several challenges in the 6 months period considered in this Case Study. Our data science team resolves these by configuring the right AI model so as to produce accurate price forecasts for our clients.
Factoring in demand effects
In the plot below you can see that PJM DA system price have occasional peaks, the highest one at the end of July followed by the second highest at the beginning of February. Weather impact could not explain these peaks. As seen in the plot below, it turns out demand data peak have a clear positive correlation with PJM system price causing these price spikes in the DA market. This was uncovered by our data science team, using our data analysis toolbox, they executed correlation analysis across many time series data published by PJM against the main PJM DA system price.
Conclusion:
- PJM demand time series should be included as an external feature or predictor in the AI model, to guide the forecast in predicting such irregularities.
- The Outlier Treatment feature of QR AI Forecaster needs to be enforced for this PJM DA system price forecast model to handle abnormal data .
5. The AI Forecast Model
By now the preliminary data analysis work has been done and we are ready to implement the best AI model for this case study, considering the specific features and data discussed previously. We use QR AI Forecaster cloud service. This platform is a no-coding automated AI platform, where modeling is done by dragging and dropping various model and feature engineering components in an intuitive AI dashboard, to automatically assemble and execute the final forecast model.
1) Data Preprocessing
We first configure a few standard data processing features discussed above in QR AI Forecaster:
- a) Gap Filling can be accomplished by several methods (Linear Interpolation, Weekly Pattern, Daily Pattern, etc. ), for price data we use Linear Interpolation.
- b) Outlier Detection can be accomplished by several methods. For the current price forecast case study, we use a standard deviation method, removing data at 3 SD, and replacing it with local average.
- c) Calendar is used by a processor to swap mid–week holiday data with Sundays in the future, and with a regular day in the past during forecast and training respectively, as well as labeling working days and weekends for the AI modeling.
2) Feature Engineering
We configure the following features in QR AI Forecaster dashboard:
| Predictor Type | Features |
|---|---|
|
Lagged Predictors |
Previous Point (which here is previous hour),
Previous Day
|
|
Built-in Predictors |
Hour, Enhanced Weekday, Month |
|
External Time Series Predictors |
Demand forecast, Wind forecast, Outage and a QR designed formula with combination of predictors |
As was discussed above, the day ahead price forecast model for PJM, is supposed to provide forecasts one day ahead at 9:00 with hourly grain for the whole next day. The so-called ‘job’ for producing the forecast in the QR AI Forecaster Machine is scheduled accordingly with a complete automated procedure which executes the AI job every day at the specified 9:00 a.m. New York time.
3) AI Model
We configure the following AI model specifications in QR AI Forecaster dashboard:
- a) Model Splitter:
- Recall that this feature allows the AI machine to split the data and define and train multiple models at run-time to forecast specific profiles. In this case considering the structure of the data, discussed in the previous chapter, we want to model specific days of the week, with a total of four models forecasting for Monday-Wednesday, Thursday-Friday, Saturday, Sunday.
- We are using augmentation in this job, the data has been scaled by 10%.
- b) Model Optimization:
- We configured a range of deep learning AI models for this case study. They have their own advantages. We present here 3 machine learning models, NGBoots, LightGBM and XGBoost. These can be configured to perform with nearly equal accuracy levels.
For example with XGBoost we configure the following 12 hyper-parameters:
| booster | number of estimators | gamma |
|---|---|---|
|
max depth |
min child weight |
max delta step |
|
Subsample |
l1 regularization coefficient |
l2 regularization coefficient |
|
base score |
evaluation metric |
objective |
The classical pitfalls of over or under fitting must be avoided. If we choose 5 values for each hyper-parameter, there are 5 12 combinations to try. This is an impossible manual task. QR AI Forecaster has an auto-ML toolbox that fine-tunes and optimizes the hyper-parameters. This toolbox runs the equivalent of thousands of scenarios under 30 minutes..
- c) Model Execution:
- Once the model is finalized, training one of the machine learning models over 6 months of hourly PJM DA system price data, and executing a DA forecast takes about 1 minute .
6. Forecast and Accuracy Analysis
1) The Forecast Display Dashboard
QR AI Forecaster has several data visualization dashboards that gather in one screen:
- a) Our forecast PJM system price data, computed and published at 8:00 a.m NY time.
- b) The actual PJM system price data, published around 1:30 p.m., after the DA market had closed around 1:30 p.m.
- c) Forecast error is computed by MAPE (mean absolute percentage error), and MAE (mean absolute error). These are listed in table format and gauges.
2) Accuracy Analysis
- The PJM DA system price forecast has a MAPE of 6.4%.
- Other PJM trading hubs DA price forecasts, using the same machine learning model, have a similar accuracy.
- As you can compare with other Case Studies we have published for DA price forecasting, in MISO and PJM, using the same machine learning model, with different parameters and features, results in a MAPE of 6 to 7%. This is mainly due to the fact that:
- The dynamic of price level change from one day to the next can be very different across markets. E.g. CAISO DA prices are a lot more correlated to fuel.
- Each market publishes different sets of supply and renewable data. Some are better correlated to DA prices.
Download Brochure
QR AI Forecaster offers a wide range of ready-for-use, advanced AI forecasting solutions, learn more about our forecasting solutions.
