Summary

pd4castr 7day price forecasts are significantly more accurate than AEMO’s forecast. Primarily this is due to the fact that participants are only obligated to submit accurate available capacity in their initial bids. Price band volumes are generally not restructured to represent genuine intent until within, or just prior to, the predispatch time horizon.

pd4castr PD price forecasts are generally more accurate than AEMO forecasts when the forecast period is many hours into the future. AEMO price forecasts may change considerably from the first predispatch to later predispatch runs as the forecast period approaches. This is primarily caused by participant's unit commitments and the repricing of volume that best matches a participants contract/portfolio position.

pd4castr forecasts are derived in a matter of seconds by a model which is formulated by a Machine Learning Algorithm (MLA) trained with historical data. This model effectively anticipates changes in participant behaviour based on known market conditions and prior to the actual changes in participant behaviour in real time.

Formulating a model

An MLA is used to formulate a model. A model is effectively a big equation that consumes real time predispatch and STPASA data to forecast price. Formulating a model takes a long time and retraining a known model (using new historical information) takes many hours. However once a model has been formulated, running it only takes a few seconds.

The components required to formulate a model follows:

• The MLA: First you must understand the plethora of available MLAs and choose one that best matches the problem you’re trying to solve. For example the MLA “random forest regression” is great for forecasting a value for a system that follows a well behaved distribution. However spot prices do not follow a well behaved distribution hence the selected MLA must be flexible enough to predict relatively frequent outlier outcomes.

• The Features: Features is the name given to the input variables used to predict the target (price forecast). Once you’ve selected your MLA you then experiment with various sets of features you will use to train the MLA.

• Training periods: Once you’ve selected the MLA and Features then you need to determine the historical time periods to train the MLA. You can also weight time periods, for example you may weight recent time periods more than distant periods.

• Tuning: There are a number of Hyper Parameters that you need to experiment with and tune in order to derive a meaningful model. These hyper parameters are part of the MLA.

• Performance: Finally you need to assess the performance of each model by testing the model and comparing it with other models and finally with AEMO predispatch.

• Release: Once you’re satisfied with a model you can release it to production. In time, we expect there to be more than one model that the user may select which will produce a price forecast in real time.

Retraining the model

Retraining improves the price forecast because it includes the most recent historical market information. I.e. market outcomes that has come to pass since the last retraining effort. However retraining could involve any of the steps listed in “formulating a model” (above).

The diagramme below shows the re-training process.

Open