1st August 2023 release
Summary
The current model was trained using data up until 24th July 2023.
We also conducted many experiments by changing the feature set (input variables) as well as boosting (weighting) more recent training periods. Results for 6th June 2023 release
Experimentation results
Retraining experimentation list, results and model selection. Includes README worksheet, feature variables, training period, validation period and benchmarking results.
| View file | ||
|---|---|---|
|
Retraining schedule
Retraining a model is scheduled every 2 months or after 3 or 4 weeks after a significant change in market dynamics. The current retraining schedule for both PD and P7DAY follows:
Retrained model release dates | Retraining process and model changes |
|---|---|
6th June 2023 | No material improvements to the model could be made, hence the model has not changed. |
1st August 2023 | |
October 2023 |
Retraining an existing model
Once a model is released to pd4castr it will be periodically retrained to include the most recent historical data. Hyper parameters and weighting variables used by the MLA in the training process may also be adjusted. Note that the model_id will not change.
Training a new model
If a model is released that was trained using new features (input training variables) or that uses a different MLA then a new model will be released with a new model_id.
Results for 1st August 2023 release
Summary
The current model was trained using data up until 24th July 2023. Retraining the model included a months data after the retirement of Liddell. We also conducted many experiments by changing the feature set (input variables) as well as boosting (weighting) more recent training periods. Results for 6th June 2023 release
Results for 6th June 2023 releasePrevious release history
6th June 2023
Summary
The current model was trained using data up until 30th March 2023. Retraining the model included a months data after the retirement of Liddell. We also conducted many experiments by changing the feature set (input variables) as well as boosting (weighting) more recent training periods.
Much to our surprise the performance of all retrained and new models demonstrated that the original model either performed better or more or less equal to all new models when measured against the validation set (which was a week of recent data not used in the training set). As a result we felt it best to keep the current model. Therefore we did not change the model as scheduled for the 6th of June.
Why retraining did not result in a better model (for 6th June)
There are a number of reasons why the original model performed better than all other retrained models however we believe that the this was due to the volatile market dynamics post Liddell retirement and that the market behaviour has since settled. Models that included the weeks after the retirement of Liddell are less representative of current market dynamics and now overstate forecast prices relative to actual market outcomes.