Backtesting Credit Risk Models: Methods, Metrics and Best Practices
Credit risk model backtesting compares model predictions with actual outcomes to determine whether a model continues to perform as expected.
For IFRS 9, backtesting is particularly important because PD, LGD, EAD and ECL estimates directly influence impairment. A model may be conceptually sound, but if predicted outcomes consistently differ from actual experience, recalibration or redevelopment may be required.
What Is Credit Model Backtesting?
Credit model backtesting asks a simple question:
Did the model predict risk reasonably compared with what actually happened?
The approach depends on the component being tested.
For PD models, predicted default probabilities are compared with observed defaults. For LGD and EAD models, predicted recoveries or exposures are compared with realised outcomes. ECL backtesting can also compare prior ECL estimates with subsequent credit losses and portfolio movements.
PD Backtesting
PD backtesting typically compares predicted PDs with actual default rates across portfolios, rating grades or risk segments.
Useful tests may include:
- Predicted vs observed default rates
- Actual-to-expected ratios
- Binomial calibration tests
- Rating-grade performance
- Default-rate trends over time
Discrimination measures such as ROC/AUC, Gini and KS can also support model performance testing by assessing whether the model correctly separates higher-risk borrowers from lower-risk borrowers.
LGD and EAD Backtesting
For LGD, predicted loss rates should be compared with realized recovery experience, including recovery amounts and timing.
For EAD models, particular attention should be given to the performance of CCF estimates and behavioral assumptions.
ECL Backtesting
ECL backtesting should go beyond checking the final impairment number.
Banks should investigate differences between expected and realized outcomes and understand whether those differences came from:
- PD movements
- Recovery assumptions
- EAD behaviour
- Stage migrations
- Macroeconomic forecasts
- Scenario weights
- Management overlays
This helps distinguish normal forecasting uncertainty from weaknesses in the modelling framework.
Backtesting and Model Validation
Backtesting is an important part of credit risk model validation, but it should not be treated as the entire validation process.
A complete validation should also review methodology, data quality, calibration, stability, implementation, assumptions and governance.
Key Takeaway
Credit risk model backtesting provides evidence of whether model predictions remain consistent with actual portfolio behavior.
Effective backtesting should be performed regularly, investigate material prediction errors and lead to recalibration or model changes where necessary.