Effective model governance takes careful planning.
At first, a business may be apprehensive about the resources and time required to create a central model repository. However, best practice and regulatory guidance show that developing a model inventory, including the risk assessment and personnel responsible for each model and supporting documentation, can result in better model quality and risk management.
With this in mind, a business must carefully consider these questions:
A well-defined Model Inventory will act as a framework for categorising and tracking models, model use, and model changes or updates.
This model repository may include information such as:
The Model Inventory process has 3 key stages:
Figure 01 - Key Model Inventory Stages
First and foremost, define and identify all models. There are regulatory guidelines to help identify models, but a business must settle on its own accepted definition.
This will help create uniform standards across the business and provide clear guidelines for all stakeholders. Some processes will categorise clearly as models, others more as complex calculation exercises. The key variable is some form of repeatable calculation process that results in an output in the form of a forecast or estimate that drives a business decision.
Once a process meets these 3 criteria, then it can be defined as a model.
Model Risk Assessment
Each model should be risk rated based on the complexity and financial impact (materiality) of the calculation process. This risk rating will determine the rigour and prioritisation of all governance and risk management activities, including performance monitoring and periodic validation.
Elements that should be considered when determining the risk classification may include the models’:
It is important to have a consistent model risk assessment process that is revisited periodically, to ensure consistency, and monitor for changes to risk.
Governance – a business should develop and maintain detailed governance guidelines and technical governance work instructions to help establish, implement and maintain a robust model governance framework.
This documentation should be stored or linked to the model inventory, to make it easier to identify the model owners who are responsible for all governance related activities. The model parameters, controls and criteria for the use of the output of each model should also be identified, governed and evidenced.
Models are now critical, if not central, to business. Model Governance therefore is key to maintaining control in an ever evolving regulatory landscape.
However, to get maximum value from this type of investment companies may need to rethink and fine-tune their strategies including changing culture and processes to deliver the control and governance needed by both Regulation and good business management.
As models have to adapt, so will model validation and governance. Therefore, make sure that you review your models and catalogue your model inventory. Then proceed to prioritising your models by importance and materiality.
The good news is that you don’t necessarily have to build all this from scratch. There are tools and technologies that you could leverage such as Azure ML or AWS SageMaker, which can be used to automate the end to end model lifecycle, including monitoring and capture of governance data i.e. model audit trail.
If you wish to discuss any of the points raised or are interested in finding out how we can help you review or implement your own MRM and Model Governance framework, please contact us here to book your free Discovery consultation.
Founder and Managing Director. Blue Label Consulting
Brendan launched Blue Label Consulting in 2011. With innovative use of Data through AI, ML and other quantitative methods, he delivers robust analytics and actionable insights to solve business problems.