This blog series explores the 10 steps of the Model Development Lifecycle so that you understand the components of model development and maintenance as well as where potential sources of model risk can reside. Read the first post here for an introduction to the series and why we are likening this process to the creation of a cookie recipe as a “real-world” example.
We wrap up the Model Development Lifecycle with a post on the 10th and final step: Model Decommission.
Cookie’s Story: Shelf life
Cookie finished reading the email and then swiveled in her chair to look out the window. It was one of those beautiful late October days. The weather had cooled, the humidity more like a damp towel than a wet blanket, and the trees. They were magnificent. She saw shades of yellow, red, orange, green, and brown spreading before her into the sunrise.
Hard to believe three years had passed since she developed the recipe for what her bakers dubbed “alchemies.” Alchemy, she guessed, because the recipe produced a high-quality cookie from lower-quality ingredients.
The recipe was used multiple times. Thankfully, there wasn’t a countrywide, or worldwide, supply chain disruption. But there had been local disruptions. A few wildfires in the west and southwest had grown so large that major highways had been temporarily closed. There were floods and that odd hurricane that pinballed its way from Texas into Iowa, wrecking homes, buildings, and infrastructure in its path.
But the email she read contained the final supply agreement she needed. While the cookie recipe had performed well, times had changed. Over the past three years, there had been a seven-fold increase in small dairy and poultry farms. There had even been growth in small mills. There were now enough farms and mills that all her bakeries could locally source the most perishable food items. This meant, for the bulk of the ingredients, the supply chain had shrunk from hundreds and hundreds of miles to less than 150 miles.
Sure, it cost more. But there were other benefits as well. First, her company was better able to support local economies. Second, it improved her company’s sustainability efforts.
By sourcing her perishables locally, she no longer needed to worry about putting lower-quality ingredients into her recipe. Along with the local suppliers, her company had invested in an application that provided real-time backup routes to the farms and mills for her delivery drivers. Not necessarily new technology, but new for the company. Also, the company had invested in…
She sighed. Too much heavy thinking on a beautiful morning. She still had an email to write to let the head bakers know they could remove the “alchemies” from their production lines. But…well, that could wait a few more minutes.
Cookie sat back in her chair and took a sip of coffee. She thought about change and innovation, successes and failures. If she was honest with herself, she was sad to see the recipe get shelved. It was inspired and served its purpose well.
She sipped her coffee again. Ah well, she thought. That’s the way the cookie crumbles.
Relating it to the MDLC
The final stage of the MDLC involves decommissioning the model.
What is the process of decommissioning a model?
Decommissioning a model means the model is being retired—it is no longer being used. The steps generally involved are:
- Documenting why the model is being decommissioned.
- Notifying stakeholders that the model is being decommissioned.
- Deactivating the model from where it is running.
- Storing the model documentation for future reference.
Decommissioning happens when the model loses its effectiveness due to changes in the business or data. Sometimes these changes are gradual, which is why model monitoring is helpful.
Often, a model is not immediately decommissioned when it starts breaching thresholds. A model refit or the creation of a model overlay may be performed first. If those are not sufficient, then the model is decommissioned and a new model, if needed, goes live.
For Cookie, the business had changed (i.e., using local suppliers). There was no longer a need for the recipe, because the company had a reliable delivery process to ensure high-quality ingredients. Therefore, the recipe was removed from the cookie production line (i.e., model decommission).
Series Conclusion
Well, there you have it—a blog series describing the MDLC using a story about creating a new cookie recipe. Our hope is that by using this story we managed to explain, in an engaging way, the 10 steps required to develop a new model.
A final thought about the MDLC:
Take another look at the graphic depicting the 10 steps of this lifecycle. You’ll see there is not an arrow that leads from Model Decommission to Define Business Objectives. This is intentional. An arrow would imply that only when a model is decommissioned does the process restart.
That implication is not always true. For example, when a model is being considered for decommissioning, and there remains a business need, then chances are another model has already been built. This development could have started due to model deterioration seen during model monitoring or as part of a model challenge process a company may have. Developing new models before decommissioning ensures the business need that the model supports can still be performed without delay.
Thanks for reading our blogs about the Model Development Lifecycle (MDLC). If you have missed any posts in this series, you can find them here:
What is the Model Development Lifecycle, or, What’s Baking at FRG?
The Model Development Lifecycle (MDLC): Defining the Business and Model Objectives
The Model Development Lifecycle (MDLC): Data Assessment
The Model Development Lifecycle (MDLC): Building and Testing the Model
The Model Development Lifecycle (MDLC): Model Validation
Jonathan Leonardelli, FRM, Director of Business Analytics for FRG, leads the group responsible for business analytics, statistical modeling and machine learning development, documentation, and training. He has more than 20 years’ experience in the area of financial risk.