In my last blog on policy frameworks I stressed the importance of principles, policies and procedures as an important ingredient to a governance framework. I mentioned that my quest for a policy framework maturity model came about when I was completing a process assessment for a client. They asked me to also provide them my opinion of the maturity level of their policy framework. I searched everywhere and could not find a definitive model specifically designed to measure policy frameworks.
Before I could measure the maturity of any policy framework, it makes sense that I identified practices and activities that support those policy frameworks. This is what I did before considering how to measure framework maturity. You can see that blog here.
This blog takes the results from the last and extends my policy framework journey to the next level – creating a method to measure policy framework maturity.
According to COBIT 2019, the latest framework for the governance and management of enterprise information and technology: “Performance management is an essential part of a governance and management system. Performance management represents a general term for all activities and methods. It expresses how well the governance and management system and all the components of an enterprise work, and how they can be improved to achieve the required level.” (COBIT 2019 Framework, Introduction and Methodology, ISACA).
Not surprisingly, I used COBIT as a major reference in this effort. COBIT’s performance management is relatively simple, it enables performance measurement of processes and governance components and it is repeatable and flexible.
Maturity model scenario
During the engagement I mentioned above, my client didn’t ask me to assess each policy individually, but the entire policy framework. My first place to go was COBIT and there was very little specific guidance on assessing a policy framework, so I built my own.
The following methodology is a suggested route for you to follow if you find yourself in the same place I was:
Step one, determine the scope and purpose
It is important to know why you need a maturity assessment. There are several reasons:
- To create a basis for improvement on the assessed process, component, focus area or domain
- To benchmark capabilities with common standards or frameworks
- To benchmark capabilities with other organizations using a common model
- To provide prioritization for resource allocation
- To gain a competitive advantage in areas where you currently excel
- To determine areas requiring more assurance efforts
- To develop action plans for closing the gaps between current and desired states
Consider what you are actually trying to assess. Are you looking at your entire framework, individual policies, or the effectiveness and relevance of those policies? In this case, we were looking at the policy framework.
Step two, Identify practices and activities
Now that you understand the why and the what you are assessing, it’s time to figure out what I call evaluation attributes. These are the specific areas that you are assessing. For example, if you are doing a process assessment, you would use process practices and activities as evaluation attributes, the same way you would assess process performance.
In my situation, there were no specific examples for this. Therefore, I thought of a policy framework as a process. I determined evaluation attributes, and those came in the form of practices and activities. Now I had a way to measure maturity with consistent measurement objects. The following are the practices that were used:
Practice 1. A policy framework is documented, approved, and enforced
Practice 2. A policy lifecycle management system is approved and recognized
Practice 3. Policies are communicated and distributed to all stakeholders
Practice 4. Policies are monitored, enforced and maintained
Practice 5. Technology is used to support the policy framework
Practice 6. Each policy should meet good practice criteria
Of course, you can add, remove or change any of these to fit your needs. Each of these practices is supported by a set of activities which support the achievement of each practice. You can find these here or stay tuned because I’ll show you below.
Step three, determine the evaluation model
Just when you think the hard part is over, think again. There is no generally accepted or formal method for assessing policy frameworks in our industry. COBIT suggests that governance components such as policy frameworks can be assessed using a maturity model (using the CMMI based 1-5 ratings) if a set of criteria can be established to evaluate. Therefore, you should use the practices and activities that I have identified here and assign levels of capability to each.
A maturity model is an assessment tool for evaluating an organization’s level of progress towards target goals. This is essentially a grid that describes typical behaviors exhibited at each of the levels, where lower levels define entry level behaviors and higher levels define best practice behaviors. Organizations go from lower to higher levels as they become more capable (mature) in the area being assessed. Additionally, these models can be used for benchmarking as well as creating conditions for meaningful discussions on the steps required to attain desired capability levels.
Generally, these models start with an assessment that determines the current state of maturity, sometimes called “as-is.” The next part is to determine the “to-be,” or desired state. The result or outcome from this analysis will facilitate the identification of potential gaps to assist in determining where priorities should be. Generally, you can only progress in a linear fashion.
I’ve seen maturity models go drastically bad. If misused, they can cause inappropriate behaviors and bad decisions. It is very important to note here that you should not strive to be at a level 5 on everything. A very important lesson learned is that the outcome of a maturity model should help you create the list of things you need to focus on to improve.
There are many factors to be considered during your evaluation, such as the size of your organization (small, medium, or large), the type of industry (financial, retail, manufacturing, engineering, federal, state and local government), the level of regulations your organization is required to comply with and the penalties for non-compliance, etc. The model, practices, and activities below are meant to be generic enough to be applicable to all of these factors and provides a baseline to perform your evaluation and analysis.
There are many reference models available in our industry, but the most common maturity model is the Capability Maturity Model Integration, or CMMI, which is the basis for the COBIT 2019 performance management information. This is a relatively easy model to modify to meet specific needs, and in my case, I could use a 0-5 scale to measure the policy framework. The table below shows the titles and descriptions of those maturity levels. These levels of maturity can be applied to almost any view of the business.
COBIT 2019 Framework, Introduction and Methodology, ISACA
Step four, conduct the evaluation
There are many ways to conduct the evaluation. You can do this internally or hire external advisors to assist in the assessment. There are pros and cons to each and I won’t cover them here. In all cases, it is important that involvement and agreement between all direct stakeholders is considered. Regardless of how you approach it, there are several techniques that can be used: surveys, questionnaires, interviews, document reviews and brainstorming sessions.
During my engagement, we used all the techniques above and below is the result. Each of the six assessed practices consisted of many activities. Our entire stakeholder team agreed on the assessment level for each activity, and the maturity level for the practice is simply an average. If you want to get really complex, you can add weights to those activities that you feel are more important than others.
The following tables are the results of this evaluation. The actual maturity numbers have been changed for the purposes of this blog.
Finally, here’s a rollup that identifies the maturity level for each practice, with an overall maturity level for the policy framework.
Step five, report results to stakeholders and create action plans
It’s one thing to conduct the assessment, but if you don’t do anything with that assessment you have wasted your time. The intent of this evaluation and analysis process is to drive action, identify potential gaps, enhance a policy framework, and provide assurance that policies are current and relevant for the organization. Remediation activities to address gaps can be incorporated in tactical and/or strategic planning based upon their criticality. They should also be approved by senior governance and management leadership teams to obtain consensus and support from the top down.
The dashboard below outlines a high-level snapshot of the current and desired maturity levels for each of the policy framework practices. If you are wondering how desired maturity levels were determined, these were determined by analyzing how each of the practices: 1) supported overall business goals, 2) satisfied compliance requirements, 3) addressed organizational risk scenarios, and 4) met the specific needs of management.
Governance Component Maturity Assessment Dashboard, Escoute Consulting
This dashboard provided a visual representation of how all the policy framework practices were assessed. It also illustrated the gap between current and desired states. This tool enabled management to determine the approach towards closing the gaps. This resulted in a comprehensive list of initiatives and their priorities which ultimately drive how they assigned resources to the improvement effort.
Tips and conclusion
This is not always a collaborative and easy process. I hope this blog, alongside my previous blog on policies, can help you create, govern and manage a policy framework that truly creates value for your enterprise.
To summarize, below are some of my suggested tips and tricks that can help you in your journey:
- Use the practices and activities identified in this blog to create your own policy framework control objectives and assessment model
- Refer to industry standards and frameworks to help such as COBIT, CMMI and applicable ISO standards
- Understand that all stakeholders have different views, and consensus must be gained before finalizing practices, activities and maturity level
- This is not a one and done process. Integrate your policies with your continuous monitoring activities on a periodic basis, at least annually.
- Integrate new, updated, and modified policies and procedures with your training awareness program.
- Make it a mandatory activity that all employees must read and be held accountable for keeping up-to-date with policies and procedures on a periodic basis, usually annually. The enforcement of which can be scoped in any number of ways; the size of your organization, job related relevance, sensitive and critical policies, etc.
- Decide where within the organizational structure this process will be incorporated: internal audit, policy and assurance, compliance, risk management, etc.
- Create a technology platform to assist you in governing and managing your policy framework
As always, your thoughts and suggestions are welcomed.
