Metrics

While the real transformation towards reference architecture is made in an incremental model by “Solution Vision”, there is a need for metrics to make sure that the organization stays on the right track. It is extremely important, especially in the continuous transformation company development mode.

Monitoring of architectural indicators will enable us to:

  • Have a knowledge if organization really changing in architectural layer 
  • Understand on how business decisions affect architecture
  • Make conscious decision on trade-offs between speed, business value and architecture changes

Proposed metrics are a good base to monitor that progress and a great source for organization and architect retrospections. Usually these metrics are collected quarterly. Time might be adjusted depending on the needs of the organization. 

Each of the proposed metrics should have a set threshold for better progress monitoring, any breach of set threshold should lead to an open discussion, retrospections, and adjustments.

Reference Implementation Metric (RIM)– each Solution Vision should be aligned  and compliant with standards, an outcome of CPD process and reference architecture moving towards its  fulfillment, but in some cases it cannot be achieved due to time-critical projects, legal regulation etc. That is why each Solution Vision should contain attributes which specify its overall compliance with Reference Architecture.  This attribute usually can take one of three options: 

  • “Compliant” – means “implementation design” which  is fully compatible with Reference Architecture and implementation will incrementally transform application landscape towards its desired state designed in a Reference Architecture, especially in terms of used Components, its capabilities and interactions between them.
  •  “Neutral” – implementation does not influence the application landscape, especially in terms of standards, used Components, interaction between them and their capabilities. It is usually a small change or UX changes.  
  •  “Not compliant” – it’s when “implementation design” can’t fulfill the requirement of being compliant with Reference Architecture. The situation happens especially when some of the legacy Components are developed or updated, not welcome technology is used, new Components or extra interactions are added, or any others breaches against Reference Architecture.

This is what the formula for calculating the indicator may look like:

CV= Number of complaint Solution Vision 

AV= number of all Solution Vision 

RIM is one of the key metrics used to track change directions and is tracked as a percentage of each attribute value on the set period e.g. once per quarter. You can read from the graph below a trend of developed changes. Based on it, the organization might adjust the pace of transformation investing more time and funds on “Compliant” Solution Vision.

Points scored

An example of reporting the status of the RIM metric

Application Reference Metric (ARM) – it is an indicator of the Applications used in the current Application environment of the organization for Applications in CPD with the status “Investment” and “Tolerance”.

Thanks to the use of this metric, we can check the quality of the component in our portfolio. We can check whether applications are created that are worth the investment and those that are weak in terms of business and technology are eliminated.

AP =Number of All Application in CPD

IA = Number of Invest and Tolerate Applications 

 For example, if all Applications are 100 in the environment and those with the status “Investment” and “Tolerance” 50 then the ARM indicator is 2. In the long run, this indicator should strive to the value of 1.

An example of reporting the status of the ARM metric

Implemented Capabilities Redundancy Metric (ICRM) – this is a measure of the redundancy of implemented capabilities. The measure is the number of occurrences of capabilities in existing Applications, relative to the target number of capabilities. 

This metric is used to monitor that functionalities are not built redundantly above the expected level.  The target value should not always be one. Sometimes it makes sense to have a few capability instances.

EC= Number of occurrences of capabilities in existing Applications,

TC= Relative to the target number of capabilities

For example, if 6 simplified “send SMS to Customer” capabilities are implemented in our simplified environment and it should be only 2 times, the ICRM coefficient for such a theoretical environment is 3. The value of the coefficient should go to 1.

An example of reporting the status of the ICRM metric

Capability Reference Metric (CRM) – it is an indicator of the capabilities used in the organization to those proposed in the organization’s reference architecture (AR).

This metric is used to monitor whether the organization is aiming for the expected capabilities landscape.  We can regularly check whether the projects / initiatives provide new capabilities  

UC= Number capabilities used in the organization’s current Application environment

TC= Number capabilities proposed in the organization’s reference architecture (AR)

For example, if 46 capabilities are implemented in the organization and there are 100 in AR, then the CRM factor should be 0.46. The metric should be measured at a specific time interval, e.g. once every quarter. In the long run, the number of possessed capabilities seek to level with the reference architecture and the indicator to level 1.

An example of reporting the status of the CRM metric


Next->