Key Performance Indicators and Key exception Indicators

The measurement of the overall process outcomes alone will not tell you anything about how your processes are actually doing. Yes, it sounds counter-intuitive. The fact is that although Key Performance Indicators (KPIs) are useful measurements for resulting business output, if you really want to know if a business process is running optimally, KPIs are simply not enough. Understanding exceptions to expectation and diagnosing their root causes is a critical piece of the continuous improvement puzzle. This White Paper addresses the good, the bad and the ugly of KPIs and why they should be accompanied by Key Exception Indicators (KEIs) to deliver true business insight. Furthermore the concept of KEIs is addressed, examples are given to further explain and the best way to implement described.

---

Executive summary

Measuring the status of organisational efficiency is a logical and proven management instrument. This often results in focussing on implementing Key Performance Indicators (KPIs) and institutionalised internal systems. These KPIs and controls are useful and form the foundation of process management. The limit of KPIs and controls is that they are lagging, after-the-fact and indicative in nature; they summarise the specific facets of the outcome of transactions as an average number.

KPIs have no eye on the actual individual business transactions. Deviations from the average and relative value are not watched nor considered. This hampers both the opportunities to remediate the occurred exception and to structurally improve the process. Thus, KPIs never show the performance improvement potential or the risk limitation potential of a process. We need another level of monitoring based upon the leading indicators underpinning and strengthening the KPIs. Where internal controls focus largely on achieving a standard data input process, we need to achieve a standard business process. Monitoring your business processes, 100% and end-to-end, will surface any performance or risk exception around the set parameters of business goals.

The central mechanism to do so is applying Key Exception Indicators or KEIs. The KEI should be a leading indicator to enhance the value of the lagging KPI.

The simple definition of a KEI is: a transactional measure for what can go wrong. In practice, more complex KEIs are required to find the less obvious exceptions. The measure of the KEI should match the value it has for the business, keeping in mind individual and aggregated value, individual remediation and structural restriction as well as risk and ethics impact.

It is strongly recommended to take an agile approach to implementing KEIs. The organisation has to learn and adopt; too often the usual suspected areas are not the ones with the biggest impact and value. KEI implementation should be seen as a Continuous Improvement cycle, often guided by larger improvement programs in the context of aspiring towards World Class Finance.

This White Paper addresses the business background and need for implementing KEIs, augmented by practical examples and suggested implementation scenarios.

---

Introduction

Key Performance Indicators (KPIs) are useful measurements of average outcomes, enabling comparison between organisation units and time periods. Stemming from the Balanced Scorecard ideas of Norton and Kaplan in the early nineties, KPIs have since seen a wide adoption as the default measurement principle within both commercial and non-profit organisations.

The strength of a KPI is, however, at the same time, its weakness: it is a calculated average outcome of a business activity. In principle, KPIs are crisp and sharp and can be re-calculated at any point in time, for any organisational topic or dimension and can be understood by all levels in the organisation. They represent the key elements for today’s executive dashboards and management information packs. But being an average, a KPI does not highlight the critical detail underlying the indicator. The KPI does not havean eye for what really happens during process execution, the heart of where improvement activities must be focussed.

What if a KPI metric remains the same for 2 consecutive periods? Does that mean that the underlying process execution has been the same? Well, the truth can lie anywhere between deterioration to improvement, as the KPI only shows a mathematical average of a specific attribute of the process (e.g. cycle time).

Continuous improvement is a well-accepted driver to make sure an organisation remains competitive in today’s ever faster changing world. It relies on finding key issues or ‘cases for action’ that can be addressed to structurally improve business performance and limit the organisation’s risks. Superficially, KPIs appear to be great tools for any continuous improvement programme: we can measure where we were yesterday, today and even predict tomorrow. But a KPI only measures one facet of the outcome, not what to improve.

To know if a business process is running optimally, every pass-through, iteration or transaction should be matched against the “desired” execution. “Desired” in this respect is any given combination of expected duration, procedure and outcome, valid dependencies and required controls.

Think of the following simple example: a KPI is set to measure the percentage of ‘touch-less’ invoices. A KPI value of above 90% is said to be acceptable. If this benchmark is met, the underlying process is implicitly seen as OK. But a myriad of questions remain unanswered, like:

• Is 90% a good result?
• What if the value of the other 10% of invoices represents 50% of the total value through the process?
• Are there significant differences per product line, department and country?
• Which invoices make up the remaining 10%?
• Are there any downstream un-intended consequences occurring in the 90%?

KPIs are simply not enough for answering these questions and analysing performance. Only when we have a mechanism to find exceptions during transaction execution that show deviations or exceptions can we judge the true opportunity for performance improvement. Exception analytics can reveal the root causes that impact process execution and thus point to the improvement potential for business results.

---

Sense and non-sense of KPIs

KPI Sense

Literature shows that organisations managed through a good set of KPIs perform better than those without them. But the words ‘managed’ and ‘good’ are of essence here. KPIs are typically used to measure individual and group performance. KPIs are lagging indicators. Importantly, KPIs are often ‘gamed’ or manipulated (especially when compensation is related to KPI achievement).

A typical example of a lagging KPI is Customer Satisfaction. Often, in attempts to measure customer satisfaction, a growth from an average 7.6 to a 7.7 “grade” is interpreted as achieving the KPI target. Just as well, if the unhappy customers stopped responding to the survey; or stopped being sent the survey; or left, the KPI measure would improve further…

KPI Non-Sense: Law of Unintended Consequences

KPIs show isolated measures of performance of processes or organisations. They can easily miss their intended target and even backfire. This, of course, is the ‘Law of Unintended Consequences’. Although I am essentially an optimistic person, reality has turned me into a life-long believer in ‘Murphy’s Law’. KPIs alone can fall into this trap.

The following is an example to illustrate how unintended consequences can result from a superficially logical KPI. The KPI in question measured the Days Sales Outstanding (DSO). Present in almost every commercial organisation, this measure should make revenue turn into cash as soon as possible.

A company in this case boasted a DSO of just 10 days, easily beating the peer-group average of 32 days. Those familiar with managing DSO will agree: a very remarkable result. Only after looking not only at the KPI but also at the underlying process, was it revealed how this KPI was met: the DSO was ‘managed’ by using ‘pro-forma’ invoices for initial A/R discussion with the customer. The DSO metric was measured from the moment the formal invoice was raised in the system, which was only initiated when the customer confirmed payment in their next payment-cycle, at which point the DSO ‘timer’ triggered.

Unsurprisingly, the analysis of the whole business process from closed sale to cash received showed a different picture: over 60 days on average.

The process was broken!

KPI Non-Sense: Institutionalised KPIs

In a counter reaction to overcome the unintended consequences of KPIs, a logical response is to institutionalise the achievement of a KPI by implementing system controls. So, when the measure becomes an absolute prerequisite, build it into the system so that the KPI is always met. Unfortunately controls suffer from the same disease as KPIs: the focus is placed on the resulting data and not on the executed process.

The following is a classic example to illustrate the limits of controls and the causes of unintended consequences.

Any state-of-the-art ERP system assures that goods received will only be accepted at the warehouse if an authorised employee matches them with an approved purchase order. Thus, the system is traceable, accountable and secure. Human reality proves, however, slightly more complex. Goods can show up at the warehouse dock. If the warehouse operator cannot find a purchase order, he or she checks with the Purchasing Department. This can lead to an instant purchase order to “correct” the fact that the goods have been physically received. Again the system is traceable, accountable and secure. The implicit KPI of zero two-way match defects are met. But is it business wise correct?

This can be a proof of a simple incident rightly corrected, missed volume discount options or evidence of fraud. Since the embedded controls will not likely bring it to our attention, we will never know…

The Missing Link

We have seen that, if the KPI benchmark is met, the underlying process is implicitly regarded as OK. Controls may enforce the KPI and management action may reinforce the KPI focus, but key questions, essential to the organisation’s process improvement, remain:

• What actually happened in the process?
• How do we identify areas for improvement?
• How did individual transactions deviate from
  the set standard?
• Is the deviation significant for the business?
• What is the root cause for the deviation?

Hence, we need to complement KPIs with an element that delivers answers to the above. Or in the terms of Norton and Kaplan, we need to complement the lagging KPI with a leading indicator that helps us drive performance improvement.

---

The need for Key exception Indicators

The Inverted Dimension

Any organisation that takes its business objectives seriously will design and implement the business processes in such a standard way that they make reaching these objectives possible. Think of policies and standards to ensure desired average margins on transactions or standard terms & conditions to optimise cash flow.

Exceptions to the set standards, rightly or wrongly applied, have a direct impact on the business goals. By limiting the number and impact of exceptions, process performance improves and associated risks decrease.

Performance and risk are intertwined cogwheels: they influence and enhance each other. Depending on the involved stakeholders, the focus can be on either one of them, but the effect will always impact both. Hence, they should be seen as two sides of the same coin. Unfortunately, in many organisations, structural monitoring is not yet embedded for these performance and risk impacting exceptions.

Exceptions: judge by Scope and Impact

Not all exceptions to standards are wrong. A special payment condition to secure a first deal with a new customer can be, as such, valid, if taken consciously and properly authorised. This, indeed, shows the need for two important parameters around exceptions: scope and impact.

Scope:

The decision to initiate an exception is usually taken by operational staff trying to ensure the process meet their demands, sometimes even their KPIs! Systems allow modification control at user level and workflow, if any, when authorisation by others is required. Delegation is required to run an operation but also creates the risk that acceptable, individually applied exceptions show substantial effects in aggregate.

Impact:

The impact of an individual exception can be very small at first sight, e.g., just giving a customer 30 days payment instead of the 14 days standard. But this means delay and, thus impacts invoice control, the reminder process, the average days outstanding of all invoices, the cash collection and ultimately the cash flow. In other words: the Law of Unintended Consequences hits again.

Exceptions: what Exceptions?

The examples given so far are of exceptions created in the assumed best interest of the business. The same type of exceptions, however, can hide errors, misuse and fraud. So, active monitoring for exceptions is indeed the three- sided coin: detection of both conscious and unconscious deviation from standards, making both performance optimisations possible while limiting risk and improving compliance. Even the best-set expectations give room for exceptions. Monitoring for exceptions to weed out the non- desired ones from the organisations objectives perspective is in itself a KPI that should be on the list for every C-level executive.

When scanning for process exceptions, the tendency is to look for the top 5% of exceptions that have either the biggest impact on the performance, involve the largest potential risk or result in non-compliance. I would certainly advocate focusing on and resolving these first. However, there is another category that deserves attention of its own: the unnecessary activities.

There are 3 possible types of reasons behind this:

Missing the System

• Not understanding the right way to use a system or apply a procedure
• Example: not searching and using the existing vendor from an available list or drop-down but start entering them again
• Indicating the need for training

Mending the System

• Bypassing flaws and omission in the system or procedure
• Example: delete an order and re-enter all over again because a field value can not be changed or lowering the status is not possible
• Indicating the need for system maintenance

Manoeuvring the System

• Using system design to bypass procedures
• Example: repeatedly changing order values back and forth so the not-yet-closed order remains under the radar screen of management inquiring and pushing for closure
• Indicating at least improper behaviour

The previous examples result in productivity waste and impact on resource capacity. So, is it important enough to act upon? Maybe. Even when the time needed to make a logically unnecessary change is only a few minutes, when this occurs 50 times a day, every day, it adds up. More importantly, the effort involved in fixing the downstream impact of these exceptions can have a disproportionate effect on overall efficiency and effectiveness. Of course the third type, manoeuvring the system, adds an even more serious element to that.

---

Key exception Indicator

Concept & Components

The basic definition of a Key Exception Indicator is: “A transactional measure for what can go wrong”. We look for it actively, on a regular basis, address it early and avoid major impact on our key KPIs as well as on the overall financial performance and the increase of risk.

Defining KEI allows us to understand detailed exception conditions for the key elements of the business processes. Given the often-enormous amounts of transactions in an organisation, fully automated monitoring for exceptions is inevitable.

Initially the number of exceptions identified might be surprisingly large. This will decrease over time as a result of fine tuning the exception definitions, adjusting the scope to get manageable and meaningful reports and, of course, structurally addressing the root causes of the exceptions.

A KEI is a set of transactions, in a context and time frame, for a specific process where the data values fall outside defined acceptable norms. The table below shows a few logical examples of KEIs and their components.

In general a KEI should be a leading indicator to enhance the value of a lagging one like a KPI. The logical components of a KEI are rather straightforward, as depicted in the table above. The reality is that required KEIs are often showing more complex structures:

• KEIs are not always elementary
  • They can include concatenations of attributes to result in more meaningful outcomes.
  • E.g. in the Expense example used above, other relevant attributes might be currency, location, number of people
    involved, customer involved, etc.
  • A norm can be set (an absolute or a KPI-value), derived (e.g. from master data) or dynamically set (top percentile of monitored transactions)
  • There might be more than one event involved in a chain of business processes required for a complete business transaction

What to measure?

If an exception is defined as an undesirable characteristic of an executed business transaction, one would expect that a KEI (the aggregate) should strive towards zero.

In some cases this might indeed be true, especially an exception indicating fraud, but also for other zero-tolerance KEIs like for duplicate invoices: any duplicate payment, large or small, is one too many.

In other cases, a KEI might have a desired value of more than zero. Normally there is a business rationale behind the choice to allow for above zero targets, mostly when the cost of remediating and/or repairing the found exception outweighs the value of the exception. A good example is the very small difference in the value of a Purchase Order and the resulting invoice due to rounding or exchange rate influence. But if this is the case for every invoice from the same regular supplier, the rounding is at least suspect and worth further investigation.

The measure of the KEI should match the value it has for the business, keeping in mind individual and aggregated value and individual remediation.

The root cause of an exception can be indicated by KEIs but certainly not automatically derived. An exception can be the result of anything from human ignorance to fraud, from “works-as- designed” to “we-never-envisaged-that”. But when designing a KEI, one should clearly keep in mind the following questions:

• What is the process we want to improve?
• What do we want to achieve?
• What do we want to prevent?
• Who should care about this?
• What is the expected norm?

Dealing with Complexity

From the above it is clear that defining KEIs means dealing with complexities. Typically there may be millions of transactions across multiple systems. Business operations spread across countries and locations may involve in-house and outsourced shared service centres. Complex data structures in ERP systems and specialised purpose built (legacy) systems need to be understood to get meaningful results.

In starting to monitor exceptions and in enabling their review and action, both the individual repair of an exception as well as the structural root-cause remediation have clear organisational impact:

• Who should worry about and lead the discovery of exceptions?
• Who owns the exception and will they take ownership?
• How will we get to the root cause analysis?
• How are you going to disseminate the results to business partners?
• How are you going to enable them to take the appropriate actions?
• Technology can be used to automate the above but what to automate?
• How to communicate in an acceptable format for the process owner to understand, be motivated and able to take action?

Needless to say, without good answers to the questions above, the full value of exception analysis will not be achieved. Next to corporate ownership as a prime prerequisite, another key point is to ensure that you have considered all aspects of complexity and have the skills needed at both the process and technical level to deliver the significant business value that will result from remediating the exceptions both operational and structural.

---

Implementing continuous monitoring for exceptions

Given the above-described complexity, it is strongly recommended to take an agile approach to implementing KEIs. The organisation has to learn and adopt, as too often the usual suspected areas are not the ones with the biggest impact and value. The KEIs themselves are likely not going to be right or even the right ones the first time, e.g., too many KEIs will cloud the focus on remediation; too many exceptions will cloud the few relevant exceptions.

The recommended 5-step approach for implementing continuous monitoring based upon KEIs is:

1. Management Workshop

• Joint definition and goals
• Agreed goals
• Agreed ownership

2. KEI Design Activity

• Analyse what is possible
• Create the best initial set of KEIs required
• Analyse available data

3. A limited exception analysis

• Run a live exception analysis for a process
• For limited scope, e.g., business unit or geographical region
• Aim at a quick insight

4. Business Planning

• Analyse results from the test exercise
• Review root cause analysis, action plans, ownership
• Define Business Case

5. Continuous monitoring for exceptions roll-out

• Implement monitoring capability or service
• Implement exception mitigation with rapid turnaround
• Extend KEI portfolio and expand scope over time

KEI implementation should be seen as a Continuous Improvement cycle, often guided by larger improvement programs in the context of aspiring towards World Class Finance. The difference between a KEI based approach and the typical process improvements, like e-Invoicing in P2P for example, is that KEI implementation is a continuous improvement process not a structural process change with all of the associated impact.

KEIs can be implemented in an agile and incremental way, in parallel with, or between, process changes giving immediate results without large investments.

---

KEI areas of use

Key Exception Indicators can be used for nearly all business functions. Typical areas are those where the financial results of business processes are managed:

• Expense Cycle, Procure to Pay (P2P), Source to Settle -Revenue Cycle, Order to Cash (O2C), Customer to Collection
• General Accounting, General Ledger (GL), Record to Report

Additionally, there is a group of “life cycle” business functions where exception monitoring can prove useful:

• Product Life Cycle management (PLM)
• Hire to Retire (H2R) -Project Management (PM)

And lastly, there are business functions that by nature are candidates for exception analysis like: -Travel & Expenses handling (T&E)

• Marketing Expenditure (ME)

---

KEI examples

There are numerous KEI possible and sensible to apply. The following examples give an overview on the diversity of the rationale, used criteria and typical findings.

1. P2P: Duplicate Invoices

Rationale:

• To ensure an Invoice is processed and paid only once, to avoid duplicate payments or inflated purchases
• Reduce/eliminate duplicate payments before they happen
• Duplicate invoices may result in inflated purchases or excess payments to vendor. Such invoices may lead to financial losses and affect cash outflow and working capital.

Criteria:

• Identify based on: Same supplier, Same material, Same invoice value, Same period, (same invoice id)

Typical findings:

• Invoices manually entered leading to input errors
• Supplier impatient for payment and resends same
  invoice
• Some suppliers submitting multiple invoices

2. Fixed Assets: Incorrect Depreciation Periods

Rationale:

• Assets, if depreciated to zero in shorter than required period, can be disposed of to third parties at preferential rate

Criteria:

• Identify Fixed Asset Records where depreciation periods are not in line with statutory guidelines for asset class, particularly where the period is lower than recommended

Example:

• Company cars have advised depreciation period of 4-5 years. Identify when this has been reduced for given assets
• Other examples … Buildings (40 years), New Machinery & Equipment (15 years), Office Technology (3 years)

Typical findings:

• Company cars depreciating in 1 year and then being disposed of… value in excess of $1m in 3 year period
• Buildings depreciating in 1 year

3. O2C: Price Changes

Rationale:

• Changes to prices may lead to fraudulent / inappropriate pricing of Sales Orders
• Price changes after creation can be a tactic to by- pass controls / approvals / workflows in place for order creation
• “Local agreements” / “Unapproved discounting”
• Prices being increased to finance intermediary

Criteria:

• Identify Sales Orders with prices changed after initial creation

Typical findings:

• 16% of orders within 1 month period had prices changed
• Plus 1000s more changed from placeholder values (e.g., 0.01) – circumventing system control & distorting financial numbers
• Many changes – post order creation / discounting / avoiding approvals

4. T&E: Fraud Issues

Rationale:

• Identify & prevent fraudulent and “creative” use of expense

Criteria:

• Identify expense records with suspicious characteristics

Example:

• Duplicate expense items within a given period (same item, same amount)
• Same expense submitted in different ways (expense item & corporate card)
• Claims for “suspicious” items – gifts / sundries / misc
• Multiple claims just under threshold where proof of purchase needed
• Excessive claim amounts for defined item categories – dinner / lunch / entertainment
• Claims for full price air tickets when discounts available

Typical findings

• Same meals & hotels claimed again a month later
• Personal expenses claimed for – taxis / trains / travel agent fees
• “Gifts” a common expense item

---

Summary

Measuring the status of organisational efficiency is a logical and proven management instrument. This often results in focussing on implementing Key Performance Indicators and institutionalised internal systems. These KPIs and controls are useful and form the foundation of process management. The limit of KPIs and controls is that they are lagging, after-the-fact and indicative in nature; they summarise the specific facets of the outcome of transactions as an average number.

KPIs have no eye on the actual individual business transactions. Deviations from the average and relative value are not watched nor considered. This hampers both the opportunities to remediate the occurred exception and to structurally improve the process. Thus, KPIs never show the performance improvement potential or the risk limitation potential of a process.

In order to find exceptions such as the ones described above, we need another level of monitoring based upon the leading indicators underpinning and strengthening the KPIs.

Where the internal controls focus largely on achieving a standard data input process, we need to achieve a standard business process. Monitoring your business processes, 100% and end-to-end, will surface any performance or risk exception around the set parameters of business goals. A lot of existing internal controls can be used to fuel this monitoring. As guideline the exception monitoring should take into account the following principles:

Asking the right questions

• Focus on issues that really impact the business in terms of direct money (revenue, costs, profit) or indirect money (risk, reputation, continuity)
• Asking the right amount of questions
• Less is more, both in attention value and precedent working. Designing and implementing monitoring will cost money in itself, so the bottom line, beyond being an insurance premium, must be positive.

Create ownership

• It makes no sense to monitor anything, if there is no one interested in the results. So, unless there is a senior manager who owns the process or target that is impacted by the business process execution, the best theoretical question will be useless to ask. In principle ownership should reside with a manager directly responsible for the business process.

Quick and transparent follow up

• Act upon the outcome of the monitoring quickly and adequately. It is senseless to have beautiful monitoring dashboards in place, if it takes a month to be discussed in the next meeting. An empowered team under the direction of the owner should promptly act to interpret, analyse and act upon the monitoring results. Monitoring of this follow up must be
  an integral part of the effort. If the problem
  does not exist, don’t keep looking for it.

Manage Continuous Monitoring by exception

• Ask the questions consistently and permanently. Either the impact is resolved, by means of business process improvement, organisational change and/or embedded control, or there are other questions with more impact. Analyse, tune, adjust and set aside at the earliest possible moment. And, if the problem does not exist, don’t keep looking for it. But remember, if you take away the monitoring permanently, you don’t know when the issue returns!

---

Conclusion and Takeaways

Exception analysis is the way forward for the continuous improvement of business performance and management of risk.

1. KPIs and internal controls are important to this but are only measuring desired outcomes and, therefore, should be complemented by measuring the unexpected.
2. Additional monitoring that focuses on exceptions impacting business performance, risk and basis for root cause analyses, is the way forward.
3. Key Exception Indicators are the foundation for this exception analysis.
4. Automated solutions make effective continuous monitoring possible in even the most complex environment.
5. Exception analysis implementation will pay for itself by recuperating cash from exceptions, eliminating inefficiency, structural prevention of leakage and less business risks.

---

About The Author

Hans van Nes is COO of Consider Solutions, a firm that provides business solutions and consulting services to help organisations on the journey to World Class Finance. Consider Solutions applies management advisory and technology capabilities focused on finance process optimisation, risk management and reducing the cost of compliance, control and assurance. Consider Solutions’ methodologies deliver rapid, cost-effective results whilst providing the flexibility required by business management.

Hans has a background of over 25 years in general management, performance improvement, process change and technology. He advises both customers and prospects on strategies for continuous monitoring and exception analytics. Hans is a true exponent of the New Working: living in The Netherlands he leads the international Services Team of Consider Solutions by phone, Skype and travelling around. His passion is discovering and importing fine wines from small independent wine makers where quality is still king and marketing an unknown feature.

Hans can be contacted at hvannes@consider.biz

To access the whitepaper in a PDF format click here Performance Improvement from Measurement to Action (KPIs & KEIs)