Measuring Impact is Evaluating Wicked Problems
The output of a particular activity is usually easy to measure: the number of patients who come for a consultation, the number of students who attend a particular training program, the number of sustainability reports submitted, etc. But measuring impact is more difficult: It is not obvious what the effect of a doctor’s visit on a patient’s health actually is, what impact a training program really has on a student and if those sustainability reports effectively lead to more sustainability. Abstract concepts like ‘health’, ‘fitness’ and ‘sustainability’ etc. are difficult to define and therefore to measure.
Measuring impact is complex because additional to quantitative measurement it usually also involves qualitative measurement. We are not only dealing with a different method, but above all with different interests and values, different opinions and individual experiences. All of which are subjectively 'true' from the perspective one uses. This topic deals with how to manage possible difficulties and paralyses that can come with these complexities.
Wicked Problems
Complex
In complex situations, the relationship between cause and effect isn’t clear; this is the domain of the ‘unknown unknowns’. The complexity arises because different parties with different perspectives are involved. Complex problems require strength and leadership to dare to go through processes of which the outcome is not controllable. You learn by going through the process together and trying new things, where solutions emerge from the best fit.
Complicated
In a complicated context, there is still a relationship between cause and effect, but it isn’t obvious. It is the domain of experts who solve the problems on the basis of deeper analysis. In many cases, there are several good solutions. Solutions emerge from good practices.
Example: sending a rocket to the moon. Not easy, but some smart scientists can figure it out and write a manual to succeed every single time.
Simple
In simple problem situations, the relationships between cause and effect are easy to describe, and thus to manipulate. Solutions arise from best practices and routines.
Chaotic
The chaotic context is characterised by crisis where decisive action must be taken as quickly as possible. Searching for the right reaction is pointless because everything is constantly changing and there are no visible patterns that can be managed. Situations like 9/11 fall into this category. Solutions emerge from powerful leadership that creates order and stability.
Dealing with Complex Problems: Adaptive Management and ‘Bring the Whole Elephant in the Room’
Societal challenges are complex, wicked, and can only be solved through collaboration with other stakeholders. Stakeholders with different perspectives and values, usually members of different organisations who each have their own interests in mind. The challenge is to hear all the voices and to streamline them to achieve the common goal.
The methodology of impact-oriented work is a great supporting tool for this, whereby:
- All the knowledge, information and activities of the stakeholders in relation to the theme are brought together into an orderly whole.
- All perspectives are brought and kept in the same direction.
- Concrete and meaningful discussions are able to emerge.
- Insight into the perspectives of all participants becomes accessible.
Through this approach the group is enabled to think critically about how they can generate social impact together. This brings clarity and an overview that is often lacking to allow real collaboration on complex issues to take place.
Achieving impact together is not a linear but circular and iterative process. It is a process of learning together, the results of which can only be measured in the long term. That’s why it’s called adaptive management. It is important that all participants are committed to working together and stick to the plan in the future to come.