Defining evaluation in my words 

In late 2023, I graduated with a Master of Public Health (MPH) from the University of Melbourne. During my degree, I specialised in both infectious disease epidemiology, as a way to link back to my previous vocation in immunology, and health program evaluation as I was already doing evaluation work in my two jobs. By completing coursework and a capstone project in the area, I gained a theoretical perspective of how evaluation works which has helped inform my work.

Now, I am working as an evaluator over two existing jobs to build my evaluation skills, experience and credentials. Eventually, I would like to work as an evaluator in either the public sector, non-government organisation or philanthropy. In that role, I would provide information and data to organisations so that they can improve programs that have a positive impact to society.

When I tell people that I work as an evaluator, many people ask what kind of work I do. I give a brief explanation of what an evaluator does and how the work I do makes a difference. This blog post is an opportunity for me to provide a detailed explanation of what evaluation is and why I have decided to pursue a career as an evaluator.

The typical definition of evaluation

The most common definition of evaluation comes from Michael Scriven who was one of the foremost evaluation theorists. In his words: 

“Evaluation is the systematic process to determine merit, worth, value, or significance [of a program]”. 

In other words, evaluation involves assessing how much a program has contributed to society and whether the outcomes produced are worthwhile. This is typically done by collecting data and evidence behind a program which can range from distributing surveys and collecting data to interviewing program staff and end-users. 

I do not agree with that definition as it is more applicable to external evaluators who evaluate programs independently of the organisation running them. Many people are resistant to the idea of someone externally evaluating their work as the process can be invasive and disruptive. This is complicated by the fact that they may not be involved or have any control over the evaluation process. 

Furthermore, they may not accept the results or recommendations of an evaluation. They may continue to run a program as is even when there is evidence that it is producing little value to society. I feel that this definition is a harsh one, and not necessarily reflective of the work that I do.

My definition of evaluation

In contrast, most of the evaluation work I have done so far has been internal. I work closely with organisations and program managers to conduct evaluations of their programs. This involves:

1. Planning the program and evaluation;
2. Collecting data (which can range from surveys to interviews and more); 
3. Analysing the data to generate results; and
4. Reporting on the results internally within the organisation as well as externally to funders and governments. 

When I report on the results, I highlight the evidence base we have built up to show that our programs are working to achieve outcomes. At the same time, I identify what we are doing well and what we need to improve to make our programs better. Both aspects are important to organisations as they face increasing calls to be held accountable by funders and governments. They not only need to show that their programs are producing outcomes for end-users, but they also need information to continue improving their programs. Without that information, organisations run the risk of not obtaining or maintaining enough funding to run their programs.

With that background, I would define evaluation as: 

“The process of collecting data to provide evidence behind the effectiveness [or ineffectiveness] of a program and to identify strengths and areas of improvement behind a program”.

To me, that definition softens the blow of evaluation as it does not place an all-or-nothing judgement on the program. Instead, it provides a health check to see how well a program is doing and what needs to be done to make it better. This provides some breathing space for organisations and program managers to abandon, pivot or refine a program based on the results of an evaluation. 

Viewing evaluation from this perspective improves organisations’ and program managers’ perceptions towards evaluation as they see a purpose behind this activity. They can use the results to highlight the evidence base behind a program, increasing the chances of receiving funding, and to identify areas of the program to improve. Both of these tasks are important for ensuring the sustainability of the program and the organisation.

Why am I pursuing a career as an evaluator?

I am pursuing a career as an evaluator as my existing background aligns well with my chosen career. I am someone who thinks through things rationally, who synthesises information and who mostly acts on decisions based on facts and data. I also have a background in science which has allowed me to take accurate measurements, generate and describe results and link them back to what we already know. 

These two elements of myself are what drove me to pursue a career as an evaluator. I feel that being an evaluator is a natural progression from a science career. That is because I am able to use my rationality and my existing skillset in science to generate and report on results that would be helpful to organisations running programs. The cross-over between science and evaluation also makes it easier for me to transition to being an evaluator, instead of moving to a career where my previous experience could be less useful. 

Evaluation is also a way to make a difference to society. I am driven to work on things that would benefit all of humanity by improving their lives and living standards. Even though I am not someone who generates novel ideas, I am someone who is willing to take on a supporting role to help people refine their ideas. Being an internal evaluator within an organisation allows me to take on a role that is and will be increasingly important to an organisation’s future. The demand for evaluation will only increase, increasing my chances that I can secure long-term work that would benefit society. 

Lastly, my previous work is what drove me to pursue an evaluation career. While I was doing my PhD in translational research, I was doing evaluation as a volunteer. In that volunteer role, my work had a more immediate impact to the organisation where I was volunteering, compared to my PhD work where the impacts of my research might either not occur or not be felt for years. Hence, I moved into a career in evaluation so that I can make a more immediate impact to society, where my work has been included in a number of reports and policy submissions.

Conclusion

In this post, I have provided my views of what evaluation is and the work that I do. I also outlined the reasons why I have decided to pursue a career as an evaluator. I hope that by reading this post, you will understand the work that I do and how valuable my work is to organisations in improving their programs and to governments and funders in supporting programs that have been shown to work.

Personal disclaimer

Any opinions I post in this blog post are solely my own and do not relate to those of my employer or any organisations or clients I work for or with.