Program Implementation – The Alphabet Soup of M&E: PDIA, HICD, MM, SCBM, etc.

A friend of mine recently commented on Monitoring & Evaluation (M&E) processes, which made me ponder as to why they are found baffling by the average person, no matter how many years of experience and education that person may have.

I discovered that many proposal evaluators get confused when reading the proposed M&E section and will acknowledge without compunction that they just could not quite follow what the organization writing the M&E plan was actually proposing.   I have also witnessed intelligent individuals turn glassy eyed at hearing about the M&E work plan’s development, that includes outputs vs. outcomes, inputs vs. indicators, activities vs. results, and the concept of an “iterative adaptation”.

Below I share some of the M&E resources that I found helpful in trying to understand what different donors had in mind when referring to the elusive “monitoring for results” in capacity building projects.  However, I have yet to find answers to my concerns about conflicts of interest and other problems in M&E and program implementation:

  • Who are the evaluators?
    • Evaluating the competition:  There is an inherent conflict of interest when the evaluators are hired to do M&E work on an implementing entity and they themselves are competitors in the contracting/grant implementation world.  This situation places the implementer in a very vulnerable position, as the competitor/evaluator is in the enviable position of learning proprietary information.
    • Evaluating a former employer:
      • When a disgruntled or aggrieved former employee is hired to evaluate the former employer’s work by the donor, who is aware of the complaints and grievances of this former employee, the integrity and the objectivity of the evaluation are in peril.
      • When a former employee is knowingly hired by the donor to evaluate that former employee’s own work, there is an inherent conflict of interest that taints the evaluation from its very beginning.  How unbiased can that former employee be?
  • How does one ensure true transparency in the M&E process?
    • Learning from failure:
      • Will the program implementer that the M&E shows is failing in certain aspects of the project not worry about the potential risk of losing the project to a competitor?
      • Donors face budgetary pressures to work on successful programs.  But M&E points out to what does not work, what needs improvement.  If the M&E plan is done internally, by the implementer itself, there are conflicts between those program experts who want to apply the learned lessons of the M&E -even if it means revising the program, readjusting it, or removing parts of the program that don’t work, and those administrators who mostly pay attention to the bottom line and do not want to see the program shrink at all.  One could argue the same conflicts exist between donor and contractor.  See the tension?
  •  How can you guarantee complete accuracy of the data being entered into a database?

    • Self-assessment via an implementer’s internal M&E process relies on the honesty, good faith, and accuracy of the employees providing the data and those entering the data.  However, when the donor is under immense pressure to produce results, the temptation to churn information that may not be verifiable is real.
    • The same issues above apply to third parties hired by the donor to gather the implementers’ data and produce charts and graphs that make beautiful infographics for future publications.  However, who monitors these third parties, who may be using flawed algorithms or erroneous excel sheet mathematical equations?

So, is M&E really that difficult to understand?  I have my own theory on why Rule of Law/Justice Sector projects are so hard to assess, but this is for another day.  Here is a list of methodologies and other resources for you to decide:

 

Eliminating Violence Against Women

Below is an illustration from a booklet published by IDLO a few years ago that was used by the Afghan Attorney General’s office to explain the Law on the Elimination of Violence Against Women.  It provides a glimpse of what was then the educational campaign needed to reach the many rural and remote provinces, communities and government officials who did not know about the laws affecting the rights of women.
The OECD has interesting statistics that show countries’ ranks in the world of violence against women (VAW).  In 2014 Jamaica appears as the least violent country, while Guinea appears to have been last.  The World Health Organization (WHO) produced an Ethical and and safety recommendations for intervention research on domestic violence against women, which focuses on “health-based interventions to address VAW…”.  The WHO Infographics shows why VAW has health-based repercussions.

Keeping things in perspective, from an international development lens, it was not until 1994 that the U.S. Government enacted the Violence Against Women’s Act, and the U.S. Department of Justice’s Office on Violence Against Women (OVW) did not open until a year later.

Of note:  Even though the U.S. Government has not ratified the UN’s 1979 Convention on the Elimination of All Forms of Discrimination against Women (CEDAW), many U.S. Government funded Rule of Law programs incorporate CEDAW in their gender-based and VAW projects.  By the way, Afghanistan ratified CEDAW on March 5th, 2003.

 

Resources on Monitoring & Evaluation (M&E)

Below are some resources that I have found helpful.  There are many documents, tools, papers, how-to suggestions, primers, etc., freely available to help beginners and experts alike.  I especially enjoy reading program evaluations and audits of program implementers that identify successes, inefficiencies, and failures.

In my own experience, I have found that one learns most from failures.  They may be a hard pill to swallow, but, at the end of the day, failures make one more perspicacious.

INPROl’s Qualitative and Quantitative Approaches to Rule of Law Research

From USAID: The Monitoring and Evaluation Handbook for Business Environment Reform 

USAID’s Project Monitoring, Evaluation and Learning (MEL) Plan

USAID Evaluation Toolkit

World Bank’s Tools, Methods and Approaches 

UNDP’s Handbook on Planning, Monioring an Evaluating for Results

Rule of Law Resources

Below is an incomplete list of Rule of Law resources that I have found very helpful throughout my work.  Because of my concentration on Afghanistan in recent years, some of these resources include specific Afghan projects.  However, some of the references and issues are applicable to other rule of law and justice sector contexts.  I will be supplementing this list as best I can as time goes by.

Afghan Women Network (AWN)

Afghanistan Legal Documents Exchange Center (ALDEC)

Afghanistan Legal Education Project (ALEP)

Afghanistan Research and Evaluation Unit

Center for International Governance Innovation (CIGI)

Harvard Islamic Research Studies

Hauser Global Law School Program: (GIRoA Legal System and Research)

Humanitarian Crisis in Afghanistan

Independent National Legal Training Center

International Committee Red Cross (ICRC) Afghanistan

International Legal Foundation

International Network to Promote Rule of Law (INPROL)

Law Library of Congress

LexisNexis – The Rule of Law

Max-Planck-Institute Organization and Jurisdiction of the Newly Established Afghan Courts

Relief Web International (Afghanistan)

The Afghanistan NGO Safety Office (ANSO)

The Siracusa International Institute for criminal justice and human rights

United States Institute of Peace

University of Michigan Law Library

Women for Women

Yale Law School – Lillian Goldman Law Library