Is Foreign Assistance to Justice Sector Reform Projects Worthwhile?

Late on Sunday afternoon, doing some research on whether recipient countries might pay lip service on Rule of Law/justice sector reform donor programs, I came across this article in the Mexican Law Review on Rule of Law, titled, “International Support for Justice Reform: Is It Worthwhile? written by Luis Pásara, a Peruvian lawyer, sociologist and professor.

I have just skimmed through it, although I already know I want to study it in full  because certain premises of his coincide with some of my thoughts.  While it addresses justice reform in Latin America, the author touches on universal problems.  Mr. Pásara’s conclusion is worth quoting:

On the one hand, it is important to keep in mind that establishing the Rule of Law is a broader and more difficult task than reforming the justice system. Therefore, building a better justice system is not enough to establish the Rule of Law; the former is just a component of the latter. The quality of the laws, the legal culture, the actual social and economic inequalities, and the role played by the government —among other elements— are important and complex components of the process of building the Rule of Law.

On the other hand, internationally-funded programs of justice system reform are not able to produce deep changes, which are badly needed for both a better justice system and the establishment of the Rule of Law, in the receiving countries. Clearly, such programs are not able to “fundamentally reshape the balances of power, interests, historical legacies, and political traditions of the major political forces in recipient countries. They do not neutralize dug-in antidemocratic forces. They do not alter the political habits, mind-sets, and desires of entire populations” and “[o]ften aid cannot substantially modify an unfavorable configuration of interests or counteract a powerful contrary actor.”175 That is why international aid in the area of justice has not delivered a new justice system in receiving countries. It simply could not do it.

But there is some room for improvement. Taking into account the analysis made in this article, some concrete suggestions can be proposed for the many people, acting in good faith in the international agencies and who are willing to find ways to do a better job of improving justice systems in the region:

Knowledge is a must. No decision about the area, content, size, timing or amount of a project should be made without detailed knowledge of the subject in the country where the work is to be done.

Learn what others produced. To gain knowledge of the prevailing conditions mainly requires: collecting the information that already exists, paying attention to national actors’ perceptions and analysis, taking advantage of the knowledge of international experts who have gained experience in that particular country, and evaluating other agencies’ experience in the field.

National actors and a clear strategy are needed. The conditions required to develop a project include: a core of national actors who are truly committed to the reform goals, and a strategy —to be designed jointly with national actors— with well-defined short, medium, and long-term goals within the project.

National actors have a crucial say. The implementation phase of any project needs to have a partnership of national and international actors, but the last word should be said by national actors who know better and ultimately are responsible for the reform process in their country.

Monitoring and evaluation are indispensable. Project implementation needs continuous monitoring and project evaluation presents opportunities to learn about both achievements and failures. External reviews of the projects —including work done by academic researchers— are powerful tools for a critical analysis on what works and what does not. Reticence to share information with capable peers is, in the long term, a way of wasting resources.

If these remedies —and other possible changes— are introduced to alter the performance of international actors and agencies, they may dramatically increase the level of quality of the outcomes of justice reform projects.

Program Implementation – The Alphabet Soup of M&E: PDIA, HICD, MM, SCBM, etc.

A friend of mine recently commented on Monitoring & Evaluation (M&E) processes, which made me ponder as to why they are found baffling by the average person, no matter how many years of experience and education that person may have.

I discovered that many proposal evaluators get confused when reading the proposed M&E section and will acknowledge without compunction that they just could not quite follow what the organization writing the M&E plan was actually proposing.   I have also witnessed intelligent individuals turn glassy eyed at hearing about the M&E work plan’s development, that includes outputs vs. outcomes, inputs vs. indicators, activities vs. results, and the concept of an “iterative adaptation”.

Below I share some of the M&E resources that I found helpful in trying to understand what different donors had in mind when referring to the elusive “monitoring for results” in capacity building projects.  However, I have yet to find answers to my concerns about conflicts of interest and other problems in M&E and program implementation:

  • Who are the evaluators?
    • Evaluating the competition:  There is an inherent conflict of interest when the evaluators are hired to do M&E work on an implementing entity and they themselves are competitors in the contracting/grant implementation world.  This situation places the implementer in a very vulnerable position, as the competitor/evaluator is in the enviable position of learning proprietary information.
    • Evaluating a former employer:
      • When a disgruntled or aggrieved former employee is hired to evaluate the former employer’s work by the donor, who is aware of the complaints and grievances of this former employee, the integrity and the objectivity of the evaluation are in peril.
      • When a former employee is knowingly hired by the donor to evaluate that former employee’s own work, there is an inherent conflict of interest that taints the evaluation from its very beginning.  How unbiased can that former employee be?
  • How does one ensure true transparency in the M&E process?
    • Learning from failure:
      • Will the program implementer that the M&E shows is failing in certain aspects of the project not worry about the potential risk of losing the project to a competitor?
      • Donors face budgetary pressures to work on successful programs.  But M&E points out to what does not work, what needs improvement.  If the M&E plan is done internally, by the implementer itself, there are conflicts between those program experts who want to apply the learned lessons of the M&E -even if it means revising the program, readjusting it, or removing parts of the program that don’t work, and those administrators who mostly pay attention to the bottom line and do not want to see the program shrink at all.  One could argue the same conflicts exist between donor and contractor.  See the tension?
  •  How can you guarantee complete accuracy of the data being entered into a database?

    • Self-assessment via an implementer’s internal M&E process relies on the honesty, good faith, and accuracy of the employees providing the data and those entering the data.  However, when the donor is under immense pressure to produce results, the temptation to churn information that may not be verifiable is real.
    • The same issues above apply to third parties hired by the donor to gather the implementers’ data and produce charts and graphs that make beautiful infographics for future publications.  However, who monitors these third parties, who may be using flawed algorithms or erroneous excel sheet mathematical equations?

So, is M&E really that difficult to understand?  I have my own theory on why Rule of Law/Justice Sector projects are so hard to assess, but this is for another day.  Here is a list of methodologies and other resources for you to decide:

 

The Rule of Law – an Australian Perspective

No one seems to have the definitive answer to what “Rule of Law” (ROL) actually means.  There are all sorts of interpretations derived from interpretations of that term of art.   I have found that, depending on one’s background, education, and life experiences, there are some variations of what we might define as “ROL”.  After all, totalitarian states have been ruled by law, whether they might be draconian laws or laws never to be applied to the rulers.  Here in the US I find that -more and more- many a lawyer and legal professional mention that laws are being applied to the “little” people, and that the powerful and rich get away with circumventing the law.  Alas, the way I see it, and because we are flawed human beings, as long as we can talk about this and not end up in a torture chamber or dark cell, we are alright.

From a philosophical perspective, though, I am intrigued by what other societies’ perceptions are about the ROL.  Below is the ROL pyramid that the Rule of Law Institute of Australia has developed:

Screenshot_20180125-095254

As I perused the site, I came across The Rule of Law:  Its State of Health in Australiawritten by University of Sydney Professor Kevin Lindgren, where he points out in his introduction key concepts as to the meaning of the ROL:

The expression signifies not a legal rule, but more generally rule by “law” as distinct from rule by power, free of legal constraint, whether by a democratically elected government, a tyrant or otherwise. So, the ideal signifies that the institutions of the state, and in particular, the individuals and bodies that are invested with power by the state, should be subject to the law rather than above it.

There are narrow and broad meanings of the rule of law. According to the narrow meaning, the rule of law is not concerned with whether a law is good or bad, but only with whether the law is applied equally to all. According to the broader meaning, the ideal embraces human rights standards. On any reckoning, both the rule of law and human rights standards should be respected, observed and protected. The only question is the semantic one of whether we properly treat the former concept as encompassing the latter.

It is quite fitting to recall that these “human rights” that involve the broader meaning of the ROL were drafted by representatives of nations of different legal and cultural backgrounds and became the Universal Declaration of Human Rights, which was proclaimed by the United Nations General Assembly almost 70 years ago, on December 10th, 1948, as a common standard for the world.

State’s OIG’s Work Plans.

You may be as quirky as I am, and enjoy knowing what the State Department’s Office of the Inspector General (OIG) plans on auditing/inspecting this year and the next (Work Plans for 2018 and 2019 from the State Department’s OIG).  I have found that, through the years of reading audits and reports from the various US Government OIGs I have learnt a lot, especially how the US Government manages its contracts and personnel, as well as what are some of the contractors’ weaknesses and strengths.

For example, I have often marveled at how bad all parties to a contract can be with poor record keeping, despite knowing that, invariably, a government contract will be audited down the road.  It is amazing to me that we do not make an extra effort to ensure that records are easily accessible.  When working on rule of law programs in other countries, we stress the importance of accountability and transparency, with good record keeping being essential to fight corruption.

 

An Afghan Prison, a Loom, a Heater, and Cinderella’s Slipper.

My Afghan friend sent me an article from The Economist today, with the harsh comment: NO COUNTRY imprisons a larger share of its people than America.” , which made me thing of my Afghan experience…

A few years ago I thought that one day I would be able to write about my experiences visiting an Afghan prison.  I am one of those lawyers who has never seen the inside of a prison, except for some, like Robben Island, that today are museums.

Despite the time that has gone by, it is still all too fresh in my mind and I cannot quite capture the myriad of emotions I went through while visiting this place.  It had to be one of the strangest, most surreal, perturbing, curious, worrying, enigmatic, perplexing, and frankly- bizarre yet hopeful experiences of my entire life.  Am sure there are more adjectives to describe my emotions.

I had the rare opportunity to visit the cells, talk to the prisoners -the young ones in English, the older ones with an interpreter- and observe a selected few in their rehabilitation or vocational environments that involved working with metal, leather and wool.

The prisoner cobblers were working on creating charming women and girls’ shoes.  There was something touching to see these men (all convicted hard-core criminals) cutting and gluing and nailing together all these shoes.  There was a master cobbler who was teaching the prisoners how to be shoemakers.

We did not speak the same language.  We come from different worlds.  They were making useful and pretty things; some prisoners seemingly delighting in the novelty of a visit by strangers… others oblivious to anything other than tending to their craft with serious and meticulous concentration.  One of these days I will share the entire experience.  Indeed, it has all to do with Rule of Law.  I just need to figure out how best to share this…  These not-so=good photos were taken by yours, truly.

A prison cobbler tending to his trade. A prison cobbler tending to his trade.

Made in prison: a Cinderella slipper. Made in the largest prison in the world:  a Cinderella slipper.

A prison loom. A prison loom.

Hand-made water heaters from corrugated metal. Hand-made water heaters from corrugated metal.

But, What Is The Rule of Law?

I still cannot fully articulate what it is for the world at large.  I know that I understand the Rule of Law to be what I identify as core principles endemic in my education, philosophy, culture and world view.

However, when I worked in the Soviet Union, despite my rejection of what I found lacking or horribly disappointing in the USSR’s understanding of what was the “rule of law” for them, not to mention the miserable record of their own citizens’ human and other rights, there were international standards that the Soviets maintained, so that if you were involved in a commercial transaction with the USSR -like I was-, you could count on the Soviets to adhere to certain “international” norms.  There was a “rule of law” of a certain kind.

I still believe those involved in international development work in the “rule of law” field cannot quite come to terms with what they actually mean by “rule of law”.  During the “first generation” of international development specialists who did work on “rule of law” issues, the emphasis was on foreign assistance on commercial aspects of the law, because, for example, the USSR and Warsaw Pact countries did not know how to negotiate commercial/international trade transactions.  The “second generation” of international development specialists thought they had a better understanding of the “rule of law”, but, in reality, they were looking at things through their own prisms.  “Rule of Law” is not just criminal justice issues, nor the “mot-du-jour”:  anti-corruption.  More on this later…

So, I share with you Louise Arbour’s speech on the “rule of law”.

Eliminating Violence Against Women

Below is an illustration from a booklet published by IDLO a few years ago that was used by the Afghan Attorney General’s office to explain the Law on the Elimination of Violence Against Women.  It provides a glimpse of what was then the educational campaign needed to reach the many rural and remote provinces, communities and government officials who did not know about the laws affecting the rights of women.
The OECD has interesting statistics that show countries’ ranks in the world of violence against women (VAW).  In 2014 Jamaica appears as the least violent country, while Guinea appears to have been last.  The World Health Organization (WHO) produced an Ethical and and safety recommendations for intervention research on domestic violence against women, which focuses on “health-based interventions to address VAW…”.  The WHO Infographics shows why VAW has health-based repercussions.

Keeping things in perspective, from an international development lens, it was not until 1994 that the U.S. Government enacted the Violence Against Women’s Act, and the U.S. Department of Justice’s Office on Violence Against Women (OVW) did not open until a year later.

Of note:  Even though the U.S. Government has not ratified the UN’s 1979 Convention on the Elimination of All Forms of Discrimination against Women (CEDAW), many U.S. Government funded Rule of Law programs incorporate CEDAW in their gender-based and VAW projects.  By the way, Afghanistan ratified CEDAW on March 5th, 2003.

 

Musings on Monitoring & Evaluation (M&E) of Justice Sector Reform Programs

Some U.S. Government agencies were late in understanding the importance of M&E to determine the impact that foreign assistance programs were having.  In the last few years, I always kept hearing that we needed to answer then Secretary of State Hillary Clinton’s “so what?” question regarding how effective our international aid projects were.

Many multi-million dollar programs had no internal nor external M&E experts to provide guidance.  In Afghanistan, for example, the U.S. Embassy’s 2013 rule of law strategy failed to incorporate any performance measures.  (For an interesting report that reveals what the problems relating to M&E were at the time, I suggest you read the Special Inspector General for Afghanistan Reconstruction (SIGAR) audit).

Through evaluation tools, M&E programs aim to demonstrate program impact.  This, in turn, provides feedback to guide program implementation staff to enhance future programming by identifying planned and unplanned results to allow donors, implementers and host country beneficiaries to understand what works and does not work, how to maximize efficiencies, and address any issues that might arise before they become a problem or a cataclysmic risk.

In government contracts, the Statement of Work (SOW) may provide the indicators to be used.  Sometimes, the implementer may develop a series of iterative evaluations as well, which might include a training evaluation and an audit, a trainee-satisfaction survey, a mentoring plan, and -depending on the program- a public outreach component.

Performance indicators may combine the Foreign Assistance Framework Indicators (F-Indicators), as well as customized indicators, with the goal to develop and utilize indicators that measure outputs and impact in the short, medium and long-term of the project.

Of course, the most perfect and all-encompassing M&E plan will not work unless both donors, implementers and beneficiaries take into account the critical risks inherent in, or coming from, the place of performance, and agree on some critical assumptions that, at the very least, encompass three contexts: political, security, and operational.

What I have learnt is that decision-makers and bureaucrats from both the government side and the corporate side make choices and issue “diktats” without having had the benefit of operating in the environment where the program is being carried out.  I never gave it much thought until I witnessed it first-hand.  Therefore, it is imperative that the “experts” who are hired to handle M&E issues understand that they may be dealing with people who have little or no knowledge of the hurdles the technical staff face day in and day out.

Sometimes, the mere fact that electricity is not available or the internet connection does not work, may mean that M&E data cannot be incorporated into a database.

While I applaud the importance of M&E in program management, I see some problem areas:

  1. Who monitors and evaluates the authenticity and the accuracy of the M&E plan and its implementation in-house?  In other words, if I am the donor, would I fully trust the contractor or grantee to monitor and evaluate itself?
  2. If the donor hires a third-party to do an independent M&E of a program, how comfortable can the donor and implementer be that the third-party will do an unbiased and truly objective M&E assessment?  What are the chances that the M&E firm will have a former implementer employee evaluating the very same program that person put in place?

Rule of law programs are not immune from a myriad of conflicts of interest.  Who pays attention to these things?

Human Trafficking or Trafficking in Persons (TIP)

In my experience, I have found that there tends to be a disconnect in program implementation between those who are technical experts and those who actually administer the government contract.

Government (USG) officials, companies, and NGOs have expressed concern about the risk of Human Trafficking in global supply chains, including in federal contracts. Victims originate from almost every region of the world; the top three countries of origin of federally identified victims in fiscal year (FY) 2017 were the United States, Mexico, and the Philippines.

Human Trafficking or Trafficking in Persons (TIP) is defined in the Trafficking Victims Protection Act of 2000 (TVPA). It is about people being bought and sold as chattel, involving the acquisition of a human being through the use of force, fraud, or coercion for the purpose of exploiting the individual for profit through forced labor or prostitution.  The USG efforts to combat such trafficking, also incorporated in the United Nation’s Sustainable Development Goal 8.7, are to prevent it, protect the victims, and prosecute the traffickers. A fourth “p”, involves “partnership” between the public and private sectors, with state and local organizations, and with survivors.

The annual State Department TIP Report classifies countries into tiers based on a government’s efforts to comply with the “minimum standards for the elimination of trafficking” found in TVPA.  Although the U.S. is a Tier 1 country according to the 2017 TIP Report, it is also “a source, transit, and destination country for men, women, transgender individuals, and children— both U.S. citizens and foreign nationals—subjected to sex trafficking and forced labor. Trafficking occurs in both legal and illicit industries, including in commercial sex, hospitality, sales crews, agriculture, fishing, manufacturing, janitorial services, construction, restaurants, health and elder care, salon services, fairs and carnivals, peddling and begging, and domestic service.

The National Defense Authorization Act (NDAA) allows the USG to terminate a contract if the prime contractor or subcontractor commits acts that directly support or advance trafficking in persons, such as:  confiscating an employee’s identity or immigration documents; offering employment using fraudulent or misleading pretenses; charging placement or recruitment fees; and providing housing that fails to meet the host country housing and safety standards.

Judicial reform programs might include capacity training on TIP, and it is important to understand how TIP issues can affect the federal contractor in its day-to-day business affairs.

Additional resources: