PPP Teams

About PPP (Project Planning & Performance) Teams

Program Planning and Performance Evaluation– “What is the Current Plan”
 
Planning for Water, Sanitation and Hygiene programs and monitoring their performance is vital to the success of Wasrag’s Start with Water program to develop sustainable WASH programs around the world. Development of the Wasrag program for WASH projects will include assessing the needs in each area, prioritizing those needs, developing projects and secure funding to address those needs and ongoing monitoring and evaluation to assure the success and sustainability of the program. All of this will be done through communication with the area residents, local Rotary Clubs and the Regional Team.  
 
To achieve this goal, Wasrag is working with the Water Institute at the University of North Carolina and the University of Cape Town, South Africa to develop, test and refine an interactive SMS/SCADA –based monitoring and evaluation system. Program Planning and Performance Evaluation will be done by teams of volunteers in the field under the direction of the Regional Teams and Wasrag and real-time performance reporting using the latest available communication technology.
 
**NEW** 
The final report from UNC is now available.  Just go to this link:  WASRAG UNC Final Report 2012 08 14.pdf
 
Also available:    the data collection protocol and decision support aid developed by UNC. 
 

What is the purpose of Program Planning and Performance Evaluation?

 
The primary goal of Program Planning is to identify all the water access points, sanitation facilities and hygiene practices in a specified region (watershed) and establish their ability to meet the needs of the local community, as established by the community, or by compliance with regional or national standards, where applicable. Where deficiencies are found, programs will be identified and priorities established to implement programs that will satisfy those needs. The planning and priority development process is to ensure that Rotary Club water, sanitation and hygiene programs undertaken in each geographic area are sustainable and effective by providing support and follow-up monitoring work.
 
Evaluation is an essential component of any program and must be outlined and set in motion before the program begins. Evaluation defines in advance indicators for assessment and what data is required to assess the impact of the program. This process helps to confirm that all necessary data will be available and confirms the essential components of evaluation.
 
Throughout program implementation, periodic regular monitoring is required.  After programs are completed, overall evaluation determines if programs are:
  • functioning as designed;
  • meeting community needs; and,
  • in compliance with applicable standards.
Programs with problems will be repaired or upgraded to regain appropriate operation and sustainability.
 
What are characteristics of a sustainable program?
 
Sustainability is a complex idea.  Here are some of the characteristics:
  • demand driven –  the program is developed based on community needs
  • community ownership – the community must be involved in developing and implementing a program. Experience has shown us that, without a strong sense of participation and ownership, communities don’t engage with programs and programs or projects are not successful;
  • cost effective
  • meet water quality and health standards
  • confirming the evaluation plan up front
  • take into account flow requirements, seasonality etc. reflecting a thorough understanding of the local environment, watershed etc.
  • meet community needs over time (at least five years)
What are the objectives of an effective WASH program?
 
Providing safe water and sanitation is a means to an end, a foundation, leading to outcomes that improve public health, community functioning and quality of life.
 
The set of objectives that should guide our programs include:
  • disease reduction
  • better education, literacy and numeracy
  • reduced morbidity and mortality in the general population
  • specific reduction of maternal and child mortality
  • reduction of poverty, improved economic well-being
  • improved health of general public
  • hunger alleviation
  • improved crop production and nutrition
  • enhanced productivity of the working public
  • enhanced gender equality
  • viable program that is sustained for the projected life
  • environmentally sensitive program that considers the impact on downstream communities
To achieve the greatest good, every WASH program must be assessed in the context of overall humanitarian, social and economic development. Sometimes, Rotary Club WASH programs are conducted in parallel with other complementary programs, which taken together, address all desired outcomes.
 
Why do programs sometimes fail?                 
 
Programs and their related projects suffer for many reasons, but some common issues include:
 
  • Use of inappropriate technologies, such as, too technical for the situation, too difficult to maintain, poor choice based on all regional needs, etc.
  • Implementation of programs or projects before the financial capacity of the community has been determined and proven,
  • Absence of community buy-in, water committee challenges (ranging from poorly trained committee members, training that is too rapid, poor financial systems and tariff policies, and turnover),
  • Weak external actors (like the private sector) who the community will need over time
  • Inability to access spare parts and technical assistance, as needed, and
  • Environmental/site factors, eg. A borehole that no longer yields water
Each potential problem will be addressed in the monitoring protocol developed by the UNC Water Institute to reduce or eliminate as many as possible. Revisions to the plan and protocol will be made as new problems are identified.

How will success be measured?                 
 
Measures of success will be developed in collaboration with Regional Teams and Wasrag . These measures could include:
  • establishment of effective regional teams;
  • baseline needs assessment(s) undertaken and completed;
  • quantity of monitoring and evaluation undertaken and completed
  • success (or failure) of programs (based on an evaluation scale that will be developed)
  • confirmation of endorsed programs/projects, identified and uploaded to the website;
  • support provided to further develop programs once rotary clubs have come onboard;
  • implementation and completion of programs that are technically, culturally, environmentally and financially sound
  • programs/projects monitoring indicates effective outcomes and sustainable results for three, five and ten-year evaluation cycles.
Who should be part of a Program Planning and Performance Evaluation Team?
 
The Regional Team responsible for a region will be consulted in the selection process for Program Planning and Performance Team members and will work with Wasrag to select the Team leader. Team members may include local Rotarians, respected professionals or NGO members living in the country. Other technically and professionally qualified members may be selected depending upon the expertise needed for each assessment or monitoring site visit. In general, the team will consist of five or six members that will spend approximately one week in country to evaluate the area, as assigned by Wasrag and the Regional Team.  
 
Initially, The Water Institute at UNC and Wasrag will pilot test the Program Planning and Performance system in five Rotary Districts – one each in Malawi, Ethiopia, India, and one in a Central or South American country yet to be identified and one in a RI/USAID partnership country (Dominican Republic, Philippines or Ghana). These initial tests will occur in 2011 and will be led by a member of the UNC team. Upon completion of  the pilot tests, feedback will be used for  improvements and modifications to the planning and evaluation protocol. Full use of the Program Planning and Performance Evaluation process is anticipated by the end of 2011.

What knowledge is required?
 
It is vital for all team members to have a good knowledge of local political and cultural issues as well as some level of technical expertise in the WASH sector. At least some team members must be proficient in the local language to facilitate communication during interviews for data collection. Members of local sponsoring Rotary clubs will be called upon to provide some of this expertise.
 
How important is an understanding of local issues?
 
Local issues and concerns frequently have a significant impact on setting priorities, planning, implementation and long-term outcomes of a program. An understanding of local issues, carefully taken into account early in the data collection and evaluation process can avoid many pitfalls. Again, local sponsoring Rotary clubs are important partners in this process.
                 
Who leads a Program Planning or Performance Evaluation Team?
 
The Team Leader for a Program Planning or Performance Evaluation Team will be a volunteer selected from the volunteer data base that is developed and managed by Wasrag. The Regional Team shall assist Wasrag in making the final selection based upon the skills required for the specific field assignment.
 
What is the role of a Program Planning or Performance Evaluation Team Leader?
 
The Program Planning or Performance Evaluation Team Leader is the key contact and lead for each specific team. S/he is responsible for:
  • organizing and implementing the Team work assignment;
  • providing leadership, training and direction to the Team;
  • communications within the Regional Team, and with Wasrag;
  • preparing preliminary and final reports for the assigned tasks within the Region;
  • gathering information about lessons-learned, what does and doesn’t work, and sharing this knowledge;
Is there a process that ‘Program Planning or Performance Evaluation’ teams should follow?
                 
The Start with Water Program Planning and Performance Evaluation program is based on a process under development through consultation with Rotarians and other WASH experts around the world. The tools and detailed protocol are being developed by WASRAG in partnership with the Gillings School of Global Public Health at the University of North Carolina under the direction of Dr. Jamie Bartram. This is a multi-step process that can be summarized as follows:
  • At the heart of the planning and performance evaluation process is a network of Regional Teams located in Rotary Districts around the world. Regional Teams are comprised of knowledgeable Rotarians, many of whom live in the local area and who have experience in water, sanitation, health and hygiene work,. This means that a Regional Team has an understanding of the local culture, political, geological and geographic realities. They live and work in the area, and can provide support, oversight, and long-term monitoring of programs.
  • Each Regional Team will be responsible for a geographic area, and will be familiar with who is working on what in their region (not just Rotarians);
  • With WASRAG support, the Regional Team selects an area (watershed) for mapping and preparing a baseline needs assessment. With WASRAG assistance, the Regional Team selects a Team Leader and team members to map all pertinent water points, sanitation facilities and local hygiene practices. Through interviews with local citizens, community leaders, and institutions, the team determines the current status of each point, needs are documented and community interest in improvement is established. A report will be prepared to document the findings of the field study that rates each site according to need. Once a good knowledge of the local situation is completed, the Regional Team will work with resources at Wasrag, the Baseline Needs Assessment team and other experts to develop a long-term plan and a list of ‘endorsed programs/projects.’
  • The Baseline Needs Assessment and corresponding list of endorsed programs/projects will be entered on the Wasrag website;
  • Completed programs/projects will be reviewed and documented using a procedure of real-time local data collection and reporting developed by the Gillings School of Global Public Health. Regional Teams and Wasrag will monitor endorsed programs/projects during and after completion, and may monitor other Rotary programs/projects located in each area in a similar fashion. If a completed program/project needs remedial work, it can be listed on the website as a program/project seeking Rotary Club involvement. It is assumed that the original Clubs involved in the program/project would be contacted first to undertake remedial work. If they don’t want to be involved, or can’t for some reason, other Clubs would be encouraged to step in.
What role does the Program Planning and Performance Evaluation team play?
 
The Regional Teams and the Program Planning and Performance Evaluation teams are the heart and soul of the Rotary WASH program. The Regional Team will select watersheds within a Rotary District and will assist in selecting field teams. The Program Planning and Performance Evaluation teams will be responsible for data collection, mapping, on-site interviews and report preparation required to identify and prioritize programs/projects. The Regional Team will select programs/projects for inclusion on the ‘endorsed programs/projects’ list at the Start with Water web site for implementation by Rotary clubs around the world. The on-site mapping technology, interviewing skills, interview format and requirements for the final report will be included in a training program, that each team will complete prior to site visits. This training will be developed by The Water Institute at UNC.

What support does Wasrag offer the Program Planning and Performance Evaluation Team?
 
The Wasrag liaison works with the Water Institute at UNC and Regional Teams around the world to support field data collection, report preparation, plus program/project selection and prioritization, once completed by field teams. Wasrag offers technical, educational, operational, evaluation and communications support to Planning and Performance Evaluation Teams.

What tools are available to the Team?
 
Various tools are available to help Program Planning and Performance Evaluation Teams, such as:
  • local arrangements in the field evaluation region, e.g., transportation, food and lodging, translators
  • equipment and materials required for interviews and data collection
  • assistance in planning transportation to and from the field site
  • our website, which includes
    • a learning centre which includes articles of interest, country information, links to other useful sites;
    • forum - to exchange ideas and lessons learned;
    • access to “ask an expert”;
    • program/project list and search capability;
    • communication tools such as brochures, presentations etc.;
    • news;
  • A cadre of experts who have volunteered their time and knowledge and/or have been part of a similar team on a similar assignment
  • Leadership at Wasrag, available to provide support;
Does the Team have a budget to work with?
 
The work of a Program Planning or Performance Evaluation team is performed on a volunteer basis; however, as finances permit, pre-approved expenses may be reimbursed on a case-by-case basis,. Transportation within country, meals and lodging may be provided by local Rotarians. Normally all time required for field visit and related transportation will be the responsibility of the volunteer. For more information about this, contact the Regional Team Liaison for each specific assignment.
 
How long does it take for a Program Planning or Performance Evaluation Team to complete an assignment?
 
The entire assignment process will normally be completed in three (3) months or less. This timeline includes Team selection, establishing dates for field data collection, training, travel arrangements, field site visit for interviews and data collection, plus preparing preliminary and final reports. The field data collection will usually take approximately one week. However, it may be longer in an area where community and water point access is difficult.

What is the projected implementation schedule?
 
The schedule for implementation of the Program Planning and Performance Evaluation system is as follows:
  • develop baseline on-site interview questions and procedure format   May 2011
  • identify hardware and software for mapping and data   June 2011
  • identify up to three regions around the world for pilot tests and have teams selected for field visits (with regional team help)                              Aug. 2011
  • train teams and conduct site visits   Sept. 2011
  • prepare and post reports   Oct. 2011
  • revise and update process and protocols    Dec. 2011
  • begin baseline needs assessments in Rotary Districts with
  • established regional teams   Jan. 2012
  • identify programs/projects from pilot visits and begin work   Jan. 2012

During 2011 while the Program Planning and Performance Evaluation processes and protocols are being developed and tested, Rotary teams are encouraged to organize and perform a Baseline Needs Assessment in a region (watershed) of a Rotary District where there are current and active program. Wasrag will work with these teams to provide as much information as is available at the time of the site visit to make your Assessment viable for programs selection and implementation. One Assessment has been completed in Remba Island Kenya in January 2011.