Improving learning: developing measures of accountability and evaluating their association with students' gains in achievement in Nepal

Research Partners:

University of Michigan

Photo: Schoolchildren in Imadol, Nepal. Photo credit: The Advocacy Project/Flickr under licence CC BY-NC-SA 2.0

Show map
We propose to develop and validate measures of accountability to be shared with the Nepal Ministry of Education (MOE) and to use those measures in an analysis of the determinants of accountability and its association with students' gains in achievement. The proposed study will build on the resources of the Chitwan Valley Family Study (CVFS), a 20-year ongoing panel study of 116 schools with 3,000 households with 3,500 school aged children in 151 communities located throughout the Western Chitwan Valley of Nepal. With funding from DFID-ESRC, we are proposing to achieve two aims:
 
  • Aim One: To Develop and Pretest a Suite of Nepali Accountability Assessment Tools (NAATs) for Use by the MOE and to Pilot these Tools within the Chitwan Valley of Nepal. Importantly, the tools will be designed so that Nepal's MOE can both assess and potentially improve its current accountability processes at multiple levels of the increasingly decentralized Nepalese education system [4]. To achieve this aim we will: (1) develop a variety of accountability assessment tools for use in Nepal's education system; (2) modify a set of instructional processes and instructional quality measures developed for use in OECD countries for use in the Nepali educational system; and (3) gather data on students' academic achievement using standardized test items developed by Nepal's MOE.
  • Aim Two: To Investigate How Accountability Processes; Environments for Student Learning in Schools, Families, and Communities; and Student Learning are Related. This involves investigating three main research questions: Are accountability processes systematically related to socioeconomic disparities among communities, schools within communities, and families within schools? In school and community settings where accountability processes are more intensive, is the quality of instructional service delivery higher? And, controlling for socioeconomic disparities related to student achievement is student learning higher in schools and communities where accountability processes are more intensive?To meet this aim, we will: (1) administer a newly designed PET-QSDS survey to 380 key stakeholders; (2) administer the NASA test at the beginning and end of the school year and a student survey to 1,740 8th graders; and (3) administer a teacher survey to 1,392 teachers and a parent survey to 1,740 parents. The results of this research will be relevant to education policy makers in Nepal and will also contribute directly to comparative education research on school effectiveness.
 
This study will generate rigorous scientific outcomes:
  1. development of a low income context adaptive accountability assessment tool;
  2. cross-cultural assessment of the reliability and predictive validity of accountability measures;
  3. identification of contextual factors with strong correlation with accountability;
  4. potential for identification of new dimensions of accountability in low income settings; and
  5. scientific advancement in our understanding of the relationship between accountability, instructional quality and students' gains in achievement. These outcomes will be made widely available to scientists and policy makers.
 
First, we will conduct dissemination workshops at local and national levels to share findings of the study and provide training on the use of the newly designed accountability assessment tool and analysis of the data generated through the various surveys mentioned above. Second, the data will be made available through ICPSR and the UK Data Service. Third, the findings will be disseminated through presentations at national and international conferences and published in scientific articles, and research and policy briefs. Finally, the participation of Nepali faculty, scientists, government representatives and school authorities throughout the project will advance the scientific and analytical capacity of their respective host institutions (DOE,TU, PABSON, PDs).

Keywords

University of Michigan
Dirgha Ghimire
The methodological tools and protocols, comprehensive measurement, and empirical findings generated by the proposed study have high potential to impact both education policy makers and academia and also contribute directly to comparative education research on school effectiveness at regional and global levels. 
 
Impact on policy: The proposed study is likely to impact education in Nepal in multiple ways. The NAAT and user protocol to measure accountability will be a great asset to the government of Nepal and an important resource to achieve school sector refom plan aims and enhance education quality. Substantively, the empirical answers to the questions we posed: (1) the extent to which accountability is (i) a significant predictor of instructional quality and (ii) distributed unequally across CVFS strata (geographical and urbanacity), different community contexts and across public and private schools; (2) the extent to which any relationships between accountability and students' gains in achievement is explained by instructional quality; and (3) the extent to which the measures of accountability developed in Nepal is a significant predictor of student performance on NASA will provide crucial information for policy makers struggling to make the best use of scarce resources. 
 
We plan to engage stakeholders in several stages of this program to ensure effective policy influence. The key stakeholders include: local teachers, school management committees, proprietors of institutional schools, parents, district education offices, district development committees, Private and Boarding Schools' Organization Nepal (PABSON-Chitwan), I/NGOs, colleges and universities (district level), Central Department of Education at Tribhuvan University (TU), Kathmandu University (KU), Department of Education, Ministry of Education, National Planning Commission, and development partners engaged in educational programs. We will ensure stakeholder engagement through their participation in consultative meetings, research staff trainings, and design and dissemination workshops.
 
Impact on academia: The empirical findings will be of great use to a wide range of academic audiences for teaching and research purposes. At the local level, these audiences include affiliates of TU, KU and other local colleges and universities in Nepal. As indicated in the letter of support, the faculty and students at the Central Department of Education at TU will directly benefit from research findings both substantively and methodologically. At the regional level, the findings will have strong relevance to academic audiences studying low-income countries in South Asia. Our UM/ISER-N joint training program on survey methodology in Nepal has strong collaborative ties with various universities and research organizations in South Asia. Faculty, research scholars, and students from Bangladesh, India, Pakistan, and Afghanistan regularly participate in the training we offer. We will use this venue to share our findings. At the global level, the publication of findings will improve and stimulate further research in this area by scholars worldwide.
We plan to make the CVFS panel data and the new data we propose to collect available through the UK Data Service. We will also promote the data at national and international conferences. This will greatly increase the use of these data and the resulting impact on academia. We expect that the larger scientific community will benefit from this research well beyond the current project period by stimulating new research agendas in other areas of science, such as parental investment in child education, teacher incentives and evaluation.
 
The findings of this study will also have broader implications for the design of education policy worldwide. Our global target audiences are UNESCO, Institute of International Education (IIE), I/NGOs, and donors engaged in educational development worldwide.
Primary theme: 
Themes: 
Grant Reference: 
ES/P005551/1
Project Status: 
Active
Grant Category: 
Research Grant
Fund Start Date: 
February 1st, 2018
Fund End Date: 
February 1st, 2018
education
governance
Country: