Opportunities, Ownership and Tailored Outputs: How to respond to demand for evidence

Opportunities, Ownership and Tailored Outputs: How to respond to demand for evidence
The Impact Lab: Demand for Evidence

Introduction

Top tips for researchers

Top tips for donors

Spotlight

Conclusion

References

Introduction

Many international development research projects aim to influence policy and practice by providing rigorous evidence that impacts on real-life decisions. In fact, the Joint Fund’s requirements from 2009 onwards specified that researchers demonstrate ‘effective demand from, and practical relevance to, decision makers and practitioners in the field’.

However, the world of policymaking – whether organisational, local, national or global – can be complex for social scientists to navigate, and researchers may sometimes find it difficult to assess what demand exists, or respond to demand when it occurs. To ensure the evidence they are generating engages their target audiences, researchers need to interact with a range of different actors, processes and systems – and work through knowledge intermediaries. This process often begins with mapping out who the target audiences are and identifying realistic pathways to reach them.

Why may it be difficult for social scientists to respond to research users’ demands?

Disconnect between supply of funded research and fast-changing demand from policymakers and practitioners
Limited opportunities to network with or influence research users
Lack of existing relationships, reputation and legitimacy with policymakers
Informal, closed or unclear policy processes
Tight timescales within which to respond to calls for evidence
Lack of resources or funded time for policy engagement activities
Lack of pre-prepared audience-appropriate outputs
Ambiguous or incomplete research findings
Conflict between research recommendations and policymakers’ priorities

This Learning Guide recommends ways in which researchers can cultivate a demand for evidence, recognise and create opportunities to influence policy and practice, and nimbly respond to opportunities when they arise. It draws on lessons from four diverse projects funded by the UK's ESRC-DFID's Joint fund for Poverty Alleviation Research, highlighting approaches taken by leading researchers to increase outreach and maximise research uptake and impact. These projects are:

Averting ‘New Variant Famine’ in southern Africa: building food secure livelihoods with AIDS-affected young people (2007–09, Principal Investigator: Professor Nicola Ansell, Brunel University London) which examined whether the way in which AIDS was affecting children was likely to diminish their prospects of food security in adult life. The project investigated evidence to support the ‘New Variant Famine’ hypothesis (suggesting a causal link between high HIV prevalence and food insecurity in southern Africa) which was popular at the time of the study. By working with young people, development practitioners and policymakers in rural Lesotho and Malawi, the project concluded that AIDS is not having the systematic impact on livelihoods assumed by the hypothesis. The New Variant Famine hypothesis is now widely recognised to be incorrect, and this project contributed to this change in thinking.
Making space for the poor: law, rights, regulation and street-trade in the 21^st century (2010–13, Principal Investigator: Professor Alison Brown, Cardiff University) which aimed to understand the risks to urban livelihoods of operating in multiple and contradictory legal and regulatory environments. Through interviews with street traders, local authorities and others in four cities with different legal traditions (Ahmedabad, Dakar, Dar es Salaam and Durban), the project found ‘widespread politicisation of street trade, harassment, evictions and marginalisation of street traders that suggests an urgent need for legal review’. The project generated new ideas on inclusive city design and the informal sector, evidenced by the take up of research findings by urban planners in Tanzania and India, and recent high-level recognition of the issues by UN-HABITAT. The research has also impacted practically on court cases relating to street vendor rights and urban planning.
Measuring complex outcomes of environment and development interventions (2013–16, Principal Investigator: Dr David Wilkie, Wildlife Conservation Society) which aimed to improve policies and practices in the environment-development sector. The project brought together the Wildlife Conservation Society (WCS) – an environmental organisation with projects in 60 countries – with academics and other partners, to identify ways to assess the human wellbeing impact of environment-development activities and to encourage their adoption by practitioners and funders. The project included methodological research and field research in Tanzania and Cambodia, in addition to a practical component to directly inform how environment-development projects are implemented and their impact on human wellbeing assessed. Although recently concluded, the project is already having significant impact on policy and practice – at present approximately 20 WCS projects are using a tool developed through the research and USAID has changed its evaluation practice.
The economic and social consequences of armed conflict in Colombia: evidence for designing effective policies in conflict and post-conflict regions (2010–12, Principal Investigator: Dr Ana María Ibáñez, Universidad de los Andes – University of the Andes, Colombia) which examined the impacts of internal conflict and the channels through which armed conflict affects households. The project used Colombia as a case study, a country that has endured a civil conflict for more than 40 years. The project aimed to understand the effect that armed conflict has had on entrepreneurial decisions in the manufacturing sector and on agricultural production, in addition to the impact on health outcomes caused by the aerial spraying of herbicides to destroy illicit crops. The research has had major impact on the practice of aerial spraying of glyphosate, which was banned in October 2015 following the project’s research paper showing the practice’s negative effects on health and its ineffectiveness to destroy coca plants.

Although these research projects took different approaches in responding to policymakers’ demands, engagement with policy and practitioner actors played a crucial role in generating impact across all four projects. The Impact Initiative studied the projects’ impact evaluation report, which assesses the impact of the first two phases of the Joint Fund for Poverty Alleviation, and conducted interviews with the key researchers involved, to identify seven practical steps that researchers can take to maximise their opportunities to respond to demand for evidence. These recommendations are set out in the next section, along with practical examples from the four projects.

Top tips for researchers

Understand the research users’ needs and priorities
Investing time and resources before the project’s inception in researching potential users – their needs, financial and political priorities – will ensure the project is set up to supply evidence that is in demand, and has a credible Pathway to Impact. This may involve desk research such as reading through existing policy documents, or arranging informal meetings, interviews and workshops with key individuals. Better still, involving a research user as a partner in the study will ensure their needs and interests are fully taken on board at every stage of the project. ESRC provide support and guidance on developing a good Pathways to Impact plan in the ESRC Impact Toolkit, whilst the UK Collaborative on Development Sciences (UKCDS) provide a useful guide on Finding and Building Effective Partnerships along with a range of resources on relationship building and collaborative working.
Example:
Measuring complex outcomes of environment and development interventions
The Principal Investigator, Dr David Wilkie, works for practitioner organisation the Wildlife Conservation Society (WCS), which meant he already knew that there was considerable demand from within the organisation for the research. He explained, ‘I meet our field staff all the time… and I get a sense from people what they are or aren’t interested in.’ This guaranteed that the project was relevant and timely in addressing questions being asked within the organisation. WCS’s long-standing relationship with the academic partners in the project was also furthered through student placements across the organisations. The partnership paid off as the project’s recommendations have been quickly rolled out within WCS – approximately 20 WCS programmes are already using a tool developed through the research, called the Basic Necessities Survey.
‘Involving a research user as a partner in the study will ensure their needs and interests are fully taken on board’.
Encourage ownership and buy-in from the start
If demand for evidence appears low in a particular subject,researchers should find ways to cultivate a demand for evidence, right from the start of their project. To encourage ownership, all four research projects systematically consulted research users throughout, from the design phrase through to dissemination. This meant the projects could be shaped and adapted – for example, modifying the research questions, field site locations or intended outputs – and the research users felt invested in the study. Some of those consulted also became knowledge intermediaries, who could package, frame and share research with users. This process of knowledge brokerage is essential for linking up research supply and demand.
Example:
Averting 'New Variant Famine' in Southern Africa: Building food-secure livelihoods with AIDS-affected young people
Prior to the project starting, the research team gathered numerous policy documents and engaged with potential users through their professional networks, to assess the gaps in the existing evidence and understand what was being demanded by research users. They then established National Steering Groups in Lesotho and Malawi, which included representatives from government, non-governmental organisations (NGOs), the UN and donor agencies. These groups provided helpful information on the policy context, shaped the research design and encouraged buy-in from stakeholders. When it came to discussing the research findings, the researchers held policy workshops where participants were invited to actively interact with the data, and draft their own policy recommendations (co-production of research). The Principal Investigator, Professor Nicola Ansell, explained the benefits of this approach: ‘If you want to have an impact on policy, you have to transfer the ownership of the findings.’ This approach meant that policymakers were able to directly apply the research to their own contexts.
‘If you want to have an impact on policy, you have to transfer the ownership of the findings.’
Map and utilise networks to identify opportunities and access policymakers
Researchers themselves may not have the appropriate networks and relationships to gain access to practitioner, policy or donor (and research funder) circles. To tackle this, teams should include in their planning a review of the quality of relationships between themselves, partners and key stakeholders in order to see what existing networks are available and where new partners or links need to be made. Stakeholder mapping and evaluative tools and methodologies such as PIPA, Outcome Mapping and Net-Map can help to identify priority stakeholders and also highlight weak areas that need to be addressed. Building up relationships that lead to awareness of what opportunities (for influence, networking or funding) may be coming up in the future, will mean researchers can plan their activities well in advance so that they are aligned with the needs of practitioners and the interests of funders.
Example:
Measuring complex outcomes of environment and development interventions
Principal Investigator Dr David Wilkie has cultivated close relationships with key stakeholders, such as with USAID, which opened up opportunities to influence policy and shape future funding calls. Dr Wilkie discussed the project with staff from USAID at a workshop, and detected a clear demand for evidence. The knowledge gained from this relationship meant that the project could be designed and adapted to include the audience’s needs. Dr Wilkie explains, ‘USAID were interested in the question of attribution, and were keen that our research tackled this. It encouraged us to tweak our study to consider this issue too.’ This approach paid off as USAID’s Central African Regional Program for the Environment has now adopted the Basic Necessities Survey as the standard way to assess human wellbeing over time.
‘Teams should include in their planning a review of the quality of relationships… to see what existing networks are available and where new partners or links need to be made.’
Adapt and respond to external events
Researchers may find it helpful to look outside their specific research interests to wider (perhaps national or global) contemporary narratives, events or political milestones that are taking place during the project’s lifecycle. Aspects of the research can then be reframed so that they are relevant to these external opportunities, which often provide a hook for communication activities such as media engagement, blogs, social media, publications and events. Funding and time for appropriate skilled communications professionals to advise on or deliver these activities should also be factored into the project’s budget. Finally, establishing a calendar of key external events during the project can be useful in keeping track of potential external opportunities.
Example:
The Economic and Social Consequences of Armed Conflict in Colombia: Evidence for Designing Effective Policies in Conflict and Post-Conflict Regions
The project benefitted from external events happening at opportune times during the project. A change of government took place, and the new government entered into peace talks with Colombia’s rebel armed forces (Fuerzas Armadas Revolucionarias de Colombia, known as FARC). The project therefore became much more relevant as the knowledge that it had generated became crucial for the peace talks. The research team used media and social media engagement to frame their project as directly relevant to these current events. For example, they distributed a Policy Briefing to media outlets – some of which went on to contact the team directly for interviews – and invited journalists to attend two workshops. This continuous external engagement led to extensive dissemination of the project’s paper on aerial fumigation and health outcomes in Colombian media outlets, and also shifted debate about the issue amongst journalists.
‘Establishing a calendar of key external events during the project can be useful in keeping track of potential external opportunities.’
Convene debates that create new understandings and strengthen relationships
Very occasionally a research project may appear perfectly timed to coincide with a policy window of opportunity – at just the moment that the research is being concluded, policymakers are seeking answers on a particular issue that the project directly addresses. However, the path to policy influence is more often less smooth, and in many cases there will be no current policy window. In this case, researchers may find it beneficial to focus on convening discourse and debate that creates new understandings and further strengthens relationships with key audiences. By engaging in a process of knowledge exchange with the wider research and practitioner community, researchers can contribute to broader debates that have the potential to generate demand or further funding for research in the future.
Example:
Making Space for the Poor: Law, Rights, Regulation and Street-Trade in the 21st Century
Within the countries of study, this project made the most of a number of existing policy windows of opportunity. For example, there was a new local government in Dar es Salaam that was receptive to fresh ideas, and a law in India to guarantee space for street vendors that was under review during the project. On an international level, however, no live policy window existed. To address this, the team created a number of opportunities to convene debate on the research, including meetings and presentations with stakeholders. For example, the Principal Investigator Professor Alison Brown developed particularly strong relations with UN-HABITAT. Staff from UN-HABITAT were included as project advisors and attended one of two international feedback workshops, commenting very positively on how the research findings were made relevant to the policymaker and urban planner audience.
Further engagement with UN-HABITAT included presenting at roundtable dialogues and the World Urban Forum in 2010, 2012 and 2014 which was attended by NGOs, the private sector, researchers and the media. Professor Brown’s input informed a UN-HABITAT and International Labour Organization issue paper for the 2016 Habitat III conference, the first time the topic had received such high-level recognition. A UN-HABITAT employee explains the project’s impact: ‘[Urban law and the link to the informal economy] became one of the thematic areas that we paid more attention to. [The project] pushed the agenda.’
‘Researchers can contribute to broader debates that have the potential to generate demand or further funding for research in the future.’
Prepare dissemination products for a variety of audiences
Often opportunities to influence policy and practice may present themselves with very short timeframes, such as a response to a media or political announcement or a chance meeting at a conference. Without the right dissemination products and tools in place, responses may be weak, unclear, or too late. To counter this, researchers should invest time at the start of the project in understanding how they prefer to receive or access information. The project’s outputs should then be tailored accordingly, including ensuring products are translated into local languages. For example, one project engaged with grass-roots organisations through events jointly hosted with local partners, where a summary paper of findings was translated into local languages. Researchers can also repackage and repurpose outputs for different scenarios – such as case studies that can be issued to media to illustrate a topical news story, or toolkits that can be adapted for different practitioner audiences.
Example:
The Economic and Social Consequences of Armed Conflict in Colombia: Evidence for Designing Effective Policies in Conflict and Post-Conflict Regions
In addition to outputs designed for an academic audience, the project team produced a range of products aimed at key stakeholder groups, such as Policy Briefings, presentations and a video. They also published a book about the project in Spanish, which was distributed widely to policymakers across Colombia, with the objective of engaging a wider audience in the project. The book has proved very popular, and it is currently in its third reprint due to high sales. When media interest in the project grew, the book proved a useful tool, and it was featured in several media outlets.
‘Researchers should invest time at the start of the project in understanding how [their audiences] prefer to receive or access information.’
Maintain contact with audiences over time
It is important that researchers continue to build and develop relationships with key stakeholders over time, even beyond the project’s lifecycle. Impact takes time, and opportunities to respond to demand for evidence may well occur in the months or years following the project. Continuing to engage with stakeholders – such as through one-to-one contact, group email newsletters or events – will maximise the project’s impact, particularly if external events mean the project becomes more relevant and topical in due course. Of course ongoing activities such as these require resourcing, and researchers should consider applying for follow-on funding (for example, ESRC's Impact Acceleration Accounts are block awards that ESRC make to research organisations to accelerate the impact of research) or identifying synergies with other projects.
Information and guidance about ESRC funding, including the IAA Accounts, can be found at: http://www.esrc.ac.uk/funding/
The Impact Initiative was set up to identify synergies across research within the ESRC DFID Joint Fund for Poverty Alleviation Research and the Raising Learning Outcomes in Education Systems Research Programmes and provides project information on these grants.

Example:
Making Space for the Poor: Law, Rights, Regulation and Street-Trade in the 21st Century
Continuity was a key characteristic of the project’s engagement activities. The research team stayed in touch with research users during and after the project, which enabled them to identify policy opportunities as they arose. By reemphasising the project’s findings time and again, doors were opened at UN-HABITAT. One UN employee commented, ‘The Principal Investigator helped us advocate for something. She really kept bringing it to our attention and helped us integrate it into our work. It’s a long-term thing but I think [the project] really started the ball rolling.’
‘Researchers should consider applying for follow-on funding or identifying synergies with other projects.’

Project spotlight

credit:kingstongal/Flickr - Looking down on Bogota from the top of Monserrate

The Economic and Social Consequences of Armed Conflict in Colombia: Evidence for Designing Effective Policies in Conflict and Post-Conflict Regions

The ‘Economic and social consequences of armed conflict in Colombia: evidence for designing effective policies in conflict and post-conflict regions’ research project took place at a crucial time in Colombia’s recent political and military history. With a new government seeking evidence that could be used in peace talks with armed rebels, the team adopted a range of strategies and tools that led to tangible policy change.

The challenge

In June 2016, the Colombian government signed an historic ceasefire with armed rebels Fuerzas Armadas Revolucionarias de Colombia (known as FARC), signalling an end to five decades of civil conflict. Colombian President Juan Manuel Santos received the Nobel Peace Prize in 2016 for his contribution to the peace deal.

One of the main strategies that has been used in Colombia to fight the illegal drug production that has fuelled the conflict is the aerial spraying of the pesticide glyphosate on coca crops, the raw material for producing cocaine. Hundreds of thousands of acres of countryside have been sprayed since 1999 – an approach that has been defended by the United States yet attacked by NGOs and opponents of the so-called ‘war on drugs’. In March 2015, the International Agency for Research on Cancer determined that glyphosate is ‘probably carcinogenic to humans’, raising questions about the health effects of the spraying campaign.

Against this backdrop, researchers at Universidad de los Andes had been researching the effects of the conflict, and the impact of aerial spraying. Their research found that exposure to glyphosate increases the number of medical consultations related to dermatological and respiratory related illnesses and the number of miscarriages. When Juan Manuel Santos was elected President of Colombia in 2010, his administration were looking for evidence to influence their negotiations with the rebels – providing the researchers with a key opportunity to influence national policy.

The solution

During the project’s design stage, the team contacted a number of policymakers to establish what evidence would be most relevant for the country, and to understand their needs and priorities. They also organised two seminars to engage policymakers. An External Advisory Committee was then established, made up of representatives of key stakeholder groups. The Committee’s purpose was to guide the research questions and provide a forum for discussing the results – it also guaranteed participation of research users, giving them a sense of ownership.

The researchers used media and social media engagement to generate interest in the research, particularly around the launch of the paper on aerial fumigation’s impact on health outcomes. The project was led and carried out by Colombian researchers who understood the national policy context, were well respected and had the necessary networks to identify opportunities to access policymakers. In this case, the reputation of Principal Investigator Dr Ana María Ibáñez was particularly important – Dr Ibáñez was known to President Santos, who invited her to present the research to his cabinet.

The project greatly benefited from external events happening at opportune times, such as the change in government and increased public awareness of the impacts of glyphosate. It was crucial that the team were prepared with tailored dissemination products to respond to these opportunities, and they were able to answer requests from journalists for interviews. They also created opportunities for debate and discourse, such as organising presentations with staff at the United Nations Office on Drugs and Crime and the US Embassy in Bogota.

The outcome

The project has seen capacity-building impact amongst research assistants, one of which has since become a government analyst advising on policy issues. It has also had conceptual impact by shifting debates among researchers and journalists about aerial spraying, with media articles about the resignation of one of the researchers as president of the Advisory Commission on Drug Policy emphasising the project’s findings.

The main impact of the project has been instrumental, as the results have significantly influenced national policy and strategy. Dr Ibáñez has advised the President directly on the consequences of conflict highlighted by the project, and the findings have been used in peace talks with the FARC. In 2015, a ban came into effect on aerial spraying of glyphosate. Although other scientists contributed to this ban, the research team certainly influenced the debate and final outcome.

Conclusion

Finding out which actors are demanding research evidence, building relationships with these individuals and organisations, and responding to windows of opportunity when they occur, can be a daunting and time-consuming task for researchers. Mapping out a clear pathway to impact before a project begins, and keeping track of opportunities and tasks using a forward planning calendar, can ensure researchers keep focused on impact throughout the project’s duration and beyond.

This Learning Guide has provided a series of steps that researchers can take, to maximise their chances of influencing their target audiences with their project’s findings. Evidence from these four research projects has shown that the value of relationships and partnerships should not be underestimated, whether in providing crucial insight on the questions being asked by policymakers behind closed doors, or in creating opportunities for networking. This suggests that time invested in developing partnerships – before, during and after a project – is time well spent.

The Impact Lab: Demand for Evidence

References

ESRC-DFID Joint Fund for Poverty Alleviation Research:
http://www.esrc.ac.uk/research/international-research/international-development/esrc-dfid-joint-fund-for-poverty-alleviation-research/
France, J., Rajania, A., Goodman, R., Ram, M., Longhurst, R., Pelka, V. and Erskine, C. (2016) Evaluating the Impact of the ESRC-DFID Joint Fund for Poverty Alleviation Research:
http://www.theimpactinitiative.net/2016-impact-evaluation-joint-fund
What is impact? ESRC impact toolkit:
http://www.esrc.ac.uk/research/impact-toolkit/what-is-impact/
Averting ‘New Variant Famine’ in southern Africa: building food secure livelihoods with AIDS-affected young people: http://www.theimpactinitiative.net/project/averting-new-variant-famine-southern-africa-building-food-secure-livelihoods-aids-affected
Making space for the poor: law, rights, regulation and street-trade in the 21st century: http://www.theimpactinitiative.net/project/making-space-poor-law-rights-regulation-and-street-trade-21st-century
Measuring complex outcomes of environment and development interventions: http://www.theimpactinitiative.net/project/measuring-complex-outcomes-environment-and-development-interventions
The economic and social consequences of armed conflict in Colombia: evidence for designing effective policies in conflict and post-conflict regions:
http://www.theimpactinitiative.net/project/economic-and-social-consequences-armed-conflict-colombia-evidence-designing-effective
Basic Necessities Survey:
https://programs.wcs.org/carbon/communities/wcs-resources/publications.aspx
UN-HABITAT and International Labour Organization issue paper for the 2016 Habitat III conference: https://www2.habitat3.org/bitcache/5374f5c259f414ce521e9704254a04d2b5b845da?vid=542875&disposition=inline&op=view
Maria Alejandra Arias, Adriana Camacho, Ana María Ibáñez, Daniel Mejía y Catherine Rodriguez. (compiladores) (2014). Costos económicos y sociales del conflicto en Colombia ¿Cómo construir un posconflicto sostenible? https://economia.uniandes.edu.co/component/booklibrary/478/view/47/Libros/837/costos-economicos-y-sociales-del-conflicto-en-colombia-como-construir-un-posconflicto-sostenible