by Sarah Brown
This edition of our quarterly highlights collection focuses on the role of evidence in policymaking. It’s a theme we’ve curated collections around regularly, but our readership figures for these articles remind us time and again how important our community find this topic.
So, our first article on this theme by authors Clementine Hill O’Connor, Katherine Smith, and Ellen Stewart explores the question of how to balance evidence with public preferences.
How can policy organisations deal with competing (and sometimes conflicting) imperatives to strengthen the role of evidence in policy, with simultaneous calls to better engage diverse publics? Academic research has much to say about both the value of evidence for policymaking to increase (or improve) the policymakers’ engagement with evidence AND investigating a wide range of methods through which publics can be involved in policymaking. Perhaps surprisingly, these contributions are rarely connected. This disconnect is the focus of Integrating Evidence and Public Engagement in Policy Work: An empirical examination of three UK policy organisations.
This article explores how these simultaneous contributions intersect within three organisations working at different levels of UK policy: local (Sheffield City Council), regional (Greater Manchester Combined Authority) and national (devolved) (Scottish Government). Employing documentary analysis and interviews, the authors argue there are similarities in approaches to evidence and engagement, including: conceiving of both ‘data’ (statistics tracked by internal analysts) and ‘evidence’ (external analysis) in primarily quantified terms; and a tendency to limit the authority of publics to advising and consulting on predefined issues. Yet, the research also found growing interest in more in-depth understandings of publics (e.g. via ‘lived experiences’) albeit with some uncertainty about how to use these qualitative insights in settings that have institutionalised quantitative approaches to evidence. The research findings identified four distinct clusters: (1) prioritising public engagement; (2) strategically using public engagement and evidence to support policy proposals; (3) prioritising quantified evidence and data; and (4) attempting to integrate these distinct knowledge types.
The authors found that the majority of interviewees were in cluster 4, trying to integrate quantified data and evidence with qualitative insights about public views and experiences. However, it was clear that this type of integration was challenging to policymakers and there were few, if any, approaches that seemed to fully achieve the kind of integrated insights they were seeking.
The authors conclude that integrating these different forms of knowledge is not only practically challenging, but is complicated by the fact that (as alluded to by the disconnect between the scholarship on evidence use and public engagement) there are differences in what each seeks to show, and how their robustness might be judged. Nonetheless, this is an important and potentially impactful avenue for future research to understand and bridge these different insights and perhaps even begin to develop mechanisms to support the better integration of these different forms of knowledge.
Our 2nd article in the collection, from Annette Boaz and Kathryn Oliver, analyses How well do the UK government’s ‘Areas of Research Interest’ (ARI) work as boundary objects to facilitate research use in policymaking? The context for the research is that articulating the research priorities of government in the form of ‘Areas of Research Interest’ developed by government departments, is one way to encourage the production of relevant research to inform policy.
In interviews with key stakeholders working on ARIs, the authors found that much of the activity designed to support research use in policy focuses on the boundary between research and policy communities, with a wide range of interventions including policy fellowships, training programmes and the development of intermediary organisations. They found less of a focus on the role of the packaged information used to support research use. These are typically boundary objects, such as guidelines and toolkits that help to convey key information. They found that these can take on a variety of forms, ranging from a policy document to a drawing or a metaphor. The intention is that these objects are used to share and exchange information across the many boundaries that impact upon all of us. Boundary objects tend to need support from boundary workers (sometimes described as intermediaries, brokers or boundary spanners) who work to make sure the object can be useful and used.
In this way, boundary objects have the flexibility to move between, and be understood by, different communities, with implications for the ways in which the ARIs are crafted and shared. For example, ARIs are often considered as a mechanism for communicating departmental research interests to external academic stakeholders, and are also used for government departments to learn about each other’s research priorities. The application of ARIs in the UK policy context involves a constant interplay between boundary objects, practices and people, all operating within the confines of existing systems and processes.
In conclusion, the authors argue that, while ARIs have an important role to play, they are no magic bullet. Nor do they tell the whole story of governmental research interests. Government departments are unlikely to include highly contentious or sensitive topics in their ARIs. Although it’s tempting to settle upon a single solution to improving research use, optimising the use of research in policy making requires the galvanisation of a range of mechanisms, including ARIs, in a coordinated way. Only then will we start to see more clearly how useful research can be to policy.
Our final article by Natália Massaco Koga, Miguel Loureiro, Pedro Lucas de Moura Palotti, Rafael da Silva Lins, Bruno Gontyjo do Couto, and Shanna Nogueira Lima examines sources that Brazilian information bureaucrats use in policymaking in order to find out what informs policy. Reflecting on the premise that the evidence-based policy (EBP) movement encourages policy actors to use scientific evidence on ‘what works’ to improve public policies, they remind us that empirical research shows that even bureaucrats in Anglo-Saxon countries who are strongly influenced by this movement, do not use academic sources widely. Instead they often prefer other sources of information such as news media, public opinion and peers. So what informs policy in countries with low EBP influence?
In the article, the authors give an overview of the sources of information used in policy work using data from 2,180 Brazilian bureaucrats. The research found that there is a prevalence of use of government sources, especially among bureaucrats performing analytical and oversight tasks, and those in higher positions. Academic sources are associated with higher analytical capacity (of the individual and organisation), but not with any particular policy sector. By investigating an important yet often neglected issue in EBP – the role of different types of information and how they inform policy – this article contributes to the literatures on policy work and policy capacity, especially given its empirical focus on Brazil.
Based on these findings, they propose two important policy prescriptions. Firstly, as contextual factors are relevant to determine which sources of information public officials use, then enhancing analytical capacity seems particularly important. This is crucial if we are to expand and improve the use of scientific knowledge in policymaking. One way of doing that is to strengthen the relationship between individual and organisational capacities. Secondly, we can’t ignore the inherent political nature of policymaking and the necessity of combining scientific sources with other sources of information. Therefore efforts to build good evidence-based governance systems need to recognise that other types of knowledge beyond scientific knowledge can also count as evidence and can be equally valuable.
We hope you enjoy this collection. As always, we welcome your feedback and thoughts on the articles featured.
Please look out for our new Open Access Virtual Issue coming on 10th May.
You can read the original research in Policy & Politics:
Boaz, A., & Oliver, K. (2023). How well do the UK government’s ‘areas of research interest’ work as boundary objects to facilitate the use of research in policymaking?, Policy & Politics, 51(2), 314-333 from https://doi.org/10.1332/030557321X16748269360624
Hill O’Connor, C., Smith, K., & Stewart, E. (2023). Integrating evidence and public engagement in policy work: an empirical examination of three UK policy organisations, Policy & Politics, 51(2), 271-294 from https://doi.org/10.1332/030557321X16698031794569
Koga, N. M., Loureiro, M., de Moura Palotti, P. L., da Silva Lins, R., Gontyjo do Couto, B., & Nogueira Lima, S. (2022). Analysing the information sources Brazilian bureaucrats use as evidence in everyday policymaking, Policy & Politics, 50(4), 483-506 from https://doi.org/10.1332/030557321X16588356122629
The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of Policy & Politics, the Policy Press and/or any/all contributors to this site.
Policy & Politics Journal Blog 