SPY00157M Exploring Policy Research: Comparative Review Sample (University of York)

For information and reference only – not to be submitted as your own work. Use this extract to study how a critical comparative review is structured, then choose your own two papers and write in your own words. Follow your institution’s rules on generative AI and academic integrity; this module’s brief allows no GenAI use at all.

This extract covers a postgraduate critical comparative review for SPY00157M Exploring Policy Research (MPA, University of York). It compares one systematic review and one case study on Housing First, condensed from the full 2,500 words. For marker expectations and a step-by-step plan, read our blog on how to write a critical comparative review of policy research papers.

What the brief asks

Title page title

A comparative review of two selected homelessness policy research papers: a systematic review and a case study of Housing First

Introduction

This review compares two papers on one policy problem: how to end long-term homelessness among adults with high support needs. The first is a systematic review and meta-analysis of randomised controlled trials (Baxter et al., 2019). The second is an evaluation of nine English Housing First services, treated here as a multi-site case study (Bretherton and Pleace, 2015).

The papers suit comparison because they study the same intervention with opposite strategies: one pools trials, the other describes practice in context. Part 1 sets out the problem and the research questions. Part 2 compares designs and evidence. Part 3 discusses ethics, Part 4 proposes an alternative design, and Part 5 evaluates the recommendations.

Part 1: The policy problem and the research questions

Both papers address the persistence of homelessness among people whom staircase-style services have failed. Housing First offers rapid, permanent housing without requiring treatment or abstinence first. Baxter et al. (2019) frame the problem as a public health one, since homelessness is linked with poor health. Bretherton and Pleace (2015) frame it as a service-delivery problem: whether an international model can work within English commissioning arrangements.

The questions differ in kind. Baxter et al. (2019) ask a causal, effectiveness question about housing stability, mental health, substance use and non-routine healthcare use. Bretherton and Pleace (2015) ask descriptive and process questions: what the services look like, how closely they follow the model, and what outcomes people report. The first suits a decision on whether to fund; the second suits a decision on how to implement.

Weiss (1979) is useful for the role of research here. A systematic review resembles her problem-solving model, supplying an answer to a defined question. A commissioned English evaluation fits the interactive and enlightenment models better, shaping how commissioners understand the model rather than settling the question. Neither paper can be assumed to travel straight into decisions.

Part 2: Comparing research designs and methods

The review’s main merit is inferential strength. Intervention groups spent more days housed and were more likely to be housed at 18 to 24 months (relative risk 2.46, 95% confidence interval 1.58 to 3.84), with fewer emergency department visits (Baxter et al., 2019). That is persuasive for the housing outcome. Its limits are equally important: every included trial was rated at high risk of bias, and short-term health effects were imprecise and inconsistent in direction. A review can only be as sound as the trials it pools, and four trials is a thin base.

The case study answers a different weakness. Randomised trials from other settings say little about whether English services, with local commissioning and housing supply, can replicate the model. Bretherton and Pleace (2015) report that 111 of 143 current service users (78 per cent) were housed at the end of 2014, which was consistent with international findings. Yet with no comparison group, this cannot show that Housing First caused the result. Outcomes forms were completed with support workers, and only about 42 per cent of service users contributed, so selection and social desirability bias are plausible. The authors acknowledge that services were young and the design cross-sectional.

Judged on its own terms, then, neither paper is weak; each is mismatched to the other’s question. The review is strong on whether and weak on how, whereas the case study is strong on how and weak on whether. Using both together is more defensible than relying on either.

Part 3: Ethical implications and reflexivity

The participants in both literatures are among the most vulnerable adults in society, and the power gap between a researcher, a funder and a person who may lose their housing is wide. Bretherton and Pleace (2015) report ethical review by the University of York’s Department of Social Policy and Social Work, told participants they could refuse, anonymised outcomes data and paid a £10 incentive for interviews. These are sound practices, but two issues remain.

First, outcomes forms were administered by support workers, who also control access to housing and support. Consent given to a person with that power may not be fully free, and the report does not explore this tension in reflexive terms. Second, a small incentive for people in poverty raises questions about voluntariness that deserve discussion rather than a passing mention.

A systematic review shifts the ethical weight upstream. It depends on the consent and conduct of primary trials and cannot revisit them, so the reviewers’ ethical responsibility lies in fair synthesis and honest reporting of high risk of bias. Ethical reflexivity is not adequately visible in either paper as summarised here; it appears as procedure rather than as critical reflection on positionality, power and what is lost when randomisation withholds or delays housing.

Part 4: An alternative research design

I propose a mixed-methods realist evaluation (Pawson and Tilley, 1997) of Housing First across several English local authorities. Realist evaluation asks what works, for whom, in what circumstances and why, which addresses the gap between the two papers: the review reports average effects and the case study reports unexplained variation.

The design would combine three elements: routinely collected housing and health service records to compare outcomes against matched comparison areas; repeated interviews with service users and staff over at least two years; and theory-testing of context-mechanism-outcome configurations, such as how housing supply shapes tenancy sustainment. Longitudinal follow-up addresses the one-visit limitation in Bretherton and Pleace (2015), and comparison areas address the lack of a counterfactual.

The costs are considerable. Matched comparison is not randomisation, so causal claims would be weaker than in a trial, and linking records raises data protection and consent questions. Even so, the design suits a policy context in which randomising access to housing is hard to defend and implementation varies by locality.

Part 5: Evaluating the policy recommendations

Baxter et al. (2019) conclude that Housing First approaches successfully improve housing stability and may improve some aspects of health, with long-term health effects needing further study. This follows from the evidence, and the hedging about health is appropriate given the imprecise results and high risk of bias.

Bretherton and Pleace (2015) recommend further, rigorously evaluated experimentation, work with specific groups such as women and young people, longer-term funding and a stronger evidence base to interest health commissioners, while accepting that Housing First is not a panacea. These are proportionate to a cross-sectional study. Their call for more rigorous evidence is, in effect, a candid admission that the design cannot carry stronger claims.

Taken together, the recommendations are credible but incomplete. Neither addresses housing supply or affordability, which determine whether any service can rehouse people. I would add an explicit recommendation that pilots secure access to settled accommodation before launch, and that evaluation funding be fixed in advance.

Conclusion

The two papers show why a policy question rarely has one best method. The systematic review gives the stronger case that Housing First sustains housing, within the limits of four high-risk-of-bias trials. The case study gives a grounded picture of English delivery but cannot isolate effects. Ethical reflexivity is thin in both, and a realist, longitudinal design would better explain how context shapes results.

How to adapt this extract

• Pick two papers on the same problem, one review and one case study, and confirm with your tutor that they qualify.
• Read the full texts, not abstracts; check every figure and quotation against the source before reusing the pattern here.
• Choose quality criteria that match each design, and say why.
• Keep ethics reflexive: discuss power, consent and what the authors did not say.
• Justify your alternative design by the gap it fills, and name its costs.
• Count words against the 2,500 limit, excluding the reference list, and declare the total on the front page.

If you want more support on method choice, see our guides to comparative analysis and social policy coursework, and read what students say about our academic support.

References

Baxter, A.J., Tweed, E.J., Katikireddi, S.V. and Thomson, H. (2019) ‘Effects of Housing First approaches on health and well-being of adults who are homeless or at risk of homelessness: systematic review and meta-analysis of randomised controlled trials’, Journal of Epidemiology and Community Health, 73, pp. 379-387. doi:10.1136/jech-2018-210981.

Bretherton, J. and Pleace, N. (2015) Housing First in England: an evaluation of nine services. York: Centre for Housing Policy, University of York. Available at: https://eprints.whiterose.ac.uk/id/eprint/83966/.

Pawson, R. and Tilley, N. (1997) Realistic evaluation. London: SAGE.

Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E. and Henry, D.A. (2017) ‘AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both’, BMJ, 358, j4008. doi:10.1136/bmj.j4008.

Weiss, C.H. (1979) ‘The many meanings of research utilization’, Public Administration Review, 39(5), pp. 426-431. doi:10.2307/3109916.

Yin, R.K. (2018) Case study research and applications: design and methods. 6th edn. Thousand Oaks, CA: SAGE.