The NHS is increasingly turning to technology as the solution to productivity and workforce issues. In mental health services we are increasingly seeing the development and introduction of technologies which hope to improve mental health (see MindTech as an exemplar). However, in inpatient mental health settings the technologies that are being introduced are controversial, such as body worn cameras, and surveillance cameras.
On acute mental health wards, the incidence of self-harm has risen over recent years, and it is widely reported to be the most frequently occurring patient safety incident (Woodnutt et al., 2024).
The current study Kekic et al., (2024) reported an evaluation of whether a ‘vision-based patient monitoring system’ (VBPMS), reduced self-harm in patients’ bedrooms.
The technology, known as Oxevision, uses infra-red sensitive cameras in people’s bedrooms to help staff visually confirm a person is safe, measuring vital signs – such as their pulse and breathing rate – without disturbing their sleep.
– Rethink news story, 30 Nov 2023
On acute mental health wards, the incidence of self-harm has risen over recent years, and it is widely reported to be the most frequently occurring patient safety incident.
Methods
Data were analysed from five quasi-experimental (before/after) studies involving 12 intervention and 7 control wards at five NHS England mental health trusts. One study was treated as research with associated ethical approval, four studies as service evaluations. Control wards were allocated in four of the studies, but not in the fifth study.
The VBPMS intervention is an “infrared-sensitive camera, housed in a secure unit in a patient’s bedroom”. It’s more than a simple surveillance camera though, as the output from the camera feeds into a computer system that measures pulse and breathing rates, as well as “real-time location-based alerts” so staff can see if a patient gets out of bed or spends a long time in the en-suite bathroom. The system can provide summaries of activity data and vital signs to help clinicians with their planning and decision-making. It can also help staff to “review anonymised (blurred) footage of unwitnessed incidents to inform care responses and facilitate accurate documentation of case notes”.
Self-harm rates were calculated as the number of incidents per 1,000 occupied days. Change in pre/post incidence was calculated, including confidence intervals, via a fixed-effect meta-analysis.
Results
Data on 12 wards across 5 Trusts was presented. Five of these were female-only wards, one male, and six mixed (five of which were in trust E). Data was collected between 1st January 2018 and 31st March 2022. Four Trusts (A-D) had 7 control groups assigned, one Trust (E) didn’t have controls.
In Trust A (already reported in Ndebele et al., 2023), 67% of patients consented to take part. In those that lacked capacity, their psychiatrist decided if they should take part. Cameras were kept off for those that did not consent. The fact that two-thirds of patients on acute mental health wards consented is high for this kind of study. For Trusts B-E implied consent was assumed (or determined by clinicians in the case of objections) because the study was classified as a service evaluation by the NHS Trusts. No data was collected on the number of patient/carer objections to the study.
It was reported that there were both increases and decreases in bedroom self-harm incident rates observed between the pre- and post- periods on the intervention wards (range: −53.4% to 58.0%). Results were calculated based on relative percentage change for Trusts A-D, and percentage change for Trust E. Trust E which contained five mixed wards, was excluded from the combined analysis.
There was substantial variation in the incidence of self-harm between Trusts and wards (from 1.32 to 43.9 incidents per 1,000 bed-days). This was also true of the reported relative change in incidence, pre- and post-intervention. None of the per-ward relative risk values were statistically significant.
When the results from Trusts A-D were combined, they showed a statistically significant reduction in incidence of self-harm of 39% (95% CI 7% to 60%). However, when Trust 3 was excluded, due to its unusually low baseline incidence, the combined result was no longer statistically significant.
The control group wards’ evaluation periods used a different time frame, and due to a high variation in rates, an average of the wards was calculated and used in the analysis for one Trust. In another Trust, the same control ward was used for two intervention wards.
Information on the method or severity of self-harm was not reported, neither were the overall incident rates of self-harm on the wards.
This is a flawed piece of research that combines heterogenous data from different wards and NHS Trusts.
Conclusion
The researchers claim that their study shows that the Oxehealth patient monitoring system can be used to improve patient safety in mental health hospitals. However, this research does not reliably demonstrate this.
The researchers highlight the limitations of their own research:
There was large between-patient variability in self-harm rates—particularly for one control ward at Trust C—making it difficult to detect statistically significant reductions in self-harm rates on a ward by ward basis.
The meta-analysis showed a significant reduction in bedroom self-harm incidents but the combined statistic should be interpreted with caution due to the relatively small number of wards included (N = 7) and the between-study nuances in methodologies (of particular importance is that Trust A obtained informed consent for the research whereas Trusts D-E were involved in service evaluations).
The researchers claim that their study shows that the Oxehealth patient monitoring system can be used to improve patient safety in mental health hospitals. However, this research does not reliably demonstrate this.
Limitations
Of the 45 studies I have critiqued for The Mental Elf over the last decade, this has been the most challenging. Undoubtedly, the intervention is controversial, but my concerns are largely due to the study design and impact on transparency that this causes. For example, there was no pre-study registration on ISRCTN. There is a lack of clarity about the control wards, for example, whether they were gender matched, and whether there was a random approach to their selection. That some wards shared the same control ward, and others didn’t have any controls at all, just adds to the confusion.
There is a substantial risk of selection bias in the studies. It is likely, therefore, that there are important differences between the study participants and the real-world population. As noted above, there is clear evidence of heterogeneity between the studies, in terms of baseline incidence and observed intervention effects.
The approach to analysis seems creative, specifically the use of different time periods, control averages calculated across wards due to variation, and the exclusion of fifteen ‘outliers’ without sufficient detail to understand how this affected the results.
We need to test interventions in the real world and encourage the use of intention to treat analysis, not exclude patients because they most frequently self-harm or are exposed to coercive interventions. Although analysis with this data is reported to be included in the appendices.
Differing time periods are important, and they don’t appear to take account of seasonal/annual variation in self-harm and levels of acuity on wards. Given that the focus of this study is solely on bedrooms, we do not know if self-harm increased elsewhere as a consequence of the cameras. Nor is there any information about the monitoring of all incidents to understand other unexpected consequences of the intervention. The paper does state that this will be reported elsewhere. Unusually, the raw data isn’t to be made available by the authors for others to undertake secondary analysis.
At the time of writing this blog, concerns have been raised with the editor of the Journal of Psychiatric and Mental Health Nursing (where this paper was published) about the non-disclosed conflict of interest. Nor is there a clear articulation of the study funder. It can only be assumed that the NHS is paying for the installation of the cameras, and enabling access to routinely collected data for the purposes of commercial exploitation.
There is a clear conflict of interest with this study as the researchers are all employed by the company (Oxehealth Limited) that sells the patient monitoring systems to the NHS.
Discussion
There is no doubt that if clinical services make changes then evaluation of the impact on patients should follow. However, increasingly quality improvement seem to be make substantial changes to clinical practice without due regard for ethical principles. This study adopted an implied consent approach, and no other ethical implications of the study or intervention appear to have been considered. I worry that implied consent doesn’t take account of capacity issues, or give patients choice to opt out of a study. This study approach is in contrast with recommendations for ethical conduct of quality improvement studies (Hunt et al., 2021). Ideally testing the efficacy of new interventions should be treated as research, and rigorously trialled. Given that this study doesn’t appear to show a difference pre-post, is it more or less effective than observations or other interventions used on acute mental health wards to keep patients safe?
The introduction of technology is not without controversy, and patient choice and consideration of the Human Rights Act seem to be a central pillars that we need to uphold. We are already seeing reports that cameras are not keeping patients as safe as they were intended. Concerns have been expressed by Stopoxevision (2024) about the impact of surveillance on mental health and human rights. These are seemingly not accounted for in this study, despite assurances by NHS England in 2023 that the implementation needs to be lawful. However, the alternative of using staff to monitor patients within line of sight eye is not without problems. This role is often delegated to staff with the least training, or those employed through bank/agencies.
Without conducting thorough and transparent studies which adhere to the standards of reporting, we are unable to make claims about the effectiveness of interventions.
It is encouraging to see published research papers evaluating the patient monitoring systems that are being rolled out in many NHS Trusts, but better quality studies are needed before we can be sure that these systems are safe and effective.
Links
Primary paper
Kekic, M., Rose, A., Baker, C., & Bayley, D. (2024). Reduced self-harm on acute mental health wards following the implementation of a vision-based patient monitoring system: Evidence from five NHS trusts. Journal of Psychiatric and Mental Health Nursing, 00, 1–9. https://doi.org/10.1111/jpm.13036
Other references
Hunt DF, Dunn M, Harrison G, et al (2021) Ethical considerations in quality improvement: key questions and a practical guide. BMJ Open Quality 2021. https://doi.org/10.1136/bmjoq-2021-001497
Stopoxevision (2024) Presentation at NHS England Digital Technologies conference, Sheffield, available from https://stopoxevision.com/2024/02/27/actions-are-stronger-than-words-we-raised-our-concerns-now-its-time-for-decision-makers-to-take-action/
Woodnutt S, Hall S, Libberton P, Flynn M, Purvis Snowden J (2024) Analysis of England’s incident and mental health nursing workforce data 2015–2022. Journal of Psychiatric and Mental Health Nursing. https://doi.org/10.1111/jpm.13027
