Patient and public involvement (PPI) members, researchers and quality improvement staff share their reflections on the new questionnaire and accompanying guide.
Prefer a quick summary over a long read? Use this link to hear PPI member Araya Gautam break down this blog in under three minutes, or alternatively, listen to the audio version of this blog.
What is the 4Pi questionnaire and guide? How did it come about?
Patient and public involvement in research (PPI) refers to an activity done ‘with’ or ‘by’ service users and carers rather than ‘about’, ‘on’ or ‘for’ them (INVOLVE, 2009).
Evaluation of PPI is important to ensure that such involvement activities are conducted with a good degree of quality and in a meaningful way.
Although the importance of community engagement in research is recognised, there are few evidence-based approaches for measuring the level of community engagement in research projects. The 4Pi Questionnaire has been created in response to this challenge.
It is based on the 4Pi National Involvement Standards of principles, purpose, presence, process and impact. The principles were developed by the National Survivor User Network with input from survivors and service users.
“The questionnaire’s intended purpose is to be used by service providers and other stakeholders that carry out patient and public involvement activities, who would like to assess the experiences of service users and carers involved in those activities,” explains Manuela Russo, a research fellow at King’s Improvement Science (KIS). “It was developed by KIS in collaboration with South London and Maudsley (SLaM) NHS Foundation Trust.”
“We created this guide to provide a robust and standardised method to evaluate patient and public involvement. Patients, service users, carers and the public were involved throughout the design process.” Richard Morton, service user and carer involvement lead at South London and Maudsley (SLaM) NHS Foundation Trust.
"It is so important to listen to patients when doing health research, and my own experiences as a patient are what make me passionate about being involved in it. Collecting data about the things we don’t know enough about can create great change. Hearing about the involvement experience itself is a very important part of that data collection process." Charlie Costa, PPI member on the 4Pi project
Why and how should the questionnaire be used by public involvement members and staff? (Araya Gautam, PPI member on the 4Pi project)
The questionnaire is a helpful tool for thinking back on your involvement experience and pinpointing areas that need improvement. The statements we ask you to consider are specific, so that staff can see exactly what worked and what didn't. We use detailed statements such as: "I had enough practical support (Examples: printing and posting information, technology support, scheduling meetings at a convenient time, support with travel arrangements, training.)," rather than generic statements like "I was involved."
The questionnaire should be used by staff members in charge of managing patient and public engagement as part of an organised strategy to track and enhance involvement procedures. It provides insights into how individuals felt about their role and influence within a project, going beyond basic indicators (like attendance).
Staff can identify a breakdown in continuity and influence if, for instance, a pattern appears in multiple questionnaires. If multiple respondents report that their feedback during the research planning phase was acknowledged, but by the implementation stage, recommendations had not been taken forward, and involvement members were no longer invited to the follow-up meetings, this could lead to an internal review of decision-making processes and how co-production is implemented.
Additionally, when used across multiple projects, the questionnaire data can be aggregated and analysed, to track whether improvements have occurred over time. This helps demonstrate accountability and supports learning. By acting on the feedback and feeding back to involvement members on how their input led to change, staff help close the loop - making involvement feel authentic and impactful.
“We're very excited that the 4Pi Questionnaire is available to use. Information gathered through the 4Pi Questionnaire will be immensely helpful in our continued efforts to meaningfully involve service users in quality improvement. We know from our experience and our evaluation that when service users are involved from the start of quality improvement that the results are better for all.” Dr Barbara Grey, Director Improvement Service (QI, SLaM Partners, Improvement Analytics, Improvement programme management) South London and Maudsley (SLaM) NHS Foundation Trust
The role of PPI in overcoming challenges (Chris Pavlakis, PPI member on the 4Pi project)
Do people enjoy filling out surveys? In both my empirical experience as a lay person partaking in quite a few surveys, and through being involved with design working groups, I find that in general, service users do not particularly enjoy this activity!
The hashtag ‘#NotAnotherSurvey’ remains popular on social media, and the term ‘survey fatigue’ was coined to describe this feeling of discomfort. This can refer to a situation when respondents are no longer interested in completing researchers’ surveys due to the volume of surveys or the effort it takes to complete them.
This type of fatigue is harmful in many ways. It can lead to lower survey response rates, when respondents who are over-surveyed experience fatigue that manifests before they take a survey, causing them to skip it entirely. It can also lead to inaccurate data or abandonment of the survey altogether.
Being aware of the risks associated with surveys as a research tool, our working group reflected hard to include the most important and relevant questions in the survey with the goal of helping respondents manage survey fatigue.
Our thinking went through the following lines:
- Ask direct questions: make sure questions are clearly worded and free from jargon or ambiguous language.
- Ask open ended questions: when respondents are given answers that allow for different ‘shades’ of experience, such as ‘neither agree nor disagree’, rather than a binary ‘yes’ or ‘no’, it can be easier for them to answer the question. We also gave space for respondents to write their own additional comments if they wished, but this additional labour was optional.
- Ask one question at a time: asking multiple questions at once confuses respondents and alters survey data.
- Ask questions that are relevant to the respondent.
"As involvement members, we spoke about what we could do to make the questionnaire more accessible for people who struggle with longer documents. One idea we included in the access section of the guide is that instead of filling out a paper or online form, respondents could complete the questionnaire over the phone, by video call or one-to-one in person. We want as many people as possible to be able to contribute." Charlie Costa, PPI member
Find out more
About the authors
Explore the profiles of this blog’s co-writers, Araya Gautum, Chris Pavlakis and Charlie Costa, and find out more about their research interests.
