Is Shorter Better When it Comes to Collecting Patient Feedback?

Patient experience is an important quality indictor, both for provider organizations and patients. The benefits to patients include better disease management, quality of life, treatment adherence, outcomes and preventive care. Provider organizations that provide a better patient experience also benefit through lower medical malpractice risk, higher employee satisfaction and better patient loyalty.

PBGH’s pioneering work to measure medical group performance on patient experience set the precedent for the development of the Agency for Healthcare Research & Quality (AHRQ)’s Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) tool. The CG-CAHPS is the gold-standard survey instrument used to measure patient experience at the provider organization level nation-wide. The survey includes 22 questions (and nine demographic questions) that elicit patient feedback on five domains of care: timely care, provider communication, care coordination, office staff and rating of care.

Provider organizations that use the CG-CAHPS survey to gather patient feedback have expressed concern that the survey is too long, leading to survey fatigue and less-than-desired response rates. Increasing response rates would yield multiple benefits, such as more patient feedback, reduce the time it takes to collect feedback, lower survey costs and reduced administrative burden.

PBGH’s Patient Assessment Survey (PAS) program tested an ultrashort version of the CG-CAHPS. Instead of asking patients to fill out the standard CG-CAHPS instrument (including 30 questions on five domains of care), each patient received a survey by email that included no more than 10 questions on just one to two domains of care. The ultrashort test survey was sent to approximately 10,000 Health Maintenance Organization (HMO) patients across California.

Three main research questions were investigated:

1. Would patients be more likely to respond to a shorter survey, and would it make a difference if we told them it was short in the invitation?
2. Would asking patients for feedback on just one or two aspects of their experience (instead of the 5 topics covered in the standard survey) lead to different patient feedback?
3. Would different types of patients respond to a shorter survey?

Key Findings

Results of the study were surprising. The researchers had anticipated that a survey invitation telling the patient the survey was short would entice patients to open the survey, and that having a small number of questions on the survey itself would lead to drastically more people taking and finishing the survey. However, results of the survey showed:

• Respondents were only slightly (1%) more likely to engage in the survey process if they were told the survey was short in advance
• Once patients started the survey, they were likely to finish it irrespective of the survey length (99% completion rate for the ultrashort survey, vs. 91% for the standard survey)
• Patients gave similar feedback on their care between the ultrashort and the standard survey
• Respondents to the email survey were younger and more educated than respondents to the mailed survey

The findings are instructive for provider organizations and health plans using the CG-CAHPS instrument to assess the quality of care delivery and are looking for methods to encourage greater patient engagement.

The survey findings suggest the following five steps can make a meaningful difference:

1. Focus on engaging patients. Telling patients a survey is short in the invitation might not lead to higher engagement – experiment with a variety of invitation language, length and visual formats to see what resonates most with users and entices them to engage with your surveys; when you find a method that increases your response rate use it as widely as possible.
2. Use email to reach patients. Most CG-CAHPS surveys are sent by mail. Emailing patients the survey can increase response rates by approximately 9%, while lowering costs (the outgoing sample can be reduced by 25%). A mixed-mode approach with email, mail and telephone follow-up will get the best response rates. Email will also help you reach younger and more educated populations.
3. If you plan to collect patient feedback on one or two topics only, keep surveys short. Surveys with 12 questions or less can increase your completion rates by 8%, compared to the regular-length CG-CAHPS survey (28 questions). However, consider how much information you are trying to gather. If you are asking patient for feedback on all five standard domains of care, you will likely need to field the full CG-CAHPS instrument. If you are only interested in the topic of access, then your response rates will likely increase if you only include the survey questions specific to that.
4. Place important questions earlier. If certain questions are essential to your project, consider placing those at the beginning of the survey to reduce the risk of patients getting distracted and not completing the survey.
5. Explore sending surveys by text message. Response rates to surveys sent to patients by email, mail and telephone are going down every year. With 80% of people owning a smartphone, text messaging could be a promising way to reach patients, and at a lower cost. Explore the legal implications of texting patients and consider sending patients a link to an online survey by text messaging; if response rates increase then consider moving more of your surveys to text message.

Read the full report here