The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician & Group Survey 3.1 (CG-CAHPS) is a standardized survey instrument that asks patients to report on their experiences with primary or specialty care received from providers and their staff in ambulatory care settings over the preceding 6 months. CG-CAHPS Survey Version 1.0 was endorsed by NQF in July 2007 (NQF #0005) and version 2.0 received maintenance endorsement in early 2015. Version 3.0 was released in July 2015 and was last endorsed in 2019. The 3.1 version of the survey updates the 3.0 version to prompt respondents to consider in-person, phone, and video visits when they answer the questions and to report which type(s) of visits they had. The survey is part of the CAHPS family of patient experience surveys and is available at https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html
The Adult CG-CAHPS Survey 3.1 is administered to patients aged 18 and over who had at least one visit to a selected provider during the past 6 months. The survey has 32 questions including one overall rating of the provider and 12 questions used to create four (4) composite measures.
The Child CG-CAHPS Survey 3.1 is administered to the parents or guardians of pediatric patients under the age of 18. The survey has 40 questions including one overall rating of the provider and 11 questions used to create four (4) composite measures.
The composite measures are:
- Getting Timely Appointments, Care, and Information (Access)
- How Well Providers Communicate With Patients (Provider Communication)
- Helpful, Courteous, and Respectful Office Staff (Office Staff)
- Providers’ Use of Information to Coordinate Patient Care (Care Coordination)
The survey also has a single-item rating measure:
- Rating of Provider
A guidance document is available on the AHRQ CAHPS website (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) which explains how to administer the survey including how to choose the sample, maintain confidentiality, collect the data, track returned questionnaires, and calculate the response rate.
The Adult CG-CAHPS Survey Rating of the Provider measure assesses the patient’s overall rating of their provider on a scale of 0 to 10, with 0 being the worst and 10 being the best.
Measure Specs
General Information
The CAHPS Clinician & Group (CG-CAHPS) Survey assesses aspects of health care delivery that are important to patients and for which patients are the best or only source of information (Cleary, 2016). Further, the CG-CAHPS survey focuses on patient-centered care (Cleary, Edgman-Levitan, 1997; Cleary, 2016), one of the six central aims identified by the Institute of Medicine for improving the health care system (IOM, 2001). A focus on the patient experience has the potential to enhance clinical outcomes, improve patient safety, and reduce unnecessary medical services. Moreover, assessing patient experience through surveys that include data on the demographic characteristics of respondents, such as race and ethnicity, can help identify the extent to which positive experiences are distributed equitably across patients (Haviland et al., 2003).
Use of this measure will benefit both patients, providers, and medical groups:
- Patients can use information from the measures to help make more informed choices about which practice or medical group to use.
- Medical groups and their providers can use data from the surveys for quality improvement initiatives and incentives.
- Researchers can use data files from the surveys to help answer important health services research questions.
Patient experience encompasses the range of interactions that patients have with the healthcare system. The terms patient satisfaction and patient experience are often used interchangeably, but they are not the same. CAHPS surveys ask patients to report on what they experienced in a healthcare encounter—for example, whether something happened or how often it happened. Patient experience of care surveys provide actionable, objective information for quality improvement. Patient satisfaction surveys, on the other hand, use ratings to measure whether a patient’s expectations about a health encounter were met.
The CG-CAHPS Survey is a standardized survey instrument for measuring patients’ perspectives on their care. The survey is generally administered annually to patients who have received care in the last 6 months.
References
Cleary, PD, Edgman-Levitan, S. (1997). Health care quality. Incorporating consumer perspectives. JAMA. 278(19), 1608-12.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law, 41 (4), 675-696.
Haviland, M. et al, (2003). Do health care ratings differ by race or ethnicity? Joint Commission Journal on Quality and Safety. 29(3), 134-145.
Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Accessible at https://nap.nationalacademies.org/catalog/10027/crossing-the-quality-ch….
The Adult CG-CAHPS Survey Rating of the Provider captures the overall impression of the provider. The provider often serves as the main point of contact for care coordination, preventive services, and chronic condition management. This rating reflects the perceived quality of care provided by the provider. Improving performance on this measure fosters stronger patient-provider relationships, which enhances continuity of care, promotes adherence to treatment plans, and reduces unnecessary hospitalizations and emergency visits. These improvements lead to better health outcomes, lower healthcare costs, and increased patient satisfaction.
The data were obtained from the Centers for Medicare & Medicaid Services (CMS) Center for Medicare and Medicaid Innovation (CMMI) Primary Care First (PCF) Patient Experience of Care Survey (PECS), which is based on the Adult CG-CAHPS Survey and adds Patient-Centered Medical Home Supplemental items (https://pcfpecs.org/Survey-and-Protocols) The survey is administered annually via mail with phone follow-up. The 2023 survey data included 2,490 practice sites and 238,204 respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
Numerator
The response options for this measure range from 0 to 10, where higher scores indicate more positive ratings. AHRQ calculates the score for this item using a top box scoring method. For the top box or “top proportion” score, the numerator is the number of respondents who answered “9” or “10.”
The rating question is as follows:
- Q18: Using any number from 0 to 10, where 0 is the worst provider possible and 10 is the best provider possible, what number would you use to rate this provider?
No additional detail, refer to 1.14.
Denominator
The measure’s denominator is the number of respondents to the survey item. The target population for the survey is patients who have had at least one visit to the selected provider/practice in the target 6-month time frame. This time frame is also known as the look back period. The sampling frame is a person-level list and not a visit-level list.
No additional detail, refer to 1.15.
Exclusions
Individuals are excluded from the denominator if:
- They are deceased.
- They did not receive care from the participating medical group or practice in the last 6 months.
- The individual is under age 18.
The denominator is the total number of surveys fielded minus the total number of ineligible surveys. The total number of ineligible surveys includes sample cases deemed ineligible: does not meet the eligible population criteria (refer to Section 1.15b). No other cases are excluded from the denominator, but cases are excluded from the denominator of the measure if they did not answer any item within the measure.
Measure Calculation
Respondents report on their experiences accessing and using care over the past 6 months.
AHRQ calculates CG-CAHPS Survey this composite measure scores using a top box scoring method.
Composite Measures:
There are two basic steps to calculating a composite measure score for a practice site:
- Calculate the proportion of responses in the top box or most positive response category for each question in a composite measure.
- Calculate the mean or average top box scores across all questions in a composite measure to determine the composite measure’s top box score.
For the top box or “top proportion” score, the numerator is the number of respondents who answered that they “Always” received the desired care or service for a given measure. For example, if 400 out of 1,000 total respondents answered “Always” to a composite measure item, the top box score for that item would be 40 percent [i.e., (400 ÷ 1,000)*100 = 40%].
Lower proportion and middle proportion composite measure scores can also be calculated following the same methodology where the lower proportion is the proportion answering “Never” or “Sometimes” and the middle proportion is the proportion answering “Usually”.
Rating Item:
For the rating item, the numerator for the top box score is the number of respondents who responded 9 or 10 on the 0-10 scale (where 10 is the “Best” and 0 is the “Worst”). For example, if 600 out of 1,000 total respondents answered “9” or “10” to a rating item, the top box score for that item would be 60 percent [i.e., (600 ÷ 1,000)*100 = 60%].
Lower proportion and middle proportion rating scores can also be calculated where the lower proportion is the proportion answering 0-6 on the 0-10 scale and the middle proportion is the proportion answering 7 or 8.
Users may also choose to calculate mean scores or linearized mean scores.
Note the survey includes screener items to identify respondents who meet the target process for each measure, such as whether the individual needed care right away. Measures are only calculated using respondents who experienced a particular service/process.
Users can also case-mix adjust the results for characteristics such as respondent age, education, general health status, and mental health status. The CAHPS Analysis Program—often referred to as the CAHPS Macro—is a free program written in SAS (version 6.0 or later) that enables survey users to case-mix adjust their data. The program also generates a distribution of survey results for each of the measures, calculates the mean score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. The results presented in these analyses are based on unadjusted top box scores unless otherwise noted.
More information about the calculation of proportion scores and mean scores can be found in these documents:
- Instructions for Preparing Data for Analysis: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
- Instructions for Analyzing Data from CAHPS Survey: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
The measure is not stratified.
Users should choose a data collection protocol that maximizes the survey response rate at an acceptable cost. Some sponsors, as well as researchers conducting field tests, have found that the mail with telephone follow-up method is most effective or email with mail or telephone follow-up.
AHRQ provides protocols for collecting responses though users can adapt it to meet their needs. The protocols include mail only, telephone only, mail with phone follow-up, or email (web) with mail or phone follow-up. AHRQ provides detailed instructions for these different protocols in the “Guidelines for Using the CAHPS Clinician & Group Survey” document survey available on the AHRQ CAHPS website: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…;
There is no minimum response rate requirement on the CG-CAHPS Survey. The CAHPS consortium has found that higher response rates are achievable if users take steps to ensure the accuracy of the sample frame and carefully follow the recommended data collection protocol, including one or more attempts to follow up with non-respondents.
In its simplest form, the response rate is the total number of completed questionnaires divided by the total number of individuals selected for the sample. Calculating the response rate is helpful in determining a more accurate starting sample size for future survey administration. For the CG-CAHPS Survey, the goal is a response rate of at least 40 percent or at least 50 completed surveys per provider.
To calculate the response rate, use the following formula: Number of completed returned questionnaires divided by the total number of respondents selected minus the sum of deceased + ineligibles.
AHRQ makes the CG-CAHPS Survey available in English and Spanish.
The sample is drawn from a list of individuals (adults aged 18 and older, or children 17 and younger) who have received care from a given provider, practice site, or medical group during a six-month time period. The list is called a sample frame.
The source of sample information will vary by survey sponsor. The data to identify individual patients may be found in the records of medical practices or health systems.
Defining the Sample Frame: Eligibility Guidelines:
- The adult questionnaires include all adults 18 years or older.
- The child questionnaire includes all children 17 years or younger.
- Include only patients who have had at least one visit to the selected practice in the last 6 months. This time frame is also known as the look back period.
- To identify the sampling frame, use the anticipated start date of data collection to determine the reference period. For example, if your anticipated start date is September 1, 2026, include all those who have had at least one visit since March 1, 2026.
- The sampling frame is a person-level list and not a visit-level list. Therefore, patients should appear only once in the sampling frame regardless of how many visits they have had in the look back period.
- Draw the sample irrespective of reason for visit and duration of patient-provider relationship, so that the full range of patients is represented.
- Include all patients who meet the sampling criteria even if they are no longer receiving care from the practice, site/clinic or provider.
- Allow the sample frame to include multiple individuals from the same household, but the sample you draw should not have more than one person (adult or child) per household. In other words, the sample that is selected for data collection should be de-duplicated to ensure that only one person per household receives a survey. The final sample must contain only one respondent per household. Where a duplicate household is sampled, it is discarded and replaced by another random draw from the frame.
- All CAHPS survey items have been designed for the general population. Appropriate screening items are included for items targeted to assess a specific experience. In order to ensure that results are comparable to those produced by other sponsors and vendors, targeted sampling, such as selecting only patients with particular conditions or experiences, is not recommended. Targeted sampling should only be used to supplement the general population sample, if desired.
- In order to administer the survey, the name of the provider must be available, even if surveying at the site/clinic or practice level. If the sampling frame does not accurately identify the provider that the patient saw, select a larger sample to account for errors in connecting health care received to a specific provider. For example, errors can occur if administrative billing data are used for the sampling frame and visits with physician assistants or nurse practitioners are billed under the supervisory physician.
Calculating the Sample Size for the Adult (Child) Questionnaire
The sample size varies depending on whether sampling is done at the level of the individual provider, practice, or medical group. In general, to produce statistically valid comparisons, the sample needs to be large enough to yield 50 completed questionnaires per provider or 300 completed questionnaires per medical group. The recommended sample size when sampling at the practice level depends on the number of providers at each site. The document Guidelines for Using the CAHPS Clinician & Group Survey has a table for recommended sample sizes based on the number of providers, and it ranges from 50 to 300 completed questionnaires. (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
Data are not reported for any item or measure with fewer than 10 valid responses and practice sites with fewer than 10 responses were not included. AHRQ recommends that there needs to be approximately 50 completed questionnaires per provider to have a sufficient number of responses for results to be statistically reliable.
Proxy Respondents
The CG- CAHPS Survey Plan does allow for proxy respondents for mail and web-based mode. At the end of the survey, there is an item that asks “Did someone help you complete this survey?” If the answer is Yes, the follow-up question is “How did that person help you?” and they are to mark one or more of these response items:
- Read the questions to me
- Wrote down the answers I gave
- Answered the questions for me
- Translated the questions into my language
- Helped in some other way
However, these the last two questions of the core questionnaire are not included in telephone scripts because telephone interviews should not be conducted with proxy respondents.
Supplemental Attachment
Point of Contact
CAHPS® is a registered trademark of the U.S. Department of Health and Human Services and managed by AHRQ
Karen Chaves
Rockville, MD
United States
Naomi Yount
Westat
Rockville, MD
United States
Importance
Evidence
The CG-CAHPS Survey measures key components of patient experience, such as provider communication and ease of access, that are consistent with patient-centered care. The CAHPS Surveys focus on aspects of care that consumers have identified as important and for which patients are the best or only source of information. Measuring patients’ perceptions of their healthcare experience is not just a means to improve services—it’s a recognition that the patient’s voice matters in and of itself. Listening to patients affirms their role as active participants in their care, and their insights are essential to truly understanding the quality and impact of healthcare delivery. In 2023, over 238,000 patients completed the CG-CAHPS Survey as part of the PCF PECS survey. It is possible that more practices are administering and using the CG-CAHPS survey beyond the practices participating in PCF. We reviewed the literature on the determinants of patient care experiences measured by CAHPS and their associations with other indicators of health care quality. CAHPS is also an actionable measure that helps clinicians and health plans target interventions that will improve the quality and patient-centeredness of care.
Review of the Evidence
Prior research has identified several features of healthcare delivery structure, including clinic accessibility, patient flow, and management, that are associated with patient experiences. Two major systematic reviews have examined the relationships among patient experience, clinical processes, and patient outcomes. A systematic review performed by researchers in the U.K. found that patient experience is favorably associated with adherence to recommended medications and treatments, preventive care such as screenings and immunizations, patient-reported health outcomes, clinical outcomes, reduced hospitalizations and primary care visits, and reduced adverse events (Doyle et al., 2013). Anhang Price et al. (2014) reviewed evidence on the association between patient experiences and other measures of health care quality in the U.S. They similarly found that better patient care experiences are associated with higher levels of adherence to recommended prevention and treatment processes, better clinical outcomes, and less health care utilization.
Health Care Process and Quality Improvement Influence on Patient Experience
Providers routinely use patient-reported measures such as CAHPS to guide quality improvement (QI) activities to improve their patients’ experience with care (Friedberg et al., 2011; Davies, Shaller, et al., 2013; Quigley et al., 2015).
For example, to improve scores for “how often the office staff were as helpful as you thought they should be”, Dean Clinic, a large integrated health care delivery system in Wisconsin, further surveyed their patients to ask them what “helpfulness” means to them and to ask how office staff could be more helpful. From this feedback, the Clinic learned about ways that the office staff can be more welcoming, friendly, and appreciative of patients. With input from both staff and management, Dean Clinic developed action plans to improve patient experience. The service department shadowed staff and provided feedback. To improve consistency in service across all sites, the Clinic developed an orientation for all new employees on customer service expectations. They also offered ongoing training in the form of service workshops, videos, and Webinars, as well as targeted interventions for the lowest scoring offices (AHRQ, 2013). Several quality improvement initiatives to improve provider communication and engagement include shadowing, coaching, and training (AHRQ, 2013; Hardee & Kasper, 2008; Quigley, Palimaru, et al., 2017). Lastly, Friedberg et al. (2011) found that physician groups that aim to improve access, communication with patients, and customer service do so by addressing office workflow, providing additional training for nonclinical staff, and adopting or enhancing an electronic health record.
Using data from the CAHPS survey and a newly installed electronic health record system, in 2015, leaders of Northeast Valley Health Corporation (Los Angeles County) pinpointed interventions including reallocating staff resources and not scheduling well-child visits first thing in the morning. System leaders found that when the first appointments of the day ran long, there was a cascading effect on the rest of the day's schedule. By 2015, NEVHC instituted changes systemwide. By 2017, total average cycle time was reduced from 82 minutes to 65 minutes; the proportion of patients with a cycle time under 60 minutes rose from 34 percent to 48 percent; and the proportion of patients seen within 15 minutes of appointment time rose from 38 percent to 47 percent. Case study found at https://www.ahrq.gov/news/newsroom/case-studies/201718.html
More examples of interventions to improve patient experience with primary and specialty care as measured by the CG-CAHPS Survey can be found in the CAHPS Improvement Guide available at https://www.ahrq.gov/cahps/quality-improvement/index.html. The Guide also includes information on analyzing survey results and identifying root causes of performance problems.
The Centers for Medicare and Medicaid Services Merit-based Incentive Payment System (MIPS) Quality Payment Program measures Medicare Part B providers in four performance categories to derive a score that could affect a provider's Medicare reimbursement positively or negatively starting at 4% in 2019 (based on 2017 performance). MIPS has a performance category called “Improvement Activities” that includes an inventory of activities that assess how a physician group can improve care processes, enhance patient engagement in care, and increase access to care. Several of these activities are geared to improve patient experience of care as measured by CG-CAHPS. A list of MIPS activities can be found at https://qpp.cms.gov/mips/improvement-activities.
Structure
Physician clinic hours of operation and availability for appointments have been found to predict patient experience in several studies. In a study with survey data from 61,839 patients of 1729 primary care physicians in California, system-level factors, such as belonging to a larger medical group and the physician’s zip code-based Primary Care Services Areas, explained between 28% to 48% of variation in patient care experience, with the highest proportion explained for the access to care composite (Rodriguez et al., 2009). Improving the infrastructure supporting certain aspects of care may have broad effects because system changes can influence multiple outcomes (Cleary, 2016).
“Expanded practice access” is a highly weighted CMS MIPS improvement activity. To improve the patient experience in access to care, practices may consider providing 24/7 access to clinicians, groups, or care teams for advice about urgent and emergent care (e.g., eligible clinician and care team access to medical record, cross-coverage with access to medical record, or protocol-driven nurse line with access to medical record). Access expansion can include one of the following: 1) expanded hours in evenings and weekends with access to the patient medical record (e.g., coordinate with small practices to provide alternate hour office visits and urgent care); 2) use of alternatives to increase access to care team by clinicians and groups, such as e-visits, phone visits, group visits, home visits and alternate locations (e.g., senior centers and assisted living centers); and/or 3) provision of same-day or next-day access to a consistent clinician, group or care team when needed for urgent care or transition management. Another example is implementation of “open access” scheduling, in which some physician time is always reserved for same-day appointments, to improve patient access to care (Murray & Tantau, 1998).
Health-related Patient Behavior and Disease Management
One dimension of the CG-CAHPS measure captures the patients’ perceptions of how well providers communicate with them. Better patient-provider communication promotes healthcare-related patient behaviors (Fuertes, Boylan, et al., 2009). A 2009 meta-analysis of 127 studies assessing the link between patient treatment adherence and physician-patient communication found a 19% higher risk of non-adherence among patients whose physician communicated poorly (Zolnierek and Dimatteo, 2009). Doyle’s (2013) meta-analysis showed positive associations between the quality of clinician-patient communications and adherence to medical treatment in 125 of 127 studies analyzed. Studies using the CAHPS measure have found that better provider communication is positively associated with adherence to hypoglycemic medications among diabetics (Ratanawongsa, Karter, et al., 2013), adherence to tamoxifen among breast cancer patients (Liu, Malin, et al., 2013), and higher rates of colorectal cancer screening among adults in the US (Carcaise-Edinboro and Bradley, 2008). Sequist and colleagues (2008) found that measures of patient experience, including doctor-patient communication, clinical team interactions, and health promotion support, were positively associated with some prevention and disease management clinical process measures in clinical practices and among individual clinicians.
Outcomes
Out of 40 evidence papers with outcome measures, Doyle’s (2013) meta- analysis found 29 studies with positive associations between patient experience and clinical outcomes, 11 with no associations, and none with negative associations. The lack of more evidence may be due to complexity between a patient’s illness level, their level of care, and their likelihood for a poor outcome such as mortality, morbidity or a readmission. Often, such associations have more than one plausible direction of causality. For example, clinicians may be especially attentive to the needs of sicker patients (Kahn et al., 2007) and patients near the end of life (Elliott, Haviland, et al., 2013; Xu et al., 2014).
Moreover, substantial evidence points to a positive association between various components of patient experience, such as good communication between clinicians and patients, and several important processes and outcomes. These include lower utilization of unnecessary healthcare services; better patient adherence to medical advice; better process of care measures for acute myocardial infarction (AMI), congestive heart failure, pneumonia and surgery; lower inpatient mortality among acute myocardial infarction (AMI) patients; lower infection rates (Anhang Price, et al., 2014); and better clinician and staff perceptions of patient safety culture (Sorra et al., 2012).
Utilization
Research suggests an association between better patient experiences and lower healthcare utilization. Children with asthma were less likely to visit the emergency department, make urgent office visits, or be hospitalized if their physicians had reviewed a long-term therapeutic plan with their parents (Clark, Cabana, et al., 2008). Among African Americans with Type 2 diabetes, those who reported that doctors or nurses usually listened carefully or spent enough time with them were significantly less likely to visit the emergency department in the 12 months following completion of a patient experience survey (Gary, Maiese, et al., 2005). Children whose parents report longer waits for primary care visits were more likely to visit the emergency department for non-urgent reasons than those who report waiting for less time (Brousseau, Bergholte, et al., 2004).
References
Agency for Healthcare Research and Quality (AHRQ). How Two Provider Groups Are Using the CAHPS® Clinician & Group Survey for Quality Improvement. Brief available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/quality-improvem….
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014). Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 71(5), 522-54.
Brousseau, DC., Bergholte, J., et al. (2004). The effect of prior interactions with a primary care provider on nonurgent pediatric emergency department use. Archives of Pediatrics & Adolescent Medicine. 158(1), 78-82.
Carcaise-Edinboro, P. and Bradley, CJ. (2008). Influence of patient-provider communication on colorectal cancer screening. Medical Care. 46(7), 738-745.
Clark, NM., Cabana, MD., et al. (2008). The clinician-patient partnership paradigm: Outcomes associated with physician communication behavior. Clinical Pediatrics. 47(1), 49-57.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law. 41(4), 675-696.
Davies, E, Shaller, D, Edgman-Levitan, S, Safran, DG, Oftedahl, G, Sakowski, J, Cleary, PD. (2008). Evaluating the use of a modified CAHPS survey to support improvements in patient-centered care: lessons from a quality improvement collaborative. Health Expect. 11(2), 160-76.
Doyle, C., Lennox, L., et al. (2013). A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 3(1). http://bmjopen.bmj.com/content/3/1/e001570.full
Elliott, MN., Haviland, AM., et al. (2013). Care experiences of managed care Medicare enrollees near the end of life. Journal of the American Geriatrics Society 61(3), 407-412.
Friedberg, MV, SteelFisher, GK, Karp, M, and Schneider, EC. (2011). Physician groups’ use of data from patient experience surveys. J Gen Intern Med. 26(5), 498-504.
Fuertes, J N., Boylan, LS., et al. (2009). Behavioral indices in medical care outcome: The working alliance, adherence, and related factors. Journal of General Internal Medicine. 24(1), 80-85.
Gary, TL., Maiese, EM., et al. (2005). Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Disease Management. 8(6), 361-371.
Hardee, JT, Kasper, IK. (2008). A clinical communication strategy to enhance effectiveness and CAHPS scores: The ALERT model. Perm J. Summer. 12(3), 70-4.
Kahn, KL., Tisnado, DM., et al. (2007). Does ambulatory process of care predict health-related quality of life outcomes? Health Services Research. 42, 63-83.
Liu, Y., Malin, JL., et al. (2013). Adherence to adjuvant hormone therapy in low-income women with breast cancer: The role of provider-patient communication. Breast Cancer Research and Treatment. 137(3), 829-836.
Murray, M and Tantau, C. (1998). Must patients wait? Jt Comm J Qual Improv. 24(8), 423-5.
Quigley, DD, Mendel, PJ, Predmore, ZS, Chen, AY, Hays, RD. (2015). Use of CAHPS™ patient experience survey data as part of a patient-centered medical home quality improvement initiative. J Healthc Leadersh. 7, 41-54.
Quigley, DD., Palimuru, AI., Chen AY., & Hays, RD. (2017). Implementation of practice transformation: Patient experience according to practice leaders. Quality Management in Health Care. 26 (3), 140-151.
Ratanawongsa, N., Karter, AJ., et al. (2013). Communication and medication refill adherence: the Diabetes Study of Northern California. JAMA Internal Medicine. 173(3), 210-218.
Rodriguez, HP, Scoggins, JF, von Glahn, T, Zaslavsky, AM, Safran, DG. (2009). Attributing sources of variation in patients' experiences of ambulatory care. Med Care. 47(8), 835-41.
Sequist, TD., Schneider, EC., et al. (2008). Quality monitoring of physicians: linking patients' experiences of care to clinical quality and outcomes. Journal of General Internal Medicine. 23(11), 1784-1790.
Sorra, J, Khanna, K, Dyer, N, Mardon, R, Famolaro, T. (2012). Exploring relationships between patient safety culture and patients’ assessments of hospital care. Journal of Patient Safety. 8(3), 131–139.
Xu, X., Buta, E., Anhang Price, R., Elliott. NN., Hays, RD., & Cleary, PD. (2014). Methodological considerations when studying the association between patient-reported care experiences and mortality. Health Services Research. 50(4), 1146-61.
Zolnierek, KB. and Dimatteo, MR. (2009). Physician communication and patient adherence to treatment: a meta-analysis. Medical Care. 47(8), 826-834.
It is important that the Adult CG-CAHPS Survey reflects the aspects of care that patients associate with high-quality healthcare. Through a literature review, focus groups, a Technical Expert Panel, and numerous other development activities, the CAHPS Consortium found that an overall rating of the provider is an important component of ambulatory care. This measure asks the respondent to rate the provider on a scale from worst (0) to best (10). The overall rating of the provider captures a comprehensive view of the patient’s experience. The simple summary rating score can help make comparisons easier for patients. The overall rating measure is associated with the other composite measures (Hays et al., 2018).
References
Hays, RD, Mallett JS, Haas A, et al. (2018). Associations of CAHPS composites with global ratings of the doctor vary by medicare beneficiaries’ health status. Med Care. 56, 736–739.
Measure Impact
The CAHPS measures, developed to complement more technical quality measures, are measures for which the patients are the best or only source of information and/or perspective, such as the degree to which care felt patient-centered (Anhang Price et al., 2014). Several studies provide evidence that patients value the CAHPS measures and find them meaningful. For example, Safran et al. (2001) found that patients who reported the poorest-quality relationships with their physicians were three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships.
Collins et al. (2017) found that a patient’s “most important CAHPS domain” varied across subgroups; racial and ethnic patient subgroups differentially valued various aspects of the care experience. To efficiently reduce disparities and improve quality, Collins et al recommend tailoring quality improvement programs to the factors most important to the racial, ethnic, and language mix of the patient population of interest. Quigley et al. (2014) found that the importance of provider communication varied significantly by practice specialty type, yet respectful treatment was consistently import across all specialties.
Patients also use information from patient experience measures to make decisions about their healthcare providers and plans. One study found that seeing publicly reported quality information was a determinant of choosing higher quality-rated health plans, although the weight given to quality information also depended on other features, such as cost and provider choice (Faber et al., 2009).
References
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014) Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 71(5), 522-54.
Collins, RL, Haas, A, Haviland, AM, Elliott, MN. (2017). What matters most to whom: Racial, ethnic, and language differences in the health care experiences most important to patients. Med Care. Nov;55(11), 940-947.
Faber, M, Bosch, M., Wollersheim, H, Leatherman, S, and Grol, R. (2009). Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 47(1), 1-8.
Quigley, DD, Elliott, MN., Burkhart, Q, Farley, DO., Skootsky, SA., & Hays, RD. (2014). Specialties differ in which aspects of doctor communication predict overall physician ratings. Journal of General Internal Medicine, 29, 447-454.
Safran, DG, Montgomery, JE, Chang, H, Murphy, J, Rogers, WH. (2001). Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2), 130–6.
The Adult CG-CAHPS questions focus on aspects of care for which the patient is the best or only source of information and that reflect elements of care that are most meaningful to patients. Published research indicates that individuals use information from patient experience measures to make decisions about their healthcare providers and plans. Patient experiences with care are also linked to their persistence with the provider. At the provider level, patients who reported the poorest-quality relationships with their physicians are three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships (Safran et al., 2001).
The Rating of Provider measure is meaningful to patients because it reflects their overall experience with the provider who often serves as their primary point of contact in the healthcare system. For many, the provider is someone they rely on for ongoing care, guidance, and support in managing their health. This measure captures the overall experience with the provider and helps support making decision about with whom to seek care. It also offers patients a meaningful way to express concerns or positive experiences with their provider.
Reference
Safran, DG, Montgomery, JE, Chang H, Murphy, J, Rogers, WH. (2001). Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2),130–6.
Performance Gap
The analyses were based on data from CMMI’s Primary Care First (PCF) Patient Experience of Care (PEC) survey, which is a version of the AHRQ Adult CG-CAHPS 3.1 Survey. The 2023 survey was administered from October 3, 2023, through December 19, 2023, and includes 2,486 practice sites and 238,204 respondents.
To examine the performance gap over time, we also analyzed the 2022 survey data. The 2022 survey was administered from October 6, 2022, through December 19, 2022, and includes 2,801 practice sites and 241,521 respondents.
Deciles in the Performance Scores by Decile tables are based on the performance scores; and for the “mean performance score” row in the tables show the average unadjusted top box scores across practice sites. For cases where the performance score was tied across decile boundaries, all practice sites with that score were assigned to the same decile.
As shown in Table 1 and the attached Table 2.4a.5 for the Rating of Provider measure:
- the 2023 mean top box score = 80.3; number of measured entities = 2,486; and number of respondents was 232,292.
- the 2022 mean top box score = 79.4; number of measured entities = 2,801; and number of respondents was 234,582.
The mean top box score increased slightly (79.4% to 80.3%). The maximum score decreased slightly from 100.0% in 2022 to 98.8% in 2023. The scores are lower than ideal (100% is ideal), given that about 80% of respondents rated their provider as a 9 or 10 on a scale of 0 to 10. These results show that while there are small increases from 2022 to 2023, indicating that the performance gap may be decreasing over time, practices need to continue to aim to improve their provider ratings.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 80.3 | 40.0 | 64.1 | 72.1 | 75.5 | 78.2 | 80.4 | 82.2 | 84.0 | 86.0 | 88.3 | 92.2 | 98.8 |
| N of Entities | 2,486 | 1 | 248 | 250 | 248 | 248 | 248 | 249 | 250 | 247 | 249 | 247 | 1 |
| N of Persons / Encounters / Episodes | 232,292 | 30 | 18,243 | 22,721 | 22,989 | 23,468 | 23,892 | 23,010 | 24,623 | 24,451 | 24,790 | 23,992 | 83 |
Care Gaps
Closing Care Gaps
Optional for Fall 2025.
Feasibility
Feasibility
The CG-CAHPS Survey is a standardized instrument designed to assess patient experience of care. As these patient-experience data are collected from patients, the structured data are not available in electronic sources outside of the data collection by the practice site or medical group.
The data are collected through a survey instrument that is administered directly to patients, not during care delivery. Surveys are generally mailed to the sampled patients, and those survey results can be entered into structured databases (e.g., Excel, SPSS, SAS). No proprietary platform is required to administer the survey. Though mixed-mode administration (i.e., mail and phone) is a viable strategy for the collection of CAHPS surveys, mail continues to be the most frequent mode for most CAHPS surveys. Users then create electronic databases of results after receipt of the completed hard copy survey through scanning or data entry. However, vendors may set up their database before data collection by populating the frame to assist in identifying nonresponse.
Traditionally, the rationale for not using electronic sources more broadly is that mail and telephone are the best ways to obtain representative samples of patients based on the contact information that is available for sampling and data collection. E-mail has been added as a mixed mode strategy for physician groups with reliable email addresses for their patient population.
Structured or unstructured fields. All items are structured and on a 4 -item Likert-type response option scale (1-4) or for the rating items on a 0-10 scale. All responses are numeric.
Electronic feasibility. CG-CAHPS Survey users can offer a web survey to respondents to complete the survey though that option should not be the only option as it may exclude patients who have limited or no access to the web and/or who do not have an email address to send an electronic version of the survey.
Missing data. Item level missing data is low on the CG-CAHPS Survey however, some items will have fewer response than others due to gate or filter questions. For example, if a respondent did not need urgent care in the last 6 months, they are skipped through items about their experiences with getting urgent care. As a result, some CG-CAHPS Survey items have higher percentages of missing data overall, but when skip patterns are considered, the percentages of inappropriate missing data are much lower (<5%).
Measure susceptibility to inaccuracies and ability to audit data: The CG-CAHPS Survey is self-reported perceptions or experiences with the care received and therefore cannot be assessed to determine if the results are accurate. The protocol for administering CG-CAHPS Survey is outlined by AHRQ and vendors normally collect the data in a standardized format. The data that has been collected can be audited to ensure no data entry errors have occurred, there are no out of range values, and skip patterns are followed.
Change to the Instrument. Since the instrument was last endorsed, there was only one change to the survey and that was to add text to the instrument to allow for respondents to include care that was done virtually (i.e., phone or by video) to account for changes in care delivery due to the COVID-19 pandemic. For example, instructions now included “by phone, or by video”: “The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.” This change did not impact data structure or availability.
Data Collection Burden
For respondents: The survey takes approximately 15 minutes to complete, dependent on the individual.
For medical groups and practice sites: Survey sampling uses administrative enrollment data that is maintained by all medical groups/practice sites and easily accessible to produce a sampling frame. Practice sites/medical groups generally hire a survey vendor to administer, track, and analyze their survey data resulting in lower burden for the practice sites/medical groups.
Cost Considerations
The CG-CAHPS Survey is freely available for use with no proprietary fees.
The cost to hire a vendor varies based on the size of the medical group and desired number of completed surveys. AHRQ provides guidance for hiring a vendor and resources for finding a certified vendor (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/hiring/in…).
Impact on Clinician Workflow
The CG-CAHPS Survey does not interfere with diagnostic thought processes or patient -physician interactions as the survey is retrospective after care has been given, not during the visit.
Potential Barriers and Mitigation Strategies
Achieving a desired response rate may be difficult for users. Phone is not optimal as the only mode of survey administration, but it is commonly used as a follow-up for CAHPS mail surveys. Phone follow-up can improve CAHPS response rates compared to mail-only (Burkhart et al., 2014; Fowler et al., 2002; Gallagher et al., 2005; Klein et al., 2011). A study of Medicare beneficiaries found that response rates continue to improve when up to 4 follow-up calls are made (Burkhart et al., 2014). In addition, phone follow-up calls help to achieve better representation of patients in terms of income, literacy/education, health status, age, gender, and race/ethnicity, above and beyond mail surveys alone (Tesler and Sorra, 2017). The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that are hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses. AHRQ also provided a webinar on how to achieve higher response rates (https://www.ahrq.gov/cahps/news-and-events/events/webinar-011124.html).
Analysis and Reporting: AHRQ makes available many resources to assist with analysis and reporting. For instance, there is a free CAHPS Analysis Program which is written for SAS that enables survey users to conduct the analyses needed to produce valid comparisons of performance across similar health care organizations. Users can also review documentation on how to prepare data for analysis (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). Further, vendors usually conduct all analyses and reports.
References
Burkhart, Q, Haviland, A, Kallaur, P, et al. (2014). How much do additional mailings and telephone calls contribute to response rates in a survey of Medicare beneficiaries. Field Methods. 27(4):409-25.
Fowler, FJ, Gallagher, PM, Stringfellow, VL, et al. (2002). Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Med Care 40(3):190-200.
Gallagher, PM, Fowler, FJ, Stringfellow, VL. (2005). The nature of nonresponse in a Medicaid survey: causes and consequences. J Off Stat 21(1):73-87.
Klein, DJ, Elliott, MN, Haviland, AM, et al. (2011). Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 51(6):843-55.
Tesler, R. and Sorra, J. CAHPS Survey Administration: What We Know and Potential Research Questions. (Prepared by Westat, Rockville, MD, under Contract No. HHSA 290201300003C). Rockville, MD: Agency for Healthcare Research and Quality: October 2017. AHRQ Publication No. 18-0002-EF. Accessible at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/about-cahps/rese….
Most vendors have established methods for tracking the sample. The Consortium suggests setting up a system to track the returned surveys by the unique ID number that is assigned to each respondent in the sample. This ID number should be placed on every questionnaire that is mailed and/or on the call record of each telephone case.
To maintain respondent confidentiality, the tracking system should not contain any of the survey responses. The survey responses should be entered in a separate data file linked to the sample file by the unique ID number. (This system will generate the weekly progress reports that should be review closely.) Data should be stored securely—preferably on encrypted or password-protected systems—with access limited. If paper responses are used, they should be shredded following de-identified data entry.
The CG-CAHPS Survey data is therefore de-identified upon data collection with a focus on protecting the confidentiality of respondents. Vendors are trained on maintaining confidentiality.
The CG-CAHPS Survey has a long history of use dating back to 2007. The CG-CAHPS Survey has gone through two main revisions since that time, using field and psychometric testing conducted by multiple partners and other stakeholders to increase the scientific rigor and relevance of the survey and the usability of the data (for more development information refer to https://www.ahrq.gov/cahps/surveys-guidance/cg/about/Develop-CG-Surveys…)
Steps which have contributed to the content and design of the CG-CAHPS Survey over time have included:
- Literature review and review of existing measures
- Development and consultation with technical expert panels
- Focus groups with consumers
- Cognitive testing of survey questions to ensure they will be understood by respondents
- Field testing to assess the reliability of the survey results
- Cognitive testing of measure labels to ensure that survey results are communicated clearly to providers and the public
- Public comment
- On-going collaboration and harmonization with key partners and stakeholders
The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that have been hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses.
To address data collection efficiency and to improve response rates, the CAHPS Consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS Consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are in the survey sample. The follow-up to the entire sample is necessary to get a representative set of responses from a practice’s population, as not all patients may have e-mail.
Proprietary Information
Scientific Acceptability
Testing Data
The data used for these analyses are from the CMMI Primary Care First (PCF) Patient Experience of Care (PEC) Survey) which includes a combination of items from the Adult CG-CAHPS Survey as well as from the Patient-Centered Medical Home CAHPS Supplement. The survey is available on the CMMI PCF website: https://pcfpecs.org/Survey-and-Protocols.
For evidence of performance gap demonstrating persistent gaps over time, we also include top box statistics on the 2022 Adult CG-CAHPS Survey data administered as part of the PCF PEC survey.
The data used to support extrapolation of Adult data to Child data, is based the 2019 maintenance endorsement submission, when both Adult and Child data were available. Using data from surveys administered from January 2016- March 2017, the Child survey included 77 practice sites and over 12,000 responses and the Adult survey included 635 practice sites and over 110,000 respondents. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Data were included in the analysis if they had at least one reportable item from the CG-CAHPS survey.
Unless noted otherwise, the top box scores presented are unadjusted since the results are not being used to compare entities, but rather for descriptive and scientific acceptability purposes.
The 2023 survey data was collected between October 3, 2023, through December 19, 2023.
The 2022 survey data was collected between October 6, 2022, through December 19, 2022.
None.
The CG-CAHPS Survey is specified for both individual clinician and group/practice. However, we only have testing for the group or “practice site” because the clinician-level identification in the CG-CAHPS database is not available. The site-level measured entity is referred to as a “practice site.” The practice site is an outpatient facility in a specific location. Practice site level survey results are calculated across the respondents within a specific site.
The data included 2,490 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
The 2022 data used exclusively in the performance gap section to demonstrate persistent gaps over time included 2,806 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,801.
Practices in the 2023 data come from 23 states and the District of Columbia, as shown in 5.1.3a available in the Supplemental 7.1 zip file.
Due to the lack of access to Child CG-CAHPS data, we present the data for Adult CG-CAHPS Survey measures which can be extrapolated to provide evidence for the Child CG-CAHPS Survey measures. The Adult CG-CAHPS Survey and Child CG-CAHPS Survey are equivalent with only minor wording changes (refer to the crosswalk between Adult and Child measures included with the instruments in Attachment 1.21). The wording changes directed attention to the experiences for the child rather than the respondent.
For example, on the Adult Survey one question is worded:
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed?
On the Child Survey, the question is worded (bolded added to differentiate):
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care your child needed right away, how often did you get an appointment as soon as your child needed?”
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
The data includes 238,204 adult respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction (N=4 sites), the total number of respondents was 238,181, the average number of respondents per site was 96, ranging from 14 to 272 respondents per site.
Tables 5.1.4a through g, available in the Supplemental 7.1 zip file, show descriptive characteristics of the respondents (sex, race/ethnicity, age, self-reported health status, survey mode, and survey language). Practice sites had adult respondents that were predominantly white and non-Hispanic (79%) and older than 54 (83%). Over 40% of respondents had at least a 4-year college degree (41%). Most responded via mail (89%) compared to by phone (11%). Most respondents completed the survey in English (99%), and only 1% completed it in Spanish.
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
In the prior maintenance endorsement in 2019, using data from surveys administered from January 2016- March 2017, we had access to data from 77 practice sites and over 12,000 responses for the Child survey and 635 practice sites and over 110,000 respondents for the Adult survey. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Respondent education distributions were similar across child and adult CAHPS surveys. For example, in the 2016-2017 CG-CAHPS data, 65% of respondents to the Adult Survey had at least some college and 76% of respondents to the child survey had at least some college. Additionally, 21% of respondents to the Adult survey and 25% of respondents to the Child survey had more than a 4-year college degree. The Adult survey respondents had a slightly higher percentage of respondents with lower levels of education (27% high school graduate or GED or less versus 16% for the Child survey). Overall, these results support the appropriateness of extrapolating adult data in the absence of child data.
Respondents to both the Adult and Child 2016-2017 CG-CAHPS Surveys tended to be female (over 50% for both), with more females responding to the Child Survey (82%) than the Adult Survey (53%).
Patient general and mental health status distributions differ between the child and adult CG-CAHPS surveys when examining the 2016-2017 CG-CAHPS child survey data. For the child survey, 78% of respondents reported the general health status of the child to be very good or excellent while only 43% of respondents to the adult survey reported their general health status as very good or excellent. Similarly, 77% of respondents reported the mental health status of the child to be very good or excellent compared with 62% of respondents to the adult CG-CAHPS survey reported their mental health status as very good or excellent.
More respondents of Child CG-CAHPS Survey were aged 25-44 than respondents of the Adult surveys (61% versus 14%).
Keeping these differences in mind, we extrapolate adult data in the absence of child data. Further, as shown in the other sections, the scientific acceptability results (reliability and validity) between Adult and Child tended to align in the past maintenance endorsement package, further justifying extrapolation. Additionally, both surveys have the same usability and goal of assessing and improving the quality of care by gathering patient or parent/legal guardian feedback on key aspects of patient experience.
Reliability
We estimated internal consistency reliability using the Cronbach’s coefficient alpha for each composite measure. A reliability of at least 0.70 is considered acceptable for group-level comparisons (Nunnally and Bernstein, 1994). For composites with more than two items, we show the impact on Cronbach’s alpha of deleting one of the items from the composite. However, CAHPS scores are designed to evaluate care across units of care such as plans, physician groups, and hospitals, not individual patients.
For the Access, Provider Communication, and Office Staff composite measures, all items had less than 3% of records with missing values. For the Care Coordination composite measure, all items had less than 4% of records with missing values. We ran the Cronbach’s alpha excluding all missing data as well as with listwise deletion and the results were the same, except for the Access composite measure in which excluding all missing data increased Cronbach’s alpha by .01. Given the similarity of results, we have presented the Cronbach’s alpha values with the inclusion of cases with missing values (listwise deletion) in section 5.2.3.
Reference
Nunnally JC, Bernstein IH. (1994). Psychometric Theory. New York: McGraw Hill.
Table 5.2.3a (attached in 5.2.3a) shows the Cronbach’s alpha for each composite measure in the Adult CG-CAHPS Survey. For items within composite measures consisting of 3 or more items, the Cronbach’s alpha if the item were deleted is provided to determine if there was room for improving coefficient alpha by dropping an item. The table also shows the standardized correlation for the standardized item to total correlations.
All Cronbach’s alphas in the adult survey were above 0.70 except for the Care Coordination composite measure with a Cronbach’s alpha of 0.61. As shown in Table 5.2.3a, removal of any questions in this composite measure would not result in a higher Cronbach's alpha and Care Coordination is an important concept to patients in their experience of care. Further, all the item to total correlations were above 0.40. While Cronbach’s alpha fell below the conventional threshold for several composite measures, it is not the most critical metric in this context. More important is the reliability at the unit level (e.g., plan-level reliability), which better reflects the measure’s utility for quality improvement. Nonetheless, internal consistency remains a relevant consideration in health care, and the Consortium will keep this in mind when implementing future revisions of the instrument.
We assess reliability at the site level, which is the most relevant level of analysis for publicly reported CAHPS measures (Hays & Arnold, 1986, pp. 144-145). Since CAHPS surveys are used to compare groups/units, site-level reliability (which is directly related to the standard error of measurement) is used to determine the number of responses needed to obtain reliable information (Hays, Shaul, et al., 1999). Site reliability, which partitions within- and between-site variance, was calculated from the ICC and the Spearman-Brown prophecy formula in SAS version 9.4. Higher levels of site reliability correspond to more accurate performance measurement and a better ability to distinguish performance among practices. Reliability is not calculated when only one site is included in the decile or max/min value, since there would be no between-site variation. For cases where the number of respondents was tied across decile boundaries, all practice sites with that number of respondents were assigned to the same decile.
Like internal consistency reliability (i.e., Cronbach’s alpha), values of 0.70 and higher are considered acceptable for site-level reliability (Nunnally and Bernstein, 1994) and group comparisons. For example, CMS does not report (labeled as “Not available”) any score with reliability below 0.60, as that is considered low reliability. CMS reports scores that meet the sample size threshold and for which reliability falls between 0.60 and 0.70 but flags these scores as having low reliability and alerts consumers to interpret such scores with caution. Scores with reliability 0.70 or greater are reported without comment. Reliabilities of 0.85 or higher, where possible, are appropriate for applications such as pay-for-performance or actions that reward or classify individual practices.
The CAHPS Consortium has reported the reliability of the CAHPS measures at the appropriate unit of comparison since the beginning of the project over 25 years ago and for measure development throughout the project (e.g., Hays, Martino, et al., 2014; Hays, Berman, et al., 2014; Price, Stucky, et al., 2018).
For the Rating of Provider measure, fewer than 3% of respondents had missing values. We have presented the site reliability, excluding missing cases where data are missing.
References
Hays, R, Arnold, S. (1986). Patient and family satisfaction with care for the terminally ill. Hospice Journal, 7, 129-150.
Hays, R, Berman, L, Kanter, M, et al. (2014). Evaluating the psychometric properties of the CAHPS Patient-Centered Medical Home Survey. Clin Ther. 36(5), 689–696.e1.
Hays, RD, Martino, S, Brown, J, Cui, M, Cleary, P, Gaillot, S, Elliott, M. (2014). Evaluation of a care coordination measure for the Consumer Assessment of Healthcare Providers and System (CAHPS®) Medicare Survey. Medical Care Research and Review, 71, 192-202.
Hays, R, Shaul, J, William, V, et al. (1999). Psychometric Properties of the CAHPS™ 1.0 Survey measures. Medical Care. Volume 37(3) SUPPLEMENT, MS22-MS31.
Nunnally, JC, Bernstein, IH. (1994). Psychometric Theory. New York: McGraw Hill.
Price, RA, Stucky, B, Parast, L, Elliott, MN, Haas, A, Bradley, M, Teno, JM. (2018). Development of valid and reliable measures of patient and family experiences of hospice care for public reporting. Journal of Palliative Medicine, 21, 924-932.
The Rating of Provider measure’s overall site reliability is 0.81. The information in Table 2 provides overall and decile-level reliability. Deciles for this table are based on the number of respondents per site, and the “mean performance score” row in this table shows the mean top box scores averaged across practice sites.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.807 | NA | 0.735 | 0.785 | 0.769 | 0.766 | 0.813 | 0.807 | 0.775 | 0.824 | 0.809 | 0.840 | NA |
| Mean Performance Score | 80.3 | 75.0 | 76.8 | 77.7 | 79.8 | 80.1 | 80.3 | 80.8 | 81.3 | 82.1 | 81.9 | 82.2 | 88.7 |
| N of Entities | 2,486 | 1 | 248 | 269 | 242 | 254 | 235 | 244 | 248 | 260 | 244 | 240 | 1 |
| N of Persons / Encounters / Episodes | 232,292 | 12 | 11,060 | 16,533 | 17,260 | 20,216 | 20,460 | 23,066 | 25,464 | 29,308 | 30,827 | 37,820 | 266 |
The site reliability assessment for the Rating of Provider measure indicates that overall and across all deciles, the measure maintains acceptable reliability (e.g., greater than 0.7) indicating the measure effectively detects systematic variation among sites relative to random variation.
Validity
Several model fit indices were examined to determine how well the hypothesized factor structure, or composite measures fit the data, including chi-square divided by its degrees of freedom (𝝌𝟐/𝒅𝒇 ) (criteria: values less than 5.0; Schumacker & Lomax, 2004), comparative fit index (CFI) (criteria: values 0.95 or greater; Hu & Bentler, 1999), root mean square error of approximation (RMSEA) (criteria: values less than 0.06; Kline, 2005), and the standardized root mean square residual (SRMR) (criteria: values less than 0.08; Kenny, 2020).
We examined standardized factor loadings for each item on its respective composite measure. Factor loadings above 0.40 indicate that the item’s relationship to the composite measure is acceptable (Stevens, 2002).
References
Hu, L., & Bentler, PM. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118
Kenny, DA. (2020, June 5). Measuring model fit. Available at http://davidakenny.net/cm/fit.htm. Accessed October 2025.
Kline, RB. (2005). Principles and practice of structural equation modeling (2nd ed.) New York: The Guilford Press.
Schumacker R., & Lomax, R. (2004). A beginner’s guide to structural equation modeling
(2nd ed.). Lawrence Erlbaum.
Stevens, JP. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum
Tables 5.3.4a and 5.3.4b (attached in section 5.3.4a) show results for the model fit indices and standardized factor loadings for the adult dataset.
The Confirmatory Factor Analysis results for the Adult CG-CAHPS survey demonstrate strong model fit based on established criteria. The chi-square divided by degrees of freedom (χ²/df) was 127.28, exceeding the recommended threshold of less than 5.0, which is common in large samples due to the sensitivity of this index. However, the other fit indices all fall well within acceptable ranges: the Comparative Fit Index (CFI) was 0.98, surpassing the criterion of 0.95 or greater, indicating excellent model fit. The Root Mean Square Error of Approximation (RMSEA) was 0.05, meeting the standard of less than 0.06, and the Standardized Root Mean Square Residual (SRMR) was 0.04, comfortably below the threshold of 0.08.
The estimates for each standardized factor loading on the items in the composite measures assess convergent validity. All standardized factor loadings are at least 0.5, with the majority above 0.8, and all are statistically significant (p < 0.001), demonstrating the convergent validity of the measures.
These results support the hypothesized factor structure for the measures in the survey.
At the individual and site level, we examined the relationships between each composite measure and item’s top box score and the top box score for the provider rating measure using Spearman rank-order correlations. We expect the composite measures to be moderately to strongly related to the overall rating (Quigley et al., 2024).
We also examined Spearman rank-order intercorrelations among the composite measures to assess the extent to which they measure different constructs. As measures of patient experience, we expected the composite measures to be positively and significantly correlated. However, very large intercorrelations (e.g., > 0.80) suggest that the composite measures may not be sufficiently distinct to be considered separate measures (O’Brien, 2007).
One rule of thumb for correlations is:
0.10 is a small correlation
0.30 is a medium correlation and
0.50 is a large correlation
For the Rating of the Provider measure, we expect a positive and medium to large relationship with the CG-CAHPS composite measures with the strongest relationship with Provider Communication and Care Coordination, since all these measures reflect the provider’s behavior, communication, and knowledge for the patient’s care.
References
Quigley, DD., Elliott, MN., Qureshi, N., Predmore, Z., Hays, RD. (2024). Associations of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group
Survey scores with interventions and site, provider, and patient factors: A systematic review of the evidence. Journal of Patient Experience, 11, 1-14.
O’Brien, RM. (2007). A caution regarding rules of thumb for variance inflation factors. Qual Quant. 41, 673–690.
The Rating of Provider measure has medium to large correlations with all four composite measures. The site-level correlations between Rating of Provider and the composite measures range from 0.43 (Office Staff) to 0.82 (Provider Communication). The individual-level correlations between Rating of Provider and the composite measures range from 0.31 (Access) to 0.58 (Provider Communication).
Please see attachment 5.3.4a for validity testing results, which includes the site-level and individua-level correlations for the Rating of Provider measure and the composite measure and items in the Adult CG-CAHPS Survey, all assessed using Spearman rank-order correlations.
In this analysis, all correlations between the Rating of Provider measure and the composite measures fell within an acceptable medium to large range at the site and individual level (between 0.30 and 0.80), except for the correlation with the Provider Communication composite measure which had a correlation of 0.82 at the site level, just above 0.80. This stronger correlation is expected given both measures focus on the experience with the provider. Overall, the composite measures were positively and significantly correlated with the Rating of Provider measure. The results support the distinctiveness of the Rating of the Provider measure as a measure of patient experience, while also confirming its meaningful relationship to broader aspects of care.
Risk Adjustment
The Adult CG-CAHPS Survey results are not required to be risk adjusted by users. However, survey users, including public reporting entities, may voluntarily choose to adjust the data to account for patient case-mix differences if comparing practices. Guidance for this process is available in two key documents: “Preparing Data from CAHPS Surveys for Analysis” (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) and “Instructions for Analyzing Data from CAHPS Surveys” dated June 2025 (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). These resources provide instructions for coding the adjuster variables, imputing missing data for the adjusters, and incorporating adjustments in analyses using the CAHPS Analysis Program in SAS. The selection of adjuster variables and the calculation of risk-adjusted scores are user-defined. Users must also decide whether to impute missing data for the adjusters using each adjuster’s entity-level mean.
The CAHPS Analysis Program is a set of free SAS programs that enable survey users to conduct risk adjustment. The programs with test modules are available for download at https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/….
The CAHPS Analysis Program adjusts the data for case mix, generates a distribution of survey results for each of the measures, calculates the average score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. AHRQ’s CAHPS Consortium developed the CAHPS Analysis Program to work with all CAHPS surveys. It is updated periodically to add functionality, produce additional output types, and correct or debug issues with previous versions.
This section describes the rationale for case-mix adjustment that is not required but that CAHPS users may elect to use. The standard methodology used is case-mix adjustment via regression in a linear model. Without an adjustment, differences in CAHPS scores between entities could be due to case-mix differences rather than true differences in quality.
The current CAHPS Analysis Program suggests adjusting for general health status, mental health status, age, and education. Studies have found that patient and consumer survey responses about experiences and satisfaction with healthcare correlate with personal characteristics like general health, mental health/depression, education, and age (Simon et al., 2009; Rahmqvist and Bara, 2010; Zaslavsky et al., 2001; Martino et al., 2011; Eselius et al., 2008; Elliott et al., 2009).
Health status and age are two patient characteristics frequently associated with reports of the quality of their medical care. People in worse health tend to report lower patient experience satisfaction and more problems with care than do people in better health perhaps because sicker patients have more complex health care needs and may tend to report more problems with coordination or communication. Older patients tend to report greater satisfaction, better patient experience, and fewer problems than younger patients, although this association is usually not as strong as that between health status and ratings (Hatfield and Zaslavsky, 2017; Eselius et al., 2008).
Education is self-reported by patients who take the CAHPS surveys. Studies have shown that more educated patients report more problems, perhaps because they have higher expectations rather than because they receive lower quality of care (Sofaer and Firminger, 2005). However, in a multivariate analysis using Medicare Advantage CAHPS data, Hatfield and Zaslavsky (2017) found that education had less influence on CAHPS dimension scores than self-reported general and mental health.
Different CAHPS surveys adjust for different variables, and the variables included here are not the only adjustment factors, even for the practice setting. It should be noted that race/ethnicity is not typically included as case-mix adjuster variables as doing so may mask disparities in care. CAHPS data can also be adjusted for other factors such as survey administration mode (Peipert et al., 2017). For example, a study by Drake and colleagues (2014) found that telephone respondents gave more positive responses than mail respondents did. The AHRQ CG-CAHPS Database did not adjust for survey mode.
The AHRQ CG-CAHPS Database adjusted only for age, education,general health status, and mental health status.
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99. doi: 10.1111/1475-6773.12160.
Elliott, MN, Zaslavsky, AM, Goldstein, E, Lehrman, W, Hambarsoomians, K, Beckett, MK, Giordano, L. (2009). Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 44(2 Pt 1), 501-18. doi: 10.1111/j.1475-6773.2008.00914.x.
Eselius, LL, Cleary, PD, Zaslavsky, AM, Huskamp, HA, Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
Hatfield, LA, Zaslavsky, AM. (2017). Implications of variation in the relationships between beneficiary characteristics and Medicare Advantage CAHPS measures. Health Serv Res. 52(4),1310-1329.
Martino, SC., Elliott, MN., Kanouse, DE., Farley, DO., Burkhart, Q., Hays., RD. (2011). Depression and the health care experiences of Medicare beneficiaries. Health Services Research. 46(6pt1), 1883–904.
Peipert, JD, Brown, JA, Cui, M, Hays, RD (2017). Differences in mail and telephone responses to the CAHPS In-Center Hemodialysis Survey. Ann Clin Nephrol. 1(1), 1.
Rahmqvist, M, Bara, AC. (2010). Patient characteristics and quality dimensions related to patient satisfaction. International Journal for Quality in Health Care. 22(2), 86–92.
Simon, GC, Rutter, M, Crosier, J, Scott, BH, Operskalski, LE. (2009). Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? Psychiatric Services. 60(1), 67–73.
Sofaer, S, Firminger, K. (2005). Patient perceptions of the quality of health services.
Annual Review of Public Health. 26, 513–59.
Zaslavsky, A, Zaborski, L, Ding, L, Shaul, JA, Cioffi, MJ, Cleary, PD. (2001) Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review. 22(3), 109–26.
Tables 5.4.3a through 5.4.3c in Attachment 5.4.3 show descriptive characteristics for the risk/case mix variables tested: age, general health status, mental health status, and education. Practice sites had adult respondents who were predominantly 55 or older (83%). Forty-one percent (41%) of respondents reported having at least a 4-year college degree. Forty-one percent (41%) of respondents reported being in excellent or very good general health, and 54% of respondents reported being in excellent or very good mental health.
Case-mix adjustment was conducted using the AHRQ CAHPS Analysis Program (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/…). In the CAHPS Analysis Program, we set the program to impute the site mean if data were missing for that variable to avoid losing observations because of missing data. This is generally acceptable because, the size of the adjustment and the amount of missing data on adjusters are typically small. The case-mix adjusted scores are based on regression analyses and the results include case-mix adjusted coefficients. For detailed information on case-mix adjustment refer to page 46 of “Instructions for Analyzing CAHPS Data” document (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…).
To quantify the effect of case-mix adjustment on the ranking of practices, using data from the Adult CG-CAHPS Survey we calculate the Pearson product moment correlation coefficient, Kendall Tau correlation coefficient, and the maximum difference between the adjusted and unadjusted practice mean scores, as shown in Attachment 5.4.5a. The adjustment factors include age, education, mental health status, and general health status. Appropriate case-mix adjusters are patient-level variables that are not under the control of the provider (“exogenous”) but result in different scores even when the quality of care is the same. For example, patient education and age are not under the control of the provider.
Eselius and colleagues (2008) published results of an analysis of case-mix adjustments for the CAHPS Health Plan Survey, a survey with similar composite measures to the CG-CAHPS Survey about access to care, provider communication and customer service. After selecting appropriate adjusters based on explanatory power in separate linear regression models, the authors determined the impact of case-mix adjustment on their sample health plans. Specifically, they examined the size of the adjustments and the extent to which adjustments impacted the ranking of health plans. Case-mix adjustments had only modest effects on health plan ratings and rankings. The authors also found that mental health status was a strong predictor of patient-reported experience. For this purpose, this question was added to CG-CAHPS: “In general, how would you rate your overall mental or emotional health?”
The Pearson product-moment correlation coefficient is widely used and understood. It assesses the linear association between the adjusted and unadjusted scores and ranges between -1 to 1. Because the ranking of scores is often important in public reports of CAHPS results, we also calculate Kendall’s Tau (Kendall rank correlation coefficient). Tau is the correlation between the rank orders of the adjusted and unadjusted scores. The Kendall Tau statistic also has a range of -1 to +1, so that it has a range comparable to other correlation coefficients. Tau can be interpreted as the percentage of pairs of units (e.g., practice sites) that switched ordering because of case-mix adjustment [100*(1-Tau)/2].
We also calculate the maximum absolute value difference between adjusted and unadjusted mean scores among practices.
Reference
Eselius, LL., Cleary, PD., Zaslavsky, AM., Huskamp, HA., & Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
The selection of case-mix adjusters is based on prior research across multiple datasets of the CG-CAHPS Database and is consistent with how scores are calculated. Results are found in Attachment 5.4.4a and 5.4.5a.
All adjusters were statistically significant in the regression models for this measure, except for education level 4-year college graduate.
The adjusters resulted in very similar top box scores, with the Pearson correlation of unadjusted and adjusted mean scores being close to 1, at 0.98 and the Kendall Tau correlation at 0.88. Further, the difference in mean scores for this measure was 0.28. This suggests that adjusting for case-mix can help level the playing field for this measure.
The final model includes four adjusters: age, education, general health status, and mental health status.
Use & Usability
Use
Provides patient experience, clinical quality and total cost of care ratings for medical groups in 44 CA counties. The patient experience ratings come from the Patient Assessment Survey (PAS) based on CG-CAHPS for adult and child practices. The program is reportedly being sunset in 2025 (https://www.pbgh.org/program/patient-assessment-survey/).
A total of 204 medical groups are included in the 44 California counties. Only 14 counties in the state do not have any medical groups reported. Number of patients: Information not available.
Medical group practice level of analyses, outpatient primary care practices
Through MIPS, clinicians can earn performance-based payment adjustments for services they provide to Medicare patients. The CG-CAHPS for MIPS survey is an optional quality measure that groups participating in MIPS can administer to their adult population.
MIPS is a national program that includes most physicians and group practices in the U.S who provide services to Medicare patients. The CAHPS sample frame includes all patients from a practice-supplied roster from groups participating in MIPS. Number of patients: Information not available.
Group practice level
Medicare Shared Savings Program ACOs collaborate to give coordinated high-quality care to people with Medicare. They are required to report the CAHPS for MIPS Survey, which is a version of the Adult CG-CAHPS Survey.
Over 470 ACOs are participating in the Shared Savings Program for Performance Year 2025. ACOs are located in almost all states and the District of Columbia. Number of patients: Information not available.
Medicare Shared Savings Program Accountable Care Organizations (ACOs)
NCQA recognizes clinicians and practices in key areas of performance. The recognition program includes optional reporting for practices that use the Adult CG-CAHPS survey with PCMH items.
The NCQA PCMH Recognition program is the most widely adopted PCMH evaluation program in the country. More than 13,000 practices (with more than 67,000 clinicians) are Recognized by NCQA. Number of patients: Information not available.
Group practice level
Primary Care First was a voluntary alternative payment model (through December 2025) that rewarded value and quality by offering an innovative payment structure to support the delivery of advanced primary care and administered a version of the Adult CG-CAHPS survey.
26 US regions and approximately 1,700 participant practices. Number of patients: Information not available.
Group practice level
UCLA Health, a large health system in southern California, administers the CAHPS Clinician & Group Survey on an ongoing basis to adult patients and the parents or guardians of pediatric patients (12 and younger) (Adult and Child). The organization has developed a set of data displays to manage and convey the tremendous amount of information they collect for quality improvement purposes.
Southern California as part of the University of California system. They collect data from over 1300 reporting physicians, 279 reporting offices, and 28 reporting clinical departments. Number of patients: Information not available.
Physician and office/clinic level
Usability
Actions to Improve Patient Experience
CAHPS® surveys play an important role as a quality improvement (QI) tool for healthcare organizations that use the standardized data to:
- Identify relative strengths and weaknesses in their performance.
- Determine where they need to improve.
- Track their progress over time.
AHRQ has made available a CAHPS Ambulatory Care Improvement Guide which is a comprehensive resource for health plans, medical groups, and other providers seeking to improve their performance in the domains of patient experience measured by CAHPS surveys. AHRQ also has created a short video to help improve patient experience https://www.ahrq.gov/cahps/quality-improvement/index.html#:~:text=CAHPS….
The steps are:
- Compare CAHPS survey scores to other health care organizations to determine how the plan is doing in comparison to others.
- Examine how CAHPS scores are changing over time.
- Identify priorities based on these comparisons
- Confirm these priorities based on other sources of information (e.g., patient complaints, patient comments)
- Find out what is actually happening with patients and why.
- Brainstorm with staff to determine the best strategies for improvement.
In addition, AHRQ held a research meeting in 2020 to discuss how to improve patient experience and provided summaries of the presentations: https://www.ahrq.gov/cahps/news-and-events/events/2020-meeting-summary…
Difficulty in Increasing Response Rates
Users are also provided advice for improving response rates:
- Improve initial contact rates by making sure that addresses and phone numbers are current and accurate (e.g., identify sources of up-to-date sample information, run a sample file through a national change-of-address database, send a sample to a phone number look-up vendor).
- Use all available tracking methods (e.g., Lexis-Nexis, Internet database services and directories).
- Improve contact rates after data collection has begun (e.g., increase maximum number of calls, ensure that calls take place at different day and evening times over a period of days, mail second reminders, use experienced and well-trained interviewers).
- Consider using a mixed-mode protocol. In field tests, the combined approach was more likely to achieve a desired response rate than did one mode alone.
- Train interviewers on how to deal with gatekeepers.
- Train interviewers on refusal aversion/conversion techniques.
AHRQ has several resources for practice sites to improve performance on the CAHPS Surveys. They created the CAHPS Ambulatory Care Improvement Guide, which is a comprehensive resource for organizations seeking to improve their performance: https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/improv…. There are also case studies and webcasts that share insights and best practices in improving patient experience with care: https://www.ahrq.gov/cahps/surveys-guidance/hp/improve/index.html.
To improve on Rating of the Provider, practices can encourage providers to treat patients with empathy and respect, make eye contact, take the time to listen carefully, and ask if all questions have been addressed. Further, they can ensure they have reviewed the patient’s medical history during the visit and have staff that are coordinating their care. Essentially, all ways listed for improving the other CG-CAHPS composite measures should help improve the rating of the provider.
Throughout the development process, the CAHPS Consortium has incorporated the data or input from these various sources in an incremental process of revision and refinement to develop measurement that is more precise and to produce survey data that would better meet the information needs of consumers and other stakeholders. Between versions 1.0 and 2.0 of Child CG-CAHPS, pediatric experts felt that the Child version of the CG-CAHPS Survey would benefit from a more comprehensive measurement of development and prevention. The CAHPS Consortium worked with the American Academy of Pediatrics and other key stakeholders to develop 11 new items that address development and prevention. These items were grouped into new two composite measures for the Child Survey version 2.0 (Gallagher et al., 2009).
The CG-CAHPS Survey was initially developed with the standard CAHPS 4-point response scale. The field test data that was used for the initial endorsement consisted of several testing sites in the US. One of the larger field tests was conducted in Massachusetts as part of the Massachusetts Health Quality Partners (MHQP) statewide surveying initiative. At that time, MHQP used a 6-point response scale instead of the standard CAHPS 4-point scale. Based on that evidence, the CG-CAHPS Survey was endorsed with a 6-point response scale. Affirmed by significant user feedback, additional testing was conducted to add to the other field test data to confirm the properties and function of the standard CAHPS 4-point response scale and the CG-CAHPS Survey was updated to the 4-point response scale so that results could be aligned across other CAHPS surveys (e.g., CAHPS Health Plan Survey, CAHPS Hospital Survey (HCAHPS)). Drake et al. (2014) examined how different response scales affect responses to the CG-CAHPS survey among 6,500 patients. They found that compared to the 4-category response options surveys, respondents to the 6-category response options surveys had 41 percent more missing items. There were no significant differences between the 4-category and 6-category response option surveys’ average composite measure score or provider-level reliability.
For the 3.0 version, the survey switched from a 12-month recall period to a 6-month recall period to improve accuracy of recall and focus on more recent visits.
The CAHPS Consortium hears user feedback during research studies and development. Users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. The CAHPS consortium also solicits feedback via focus groups with patients in developing survey content and design. When changes are proposed to the survey, the changes also often go through a public comment period and those comments are summarized and posted on the AHRQ site (e.g., https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99.
Gallagher, P, Ding, L, Ham, HP, Schor, EL, Hays, RD, Cleary, PD. (2009). Development of a new patient-based measure of pediatric ambulatory care. Pediatrics. 124(5), 1348-54.
No additional detail.
In late 2009, version 2.0 addressed the fact that many people receive care in the ambulatory setting from non-physician providers such as nurse practitioners and physicians´ assistants. This change to “this provider” was in response to requests from the medical community for a survey instrument that would allow patients to report on their experiences with all their health care practitioners.
The items designed to identify chronic conditions were moved from the core survey version 2.0 to a supplemental item set.
After extensive testing and receiving feedback from users, with the release of CG-CAHPS Version 2.0, the CAHPS consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are surveyed.
Based on the stakeholder and user feedback obtained through public comment, technical expert review, and further testing, the following key changes were made in the release of the CG-CAHPS 3.0 version:
- One instrument, in contrast to the three instruments available for the 2.0 version.
- Use of a 6-month reference time period rather than a 12-month reference period.
- New and modified composite measures:
- New composite measure for "Care Coordination."
- Modified the composite measure for "Access."
- Modified the composite measure for "Communication."
- A modified Patient-Centered Medical Home Item Set.
- Shift of development and prevention items from the core Child Survey to the Patient-Centered Medical Home Item Set.
- Overall reduced length.
For the 3.1 version, with the COVID-19 pandemic changing how some health care was delivered (e.g., video or phone rather than in-person), the instrument was updated again to change instructions and gate question wording to include these types of visits.
No additional detail.
Rating of Provider has shown minimal improvement between 2022 and 2023, increasing only slightly from 79.4% to 80.3%. In comparison, the 2019 AHRQ CG-CAHPS Chartbook reported an average of 84% in 2015, decreasing to 79% in 2019. This trend suggests a gradual decrease in performance from 2015 to 2022, with only modest gains in 2023. These results underscore a need for focused efforts to enhance the overall patient care experience.
No unexpected findings.
No unexpected findings.
Comments
Staff Preliminary Assessment
CBE #0005-5 Staff Preliminary Assessment
Importance
Strengths
- The developers provided information about identified performance gaps.
- Data from the 2023 Primary Care First Patient Experience Survey, of which CG-CAHPS is embedded, show a performance gap, with decile ranges from 64.1% to 92.2%, indicating variation in measure performance and less than optimal performance across the target population.
Limitations
- The logic model provided does not clearly articulate the relationships between inputs, activities, and outcomes for the Rating of Provider measure. For example, actions cited within the logic model are patient-centric, but do not include actions that can be taken by providers to improve their ratings. Adding such information would strengthen the logic model.
In terms of meaningfulness of the Rating of Provider measure to patients, the developer provides narrative content about how the measure can be useful to providers and to patients. However, no empiric evidence is sited and no input from patients, for this endorsement cycle, was included.
The developer cited one article from 2018 that indicated rating of ones provider is associated with other composites. However, the other composites were not included in the submission.
Rationale
- This maintenance measure is rated as 'Not Met But Addressable' for importance due to insufficient evidence that the Rating of Provider measure is meaningful to patients. The submission could be strengthened by engaging patients directly to elicit feedback on the meaningfulness of the Rating of Provider measure and/or citing empiric evidence within the past 5 years that indicates provider rating is a meaningful concept for patients and contributes to their overall experience of care. There is also a lack of evidence for the importance of the measure. Lastly, the logic model could be strengthened by adding provider actions that can improve care coordination for patients.
There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient experience justify the use of the measure, due to the demonstrated performance gap.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- The developer indicated that the only change in the Adult CG CAHPS instrument was the addition of language to expand the context for questions to include virtual and telehealth visits.
The AHRQ website contains the documentation on CG CAHPS administration, confidentiality, scoring, and other topics.
There are no fees, licensing, or other requirements to use any aspect of the Adult CG CAHPS instrument or the resulting composites.
Limitations
- The feasibility domain can be strengthened by providing the median cost of vendor engagement to administer CAHPS (or a similar metric).
Rationale
- This maintenance measure meets the criteria for 'Met' due to availability of data and low degree of missingness, clear and implementable data collection strategies, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths
- Data sources used for reliability analysis are adequately described and include a database with survey results collected from October 3, 2023 through December 19, 2023 via mail with phone follow-up.
The developer conducted reliability testing using the ICC and the Spearman-Brown prophecy formula at the accountable entity-level.
Limitations
- The developer performed reliability testing for this maintenance measure, namely, they conducted accountable entity-level reliability testing at the site level using the unadjusted measure rather than the case-mix adjusted measure with the rationale being case-mix adjustment is not needed when entities are not compared to each other.
The entities included in the testing were characterized by practice site and a minimum sample size of 10 completed surveys. Developer states approximately 50 completed surveys per practice are needed for statistically reliable results and does not give a rationale for minimum sample size of 10.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for reliability because the developer performed the required reliability testing for this measure but it is unclear whether the results demonstrate sufficient reliability at the accountable entity-level. However, the identified limitations are deemed addressable, as the developer may consider using case-mixed adjusted data for reliability testing and providing their rationale for a minimum sample size of 10 completed survey per practice site. By addressing these issues, there is potential to demonstrate sufficient reliability at the accountable entity-level.
Strengths
- The developer performed the required validity testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) validity testing at the clinician group level. The data source used for validity analysis was Primary Care First (PCF) Patient Experience of Care (PEC) Survey administered to adults in October through December of 2023. Validity analyses included 2,486 practices in 23 states.
The developer conducted empirical validity testing at the accountable entity level using Spearman’s rank-order correlation on unadjusted top box scores. The developer hypothesized that Rating of the Provider would be positively and significantly correlated with the four composites, positing that each are key components of the care experience, but not so highly correlated as to suggest the measures are not distinct, citing rho > 0.80 as an example of a very large correlation. Provider Rating was significantly, positively correlated at the clinician group level with the composites Access (rho = 0.49) and Office Staff (rho = 0.43), in line with hypotheses.
The developer conducted statistical case-mix adjustment, selecting case-mix indicators that are present at the start of care and have a significant correlation with the outcome.
Limitations
- With respect to entity-level validity testing, while the developer has indicated that quality improvement activities can affect more than one IDM, their hypothesis regarding the mechanisms involved and the degree to which these mechanisms are shared between IDMs is not clearly supported by the evidence review and logic model provided. In the absence of an external gold standard against which to validate at least one of the IDMs, this submission would be strengthened by additional support guiding development of hypotheses about expected magnitudes of each correlation.
The developer used unadjusted scores for validity testing when use of adjusted scores might help rule out known sources of confounding.
The developer states that case-mix adjustment of the measure is optional by the user, and does not provide guidance or supporting rationale stating when adjustment is or is not appropriate. The developer did not provide evidence demonstrating variation in the prevalence of case mix factors across accountable entities. The statistical testing results provided by the developer do not reflect the impact of adjustment on providers at the high or low extremes of the case mix.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the accountable-entity validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on patient experience of care and can distinguish good from poor performance to a limited extent. This submission would be strengthened by explicitly ruling in mechanisms and ruling out confounders for the effect of quality of care on patients' Ratings of Provider.
The developer employed a statistical case-mix adjustment approach, utilizing a conceptual model designed to account for demographic case-mix factors. Variation in the prevalence of case-mix indicators across different entities was not shown and the model testing results provided do not reflect whether case-mix differences are being appropriately accounted for.
Use and Usability
Strengths
- The developer noted that users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. Also, when changes are proposed to CAHPS surveys, requests for public comment is posted on the AHRQ website.
The developer does provides some, albeit limited, information about how measured entities can improve their Rating of Provider measure score: empathy, respect, listening, and preparedness for appointments.
The Rating of Provider measure is currently used in the California Health Care Quality Report Cards program, the Medicare Shared Savings Program Accountable Care Organizations (ACOs), the NCQA Patient-Centered Medical Home (PCMH) Recognition Program, and the CMMI Primary Care First Model.
Limitations
- The developer reports a slight increase in the Rating of Provider score from 2022 (79.4%) to 2023 (80.3%). Despite the slight increase, the Rating of Provider composite scores have declined and then flattened over time. The scores from 2015, 2019, and 2023 were 84%, 79%, and 80% respectively. The developer does not provide an explanation for the lack of improvement over time.
Rationale
- This maintenance measure is rated 'Not Met But Addressable' for Use and Usability. The measure is actively used in at least one accountability application and the developer provides a small number of actions that providers can take to improve their Rating of Provider measure scores. However, the submission can be strengthened by including evidence-based actions with citations and an explanation of the limited improvement in the Rating of Provide composite scores from 2015 to 2023.
Committee Independent Review
Committee Independent Review Comment
Importance
The Rating of Provider measure reflects patients’ overall assessment of their care experience and is widely used across accountability and quality improvement programs. Recent data demonstrate meaningful variation in performance across practices, indicating opportunity for improvement and supporting a reasonable business case for continued measurement. However, the submission does not include recent empiric evidence or direct patient input demonstrating that the single-item Rating of Provider measure is meaningful to patients as a standalone construct. In addition, the logic model does not clearly articulate provider-level actions that directly influence performance on this measure. These gaps are addressable through engagement with patients to assess meaningfulness and by strengthening the logic model to link provider behaviors to outcomes.
Closing Care Gaps
The developer did not address the Closing Care Gaps domain for this measure. The submission does not describe how the Rating of Provider measure is intended to identify disparities, target specific populations, or inform interventions to reduce gaps in care or experience across patient groups. As a result, there is insufficient information to assess whether or how this measure contributes to closing care gaps.
Feasibility Assessment
This measure is feasible to implement in real-world settings. The data collection process is well established, with clear and standardized guidance available through AHRQ on survey administration, confidentiality, scoring, and response rate calculation. The measure relies on routinely collected patient-reported survey data, demonstrates low levels of missingness, and can be implemented without licensing fees or proprietary requirements. Although the submission could be strengthened by providing additional information on the typical cost of vendor administration, this limitation does not materially affect feasibility. Overall, the measure meets feasibility criteria due to its accessibility, standardized methodology, and demonstrated implementability across diverse practice settings.
Scientific Acceptability
The reliability criterion is not met but addressable. The developer conducted accountable entity–level reliability testing using recent survey data and appropriate statistical methods, including ICC and the Spearman-Brown prophecy formula. However, reliability testing was performed using unadjusted scores, and the rationale for not using case-mix adjusted data was not fully justified. In addition, the minimum sample size of 10 completed surveys per practice site was not adequately supported, particularly given the developer’s statement that approximately 50 completed surveys are needed for statistically reliable results. These limitations reduce confidence that the measure reliably distinguishes performance across entities. The identified gaps are addressable through clearer justification of sample size thresholds and additional testing using adjusted data.
The validity criterion is not met but addressable. The developer conducted required accountable entity–level validity testing using recent Adult CG-CAHPS data and demonstrated statistically significant, moderate correlations between the Rating of Provider measure and related composites, consistent with stated hypotheses. These findings provide partial support for construct validity. However, the submission does not clearly articulate the mechanisms linking provider actions to patient ratings, and hypotheses regarding expected correlation magnitudes are not fully supported by the evidence review or logic model. In addition, validity testing relied on unadjusted scores, limiting the ability to rule out confounding. These gaps are addressable through clearer causal pathways, strengthened logic models, and expanded analyses incorporating case-mix adjustment.
Use and Usability
The Rating of Provider measure is actively used in multiple accountability and quality improvement programs, demonstrating baseline usability and relevance. The developer provides limited guidance on actions providers can take to improve performance, such as demonstrating empathy, respect, and preparedness. However, these recommendations are high-level and not supported by recent empirical evidence. In addition, although scores increased slightly from 2022 to 2023, performance has declined and then flattened over time, and the submission does not offer an explanation for this trend. The measure’s usability would be strengthened by including evidence-based improvement strategies with citations and clearer context for observed performance patterns. These gaps are addressable and do not negate the current utility of the measure.
Summary
The Adult CG-CAHPS Rating of Provider measure remains an important component of patient experience measurement and is widely used across federal, state, and value-based accountability programs. The measure captures patients’ overall assessment of their provider and reflects aspects of care that only patients can reliably report. Performance data demonstrate meaningful variation across practice sites, supporting the business case for continued measurement.
While several domains in this maintenance review were rated as not met but addressable, the identified limitations do not undermine the relevance of the measure. Opportunities exist to strengthen the logic model, provide more recent empiric evidence of patient meaningfulness, clarify reliability testing assumptions, and expand evidence-based guidance for improvement. These gaps are addressable and would enhance confidence in the measure’s interpretability and impact.
Overall, the Rating of Provider measure continues to provide valuable insight into patient experience and should remain part of the CG-CAHPS survey set, with targeted refinements to strengthen scientific acceptability and usability over time.
CBE0005-5
Importance
Met
Closing Care Gaps
Not addressed
Feasibility Assessment
It is doable
Scientific Acceptability
Looking at the staff assessment, I agree with that feedback
Looking at the staff assessment, I agree with that feedback
Use and Usability
Again, only high-level tactics to improve, nothing concrete or specific.
Summary
Measure is a valuable measure, and while aspects are not met, with some improvements around reliability and clear strategies to promote improvements, it will only strengthen the measure.
0005
Importance
This measure is important to patients/families.
Closing Care Gaps
No Comments
Feasibility Assessment
No Comments
Scientific Acceptability
No Comments
No Comments
Use and Usability
No Comments
Summary
No Comments
High Importance, Case-Mix Adjustment Approach Acceptable
Importance
Disagree with staff assessment. The wide distribution of scores overall suggests that patients have identified clear issues with the health care system reflected in their responses here, and several studies are provided supporting the link between overall patient experience with their providers and clinical quality outcomes.
Closing Care Gaps
N/A
Feasibility Assessment
Agree with staff assessment. Measure is clearly feasible and widely used.
Scientific Acceptability
Disagree with staff assessment. I do not anticipate issues with reliability of the measure score if it was re-ran with case mix adjusted figures, as the developer provided evidence that case mix adjustment has only a modest effect on the measure score, and in any case is an optional “add-on” to the existing measure score calculation. A mitigation strategy is presented for low reliability entities.
Disagree with staff assessment. I do not find that the issues presented with the case mix adjustment specific to validity are sufficient to affect the rating of the criterion, as these are optional adjustments and largely under the discretion of the individual adjuster.
Use and Usability
Disagree with staff assessment. Although there is little explanation provided for changes in performance, the changes in performance are slight and could be consistent with a positive upward drift.
Summary
Although the submission could be strengthened in some areas, the specific weaknesses are not sufficient to threaten the continued endorsement of this measure. This measure is a rare source of patient-reported data about the health care system, and reflects the performance of entities that are increasingly critical in guiding the course of health care in the United States, as health plans assume ever greater levels of control over provisioning care for their members.
Not met
Importance
The developer appears to be resting on the fact that the measure is in use; therefore, it must be useful. If it is useful, it must be important. If that is the case, where are the data? The developers cite old studies, present no new data, and do not provide any real evidence that just because there are users there is importance. I am not convinced. The developers need to do the work to establish importance. The measures have been around long enough to have a history of importance--if such importance exists--and a history of effectiveness--if such effectiveness exists.
Closing Care Gaps
Not addressed.
As indicated by the commenters and the staff in the importance discussion, this measure ought to be closing care gaps. The fact that the developer is not addressing this criterion reinforces the posit that it is not important, either.
Feasibility Assessment
The measure's feasibility ought to reflect not only its data collection possibilities, but also whether it has a benefit relative to its cost--including its personnel time and respondents' time. It provides no such data. For as long as it has been in use, such data ought to be available. If the measure is truly feasible, the developer ought to present the data with pride and vigor. That is not the case.
Scientific Acceptability
Consistent with the staff assessment, I agree that the developer submitted reliability data; however, examination of the measure's reliability ought to address all instances in which users employ the measure. As reported, this measure lacks such reliability measurement.
As indicated by the staff, this measure's validity testing does not circumscribe the boundaries in which confounding might occur. It is not clear that the measure can distinguish signal from noise given that the developers provide no boundaries for either.
Use and Usability
I am in complete concurrence with the staff assessment; however, given the period over which users have employed the measure the developer ought to have evidence on its usability that goes beyond outlining where organizations use it. If a measure is usable, it stands to reason that people have no difficulty in its use and that there is evidence to support that lack of difficulty. The developers present none of that.
Summary
Considering how long this measure has been in use, the developers ought to be able to produce an adequate supply of current measure data to support importance, feasibility, acceptability, and use and usability. They do not. This measure is not acceptable in its current form.
Rating of the Provider
Importance
The Adult CG-CAHPS Rating of the Provider measure captures a core element of patient experience: the patient’s overall assessment of their clinician. This global rating reflects patients’ integrated perceptions of communication, trust, respect, and preparedness, and remains a meaningful signal of care quality from the patient perspective. Performance data from the 2023 Primary Care First Patient Experience Survey demonstrate a clear performance gap, with decile ranges from 64.1% to 92.2%, indicating substantial variation across clinician groups and opportunity for improvement.
However, the importance criterion is rated Not Met but Addressable due to limitations in the supporting evidence and conceptual framing. The submitted logic model does not sufficiently articulate how provider-level actions influence the overall rating, nor does it clearly link improvements in provider ratings to downstream outcomes such as care engagement, adherence, or continuity. In addition, while narrative rationale is provided, the submission lacks recent empirical evidence and direct patient input demonstrating the specific meaningfulness of the Rating of Provider measure for this endorsement cycle.
These gaps are addressable. Strengthening the logic model to include provider-driven improvement levers and incorporating recent patient-centered evidence would more clearly demonstrate the importance of this measure and reinforce its continued relevance for accountability and quality improvement.
Closing Care Gaps
The Rating of the Provider measure demonstrates a meaningful performance gap, with wide decile variation across clinician groups, indicating opportunity for improvement and relevance for accountability programs. However, the submission does not explicitly address the Closing Care Gaps domain. Specifically, the developer does not describe how results from this measure are intended to directly drive targeted interventions to reduce disparities, improve equity, or close identified gaps in patient experience over time.
While the measure is widely used across multiple federal and state programs and serves as a high-level indicator of patient experience, additional information is needed to demonstrate how stratified results, trend analyses, or subgroup reporting are used to identify and close care gaps among different populations. The submission would be strengthened by describing how organizations can use Rating of Provider results in combination with composite measures or demographic stratification to implement focused improvement strategies and monitor progress in closing gaps in patient experience.
These limitations are addressable. Providing examples of how the measure supports equity-focused quality improvement or contributes to closing experience gaps across patient populations would strengthen alignment with this domain.
Feasibility Assessment
The Adult CG-CAHPS Rating of the Provider measure meets the feasibility criterion. The measure relies on a well-established, standardized survey instrument with clearly defined administration, scoring, and reporting guidance available through AHRQ. Data collection processes are well documented, and the survey can be administered using multiple modes, including mail with telephone follow-up, supporting broad implementation across diverse practice settings.
The developer adequately addresses respondent burden, data completeness, and data integrity. The survey is administered outside of clinical encounters, minimizing disruption to patient care and provider workflows. Required data elements are readily available, and safeguards are in place to protect patient confidentiality, including de-identification of responses and suppression of results for small sample sizes.
There are no licensing fees or proprietary requirements associated with use of the measure, which further supports scalability and sustainability. While feasibility could be strengthened by providing additional information on the median cost of vendor administration, this limitation does not materially affect the implementability of the measure.
Overall, the measure demonstrates strong feasibility and can be reliably implemented across healthcare settings for performance measurement, benchmarking, and quality improvement purposes.
Scientific Acceptability
The developer conducted accountable entity-level reliability testing for the Rating of the Provider measure using data from the Primary Care First Patient Experience of Care Survey collected between October and December 2023. Reliability was assessed using the intraclass correlation coefficient (ICC) and the Spearman-Brown prophecy formula, and data sources and methods were clearly described.
However, reliability testing was conducted using unadjusted scores rather than case-mix adjusted scores, with the rationale that adjustment is not required when entities are not directly compared. In addition, testing included practice sites with a minimum of 10 completed surveys, despite the developer noting elsewhere that approximately 50 completed surveys per practice are typically needed for statistically reliable results. The rationale for this lower minimum sample size was not fully explained.
These limitations introduce uncertainty regarding whether sufficient reliability has been demonstrated at the accountable entity level. The concerns are addressable through use of case-mix adjusted data for reliability testing and clearer justification of minimum sample size thresholds.
The developer performed required accountable entity-level validity testing using data from the 2023 Primary Care First Patient Experience of Care Survey, including 2,486 practices across 23 states. Empirical validity testing demonstrated statistically significant, moderate correlations between Rating of the Provider and related composites, including Access and Office Staff, consistent with stated hypotheses and supporting construct validity.
The developer also applied a statistical case-mix adjustment model based on demographic factors present at the start of care. However, validity testing relied on unadjusted scores, and the submission did not clearly demonstrate how adjustment affects results across entities or at performance extremes. In addition, the logic model and evidence review did not sufficiently support hypotheses regarding shared mechanisms across measures or rule out potential confounding factors.
These limitations are addressable. Additional justification for hypotheses, use of adjusted scores in validity testing, and clearer guidance on when adjustment is appropriate would strengthen the validity evidence.
Use and Usability
The Rating of the Provider measure is actively used across multiple accountability and quality improvement programs, including the California Health Care Quality Report Cards, the Medicare Shared Savings Program Accountable Care Organizations, the NCQA Patient-Centered Medical Home Recognition Program, and the CMMI Primary Care First Model. Its continued use across federal, state, and accreditation programs demonstrates strong usability, relevance, and feasibility for benchmarking and public reporting.
The developer provides general guidance to support improvement, including emphasizing empathy, respect, active listening, and preparedness for appointments. These actions align with established patient-centered care principles and offer a practical starting point for providers seeking to improve performance.
However, the measure does not fully meet the Use and Usability criterion. While performance scores show a slight increase from 2022 to 2023, longer-term trends indicate a decline followed by relative stagnation since 2015, without sufficient explanation of contributing factors. In addition, improvement strategies are described at a high level and are not clearly embedded within the measure’s logic model to guide systematic and sustained improvement.
These limitations are addressable. Providing clearer interpretation of longitudinal performance trends and strengthening alignment between measure results, evidence-based improvement strategies, and the logic model would enhance usability and support more actionable use by providers and stakeholders.
Summary
The Adult CG-CAHPS Rating of the Provider measure remains an important and widely used indicator of patient experience and perceived quality of care. The measure benefits from strong uptake across federal, state, and accreditation programs, underscoring its relevance for accountability, benchmarking, and quality improvement.
The submission demonstrates that the measure continues to identify meaningful performance variation across providers, supporting its ongoing use. The developer has also maintained transparency around survey administration, scoring, confidentiality, and case-mix adjustment, which supports consistent and feasible implementation in real-world settings.
Several areas are appropriately identified as not met but addressable. These include opportunities to strengthen the logic model by more clearly linking provider actions to improvements in patient ratings, expanding the empirical evidence base supporting the measure’s meaningfulness to patients, and providing clearer interpretation of longitudinal performance trends. In addition, further alignment between evidence-based improvement strategies and measure results would enhance usability for providers and health plans.
Overall, the measure remains valuable and fit for continued endorsement. The identified limitations are addressable and do not outweigh the demonstrated utility, widespread adoption, and importance of the measure in capturing the patient’s overall assessment of their provider.
(No subject)
Importance
-
Closing Care Gaps
-
Feasibility Assessment
-
Scientific Acceptability
-
-
Use and Usability
-
Summary
-
No additional comments.
Importance
The patient/provider relationship is a strong indicator of outcomes.
Closing Care Gaps
Optional item not addressed by measure developer.
Feasibility Assessment
Feasible.
Scientific Acceptability
Agree with staff that this can be addressed.
Agree with staff that this can be addressed.
Use and Usability
Useful.
Summary
No additional comments.
Support
Importance
No additional comments.
Closing Care Gaps
Optional item not addressed by measure developer.
Feasibility Assessment
No additional comments.
Scientific Acceptability
Measure developer can address the items noted in staff preliminary assessment.
Measure developer can address the items noted in staff preliminary assessment.
Use and Usability
No additional comments.
Summary
No additional comments.
summary
Importance
.
Closing Care Gaps
.
Feasibility Assessment
.
Scientific Acceptability
.
.
Use and Usability
.
Summary
Disagree with a rating scale of 1-10. All the various criteria that an individual would have to consider in distinguishing between the number ranges would make this survey ineffective. Allow clear options for individuals to choose and only limit to 3 choices- below average, average, above average. Definitions for below/avg/above should also be clearly delineated to reduce ambiguity and subjective nature of the question as those physicians with more outgoing personalities may be higher rated though not providing quality care. The purpose of any surveys like this on individual providers should be for internal use to improve how providers engage with patients.
Public Comments
Verbal Comment Shared During December 10 Listening Session
This is Traci Petrino. I am a director with the Pennsylvania Clinical Network and the ACO. Practices that we have in our network are saying that it seems like the patients are confused when it says “provider.” Could there be more clarification around providers, nurse practitioners? They [patients] think it's just an MD or the DO. Is this an opportunity to maybe change that verbiage a little bit? It [this question] does apply to all of them [the measures] because I think there's a lot of confusion, they think it's just the doctor, and they may be seeing a certified nurse practitioner, so I think it's kind of relevant to all of them.
Response to Traci Petrino
Cognitive testing with patients demonstrated that participants appropriately understood the term “provider.” The term “provider” was intentionally selected to include not only physicians (MDs, Dos) but also nurse practitioners. In addition, the CG-CAHPS Survey instructions further clarify who “this provider” refers to. The first question asks respondents to confirm their specific provider’s name and explains that all subsequent questions will refer to that person as “this provider.” Here is the wording from Question 1:
“1. A health care provider can care for patients in person, by phone, or by video. Our records show that you got care from the provider named below in the last 6 months.
Name of provider label goes here
Is that right? (Yes/No).
The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.”
If vendors or practice sites omit or alter these instructions, clarity may be lost. However, when the survey wording is followed, the term “provider” is defined for respondents and they are responding about the specific individual (MD, DO, NP, etc.) that they got care from.
0005 Consumer Assessment of Healthcare Providers and Systems (CA
The American Medical Association (AMA) appreciates the opportunity to comment on this survey and its associated composite measures. We are extremely concerned over the lack of any performance scores and testing provided for the child survey and composite measures and extrapolation of the adult data to this version. The child survey and composite measures are currently in use, and it is not clear why the developer was unable to access these data for this submission. Without information on the degree to which the child version continues to demonstrate a gap in care and remains reliable and valid, we do not believe that endorsement should be maintained.
In addition, it is our understanding that measures that undergo endorsement maintenance should provide testing at the measure score level for any of the composite measures but none of this information is included in the submission. We believe that it is important for health plans, physicians, patients and caregivers, and others to know how the measures perform, particularly since they are used in many accountability programs and this information has previously been submitted for review. In addition, one of the composite measures for the adult version produced Cronbach’s alpha internal consistency reliability results below 0.7, which is less than desirable, and we do not know if results for the child version would be comparable.
Given the lack of any data and testing for the child version, the omission of testing at the measure score level, and lower Cronbach’s alpha results for some of the composite measures, we do not believe that the minimum endorsement criteria have been met and endorsement should be reconsidered until these concerns are addressed.
Response to AMA
Most of this comment seems to apply to the Child measures rather than the Adult measures. Please refer to our responses to this comment on the Child measures. Regarding the internal consistency reliability falling below 0.70 for one measure, this measure includes only two (2) items. Reliability is influenced by both item correlation and the number of items; a measure with a greater number of items would likely have higher reliability but would also increase respondent burden. In addition, internal consistency reliability is only one of the criteria for determining scientific acceptability, and the measure meets other required criteria, so it is important to examine the totality of the evidence rather than a single metric in isolation.