Skip to main content

Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey (HP CAHPS), Version 5.1

CBE ID
0006
1.0 New or Maintenance
1.1 Measure Structure
1.1a Instrument or Derived Measure
Previous Endorsement Cycle
Is Under Review
No
Next Maintenance Cycle
Fall 2030
E&M Cycle Comments

To view endorsement decisions for each measure derived from this instrument, please refer to the individual measure pages.

1.6 Measure Description

The CAHPS Health Plan (HP CAHPS) Survey is a survey that asks health plan enrollees to report about their care and health plan experiences as well as the quality of care received from physicians. HP CAHPS Version 4.0 was endorsed by NQF in July 2007, and Version 5.0 received maintenance endorsement in January 2015 and was last endorsed in Spring 2019 (CBE #0006). The 5.1 version of the CAHPS Health Plan Survey, released in the fall of 2020, explicitly asks about respondents’ experiences with care received in person, by phone, and by video to account for changes in care due to the pandemic. The survey is part of the CAHPS family of patient experience surveys and is available in the public domain at https://www.ahrq.gov/cahps/surveys-guidance/hp/index.html.

 

The Adult CAHPS Health Plan Survey is designed to be administered to includes individuals 18 years and older who have been enrolled in a health plan and have received care for a specified period (6 months or longer for Medicaid version, 12 months or longer for Commercial version) with no more than one 30-day break in enrollment.  The CAHPS Adult Health Plan Survey has 39 items. Ten (10) of the survey items are used to form 4 composite measures.  The survey also has 4 single-item rating measures. 

 

The Child CAHPS Health Plan Survey is designed to be administered to parents or guardians of children aged 0-17 who have been enrolled in a health plan and have received care for a specified period (6 months or longer for Medicaid version, 12 months or longer for Commercial version) with no more than one 30-day break in enrollment.  The CAHPS Child Health Plan Survey has 41 items.  Eleven (11) of the survey items are used to form 4 composite measures. The survey also has 4 single-item rating measures. 

 

The composite measures are:

  • Getting Needed Care
  • Getting Care Quickly
  • How Well Doctors Communicate
  • Health Plan Customer Service 

The survey also has 4 single-item rating measures: 

  • Rating of Personal Doctor
  • Rating of Specialist
  • Rating of Health Care
  • Rating of Health Plan

The only difference between the Medicaid and commercial versions of the CAHPS Health Plan Survey is the reference period: 6 months for Medicaid enrollees and 12 months for commercial enrollees.

 

A guidance document is available on the AHRQ CAHPS website (https://www.ahrq.gov/cahps/surveys-guidance/hp/index.html) which explains how to field the CAHPS Health Plan Survey and gather the data needed for analysis and reporting. It provides instructions and advice related to the following topics: constructing the sampling frame, choosing the sample, maintaining confidentiality, collecting the data, tracking returned questionnaires, and calculating the response rate. 

 

    Measure Specs
      General Information
      1.8 Level of Analysis
      1.9 Care Setting
      1.9a Rationale for No Applicable Care Setting
      Health Plan level
      1.9b Other Care Setting
      The CAHPS Health Plan Survey asks about experiences with care while enrolled in a health plan for 6 months or longer.
      1.10 Measure Rationale

      The CAHPS Health Plan (HP CAHPS) Survey assesses aspects of health care delivery that are important to patients and for which patients are the best or only source of information (Cleary, Edgman-Levitan, 1997; Cleary, 2016; Solomon et al., 2005).  Further, the HP CAHPS Survey focuses on patient-centered care, which is a key element of health care quality (IOM, 2001). A focus on the patient experience has the potential to enhance clinical outcomes, improve patient safety, and reduce unnecessary medical services. Moreover, assessing patient experience through surveys that include data on the demographic characteristics of respondents, such as race and ethnicity, can help identify the extent to which positive experiences are distributed equitably across patients (Haviland et al., 2003). Use of this measure will benefit both patients and health plans:

      1. Patients can use information from the measures to help make more informed choices about which health plan to use.
      2. Health plans and their providers can use data from the surveys for quality improvement initiatives and incentives.
      3. Researchers can use data files from the surveys to help answer important health services research questions.

      Patient experience encompasses the range of interactions that patients have with the healthcare system. The terms patient satisfaction and patient experience are often used interchangeably, but they are not the same. CAHPS surveys ask patients to report on what they experienced in a healthcare encounter—for example, whether something happened or how often it happened. Patient experience of care surveys provide actionable, objective information for quality improvement. Patient satisfaction surveys, on the other hand, use ratings to measure whether a patient’s expectations about a health encounter were met.

       

      The HP CAHPS Survey is a standardized survey instrument for measuring enrollees’ perspectives on their care. The survey is generally administered annually to patients who have received care in the last 6 months (12 months for Commercial).

       

      References

      Cleary, PD, Edgman-Levitan, S. (1997). Health care quality. Incorporating consumer perspectives. JAMA. 278(19), 1608-12.

       

      Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law, 41 (4), 675-696.

       

      Haviland, M. et al. (2003).  Do health care ratings differ by race or ethnicity? Joint Commission Journal on Quality and Safety. 29(3), 134-145.

       

      Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Accessible at https://nap.nationalacademies.org/catalog/10027/crossing-the-quality-ch….

       

      Solomon, L., Hays, RD., Zaslavsky, A., & Cleary, PD.  (2005). Psychometric properties of the Group-Level Consumer Assessment of Health Plans Study (CAHPS) instrument.  Medical Care, 43, 53-60.

      1.20 Types of Data Sources
      1.20c Format: Patient-Reported Data and/or Survey Data
      Non-digital
      1.25 Data Source Details

      The CAHPS Health Plan Survey (HP CAHPS) Database is a central repository of survey data from State Medicaid agencies, State Children's Health Insurance Programs (CHIP), and individual health plans that have administered the HP CAHPS Survey and chose to submit their data to the Database. The 2024 HP CAHPS Database included 69,505 Adult Medicaid respondents from 233 health plans and 111,833 Child Medicaid respondents from 234 health plans. 

      1.13 Data Dictionary
      Not attached. I attest that all information will be provided where codes and/or value sets are needed (1.14a - 1.15c).
      1.16 Type of Score
      1.17 Measure Score Interpretation
      Better performance = Higher score
      1.18 Calculation of Measure Score

      Respondents report on their experiences accessing and using care, and interacting with their health plans, over the past 6 months (Medicaid) or 12 months (Commercial Health Plans).

       

      AHRQ calculates HP CAHPS Survey measure scores using a top box scoring method. 

       

      Composite Measures: 

      There are two basic steps to calculating a composite measure score for a health plan:

      1. Calculate the proportion of responses in the top box or most positive response category for each question in a composite measure.
      2. Calculate the mean or average top box scores across all questions in a composite measure to determine the composite measure's top box score.

      For the top box or “top proportion” score, the numerator is the number of respondents who answered that they “Always” received the desired care or service for a given measure. For example, if 400 out of 1,000 total respondents answered “Always” to a composite measure item, the top box score for that item would be 40 percent [i.e., (400 ÷ 1,000)*100 = 40%].

       

      Lower proportion and middle proportion composite measure scores can also be calculated following the same methodology where the lower proportion is the proportion answering “Never” or “Sometimes” and the middle proportion is the proportion answering “Usually”. 

       

      Rating Items: 

      For the rating items, the numerator for the top box score is the number of respondents who responded 9 or 10 on the 0-10 scale (where 10 is the “Best” and 0 is the “Worst”). For example, if 600 out of 1,000 total respondents answered “9” or “10” to a rating item, the top box score for that item would be 60 percent [i.e., (600 ÷ 1,000)*100 = 60%].

       

      Lower proportion and middle proportion rating scores can also be calculated where the lower proportion is the proportion answering 0-6 on the 0-10 scale and the middle proportion is the proportion answering 7 or 8. 

       

      Users may also choose to calculate mean scores or linearized mean scores. 

       

      Note the survey includes screener items to identify respondents who meet the target process for each measure, such as whether the individual sought any medical care, saw a personal doctor, saw a specialist, or interacted with the health plan’s customer service. Measures are only calculated using respondents who experienced a particular service/process.

       

      Users can also case-mix adjust the results for characteristics such as respondent age, education, general health status, and mental health status. The CAHPS Analysis Program—often referred to as the CAHPS Macro—is a free program written in SAS (version 6.0 or later) that enables survey users to case-mix adjust their data. The program also generates a distribution of survey results for each of the measures, calculates the mean score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. The results presented in these analyses are based on unadjusted top box scores unless otherwise noted. 

       

      More information about the calculation of proportion scores and mean scores can be found in these documents:

      1. Instructions for Preparing Data for Analysis:  https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
      2. How Results are Calculated:  https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/cahps-database/2…
      3. Instructions for Analyzing Data from CAHPS Survey: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…;

       

      1.21b Attach Data Collection Tool(s)
      1.22 Proxy Responses
      Yes
      1.23 Survey Respondent
      1.24 Data Collection and Response Rate

      Users should choose a data collection protocol that maximizes the survey response rate at an acceptable cost. Some sponsors, as well as researchers conducting field tests, have found that the mail with telephone follow-up method is most effective or email with mail or telephone follow-up. 

       

      AHRQ provides protocols for collecting responses though users can adapt it to meet their needs. The protocols include mail only, telephone only, mail with phone follow-up, or email (web) with mail or phone follow-up.  AHRQ provides detailed instructions for these different protocols in the “Fielding the CAHPS Health Plan Survey” document survey available on the AHRQ CAHPS website:  https://www.ahrq.gov/cahps/surveys-guidance/hp/index.html in the “Guidance for using the CAHPS Health Plan Survey” zip file. 

       

      There is no minimum response rate requirement on the HP CAHPS Survey. The CAHPS consortium has found that higher response rates are achievable if users take steps to ensure the accuracy of the sample frame and carefully follow the recommended data collection protocol, including one or more attempts to follow up with non-respondents. 

       

      In its simplest form, the response rate is the total number of completed questionnaires divided by the total number of individuals selected for the sample. Calculating the response rate is helpful in determining a more accurate starting sample size for future survey administration. For the CAHPS Health Plan Survey, the goal is a response rate of at least 40 percent for Medicaid plans (and/or 300 completed surveys) and 50 percent for commercial plans. 

      To calculate the response rate, use the following formula: Number of completed returned questionnaires divided by the total number of respondents selected minus the sum of deceased + ineligibles. 

       

      AHRQ makes the HP CAHPS Survey available in English and Spanish. 

       

       

      1.26 Minimum Sample Size

      The sample design is based on the units for which users want to compare results, such as health insurance plans or products within health plans. For the purposes of this discussion, “Health insurance plan” is the entity that offers the health insurance (e.g., Plan A), and the “product” is the specific benefit plan design or coverage offered by the plan (e.g., Plan A’s HMO product). Users draw a sample for each health insurance plan or product about which they want to make inferences, separating plans into products, or other groups, such as if there are differences in geography, provider networks, or administrative structure.

       

      The sample that a vendor selects to survey should be drawn from a list of individuals (adults aged 18 and older, or children 17 and younger) covered by the plan or product. This list, which typically would be provided by the sponsor, is the sample frame. 

       

      Defining the Sample Frame: Eligibility Guidelines

       

      Below are the CAHPS guidelines for determining who to include in the sample frame for the commercial survey (Medicaid survey):

      • If surveying adults, include all individuals 18 years or older who have been enrolled in a health plan or product for 6 (12) months or longer, with no more than one 30-day break in enrollment during the 6 (12) months.
      • If surveying children, include all individuals 17 years or younger who have been enrolled in a health plan or product for 12 (6) months or longer, with no more than one 30-day break in enrollment during the 12 (6) months.
      • To identify those who have been enrolled in the plan or product for 12 (6) months or longer, use the anticipated start date of data collection to determine whether the person meets the 12 (6)-month eligibility requirement. For example, if the anticipated start date is March 1, 2026, include all those who have been continuously enrolled since March 1, 2026 (September 1, 2026).
      • Allow the sample frame to include multiple individuals from the same household, but the sample drawn should not have more than one person (adult or child) per household. The final sample must contain only one respondent per household. Where a duplicate household is sampled, it is discarded and replaced by another random draw from the frame.
      • Include individuals with primary health coverage through the plan. Do not include individuals with only other types of coverage, like a dental-only plan.
      • In the case of individuals who switch (or children who are switched) from one product to another within the same plan during the continuous enrollment period, count them as enrolled in the product in which they were enrolled the longest. For example, in the last 6 months, if the individual who was enrolled in a health plan’s HMO product for 4 months switched to the same health plan’s POS product, consider that person continuously enrolled in the health plan’s HMO product.
      • All CAHPS survey items have been designed for the general population. Appropriate screening items are included for items targeted to assess a specific experience. In order to ensure that results are comparable to those produced by other sponsors and vendors, targeted sampling, such as selecting only patients with particular conditions or experiences, is not recommended.  Targeted sampling should only be used to supplement the general population sample, if desired (e.g., adding sample to target children with chronic conditions).

       

      The following section explains how to calculate the appropriate sample size for the HP CAHPS Survey. The instructions are the same for both the Adult and Child versions as well as the Commercial and Medicaid versions.

       

      Calculating the Sample Size for the Adult (Child) Questionnaire

       

      It is recommended that the user select enough individuals to obtain approximately 300 completed adult (child) questionnaires per plan/product. For example, for an anticipated response rate of 50 percent, the user would need to start with a minimum sample size of 600. 

       

      If users anticipate that poor contact information (addresses and telephone numbers) will decrease the number of questionnaires that reach the sampled individuals, a larger sample may be needed. 

       

      If one or more of the plans do not have a membership large enough to draw the required sample size, the sample will be everyone in the health plan enrollee population who meets all of the eligibility criteria. Even under these circumstances, the sample may include only one adult (child) per household.

       

      Sampling information is provided to users as part of the “Fielding the CAHPS Health Plan” survey document available in the “Guidance for using the CAHPS Health Plan Survey” zip file: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…

       

      Data are not reported for any item or measure with fewer than 20 valid responses and health plans with fewer than 20 responses were not included. AHRQ recommends that there needs to be approximately 300 completed questionnaires per plan/product to have a sufficient number of responses for results to be statistically reliable.

       

      Proxy Respondents

       

      The HP CAHPS Survey Plan does allow for proxy respondents for mail and web-based mode. At the end of the survey, there is an item that asks “Did someone help you complete this survey?”  If the answer is Yes, the follow-up question is “How did that person help you?” and they are to mark one or more of these response items:

      1.         Read the questions to me

      2.         Wrote down the answers I gave

      3.         Answered the questions for me

      4.         Translated the questions into my language

      5.         Helped in some other way

       

      However, these the last two questions of the core questionnaire are not included in telephone scripts because telephone interviews should not be conducted with proxy respondents.

      Supplemental Attachment
      Steward Organization
      Agency for Healthcare Research and Quality
      Steward POC email
      Steward Organization Copyright

      CAHPS® is a registered trademark of the U.S. Department of Health and Human Services and managed by AHRQ. 

      Steward Address

      Karen Chaves
      Rockville, MD
      United States

      Measure Developer POC

      Naomi Yount
      Westat
      Rockville, MD
      United States

        Evidence
        2.2 Evidence of Measure Importance

        The HP CAHPS Survey measures key components of patient experience, such as how well doctors communicate and getting needed care, that are consistent with patient-centered care. The CAHPS Surveys focus on aspects of care that consumers have identified as important and for which patients are the best or only source of information. Measuring patients’ perceptions of their healthcare experience is not just a means to improve services—it’s a recognition that the patient’s voice matters in and of itself. Listening to patients affirms their role as active participants in their care, and their insights are essential to truly understanding the quality and impact of healthcare delivery. In 2024, over 200,000 health plan enrollees throughout the country completed the HP CAHPS Survey for their health plan and their health plan submitted this data to the AHRQ CAHPS Database. Since submission to the AHRQ CAHPS Database is not mandatory, it is likely that far more health plans are administering and using these data. Public reporting of these survey results creates incentives for health plans and state agencies to improve their quality of care, directly impacting the patients who receive it. Because of this, it is important to ensure that the survey aligns with what patients believe constitutes high-quality care. We reviewed the literature on the determinants of patient care experiences measured by CAHPS and their associations with other indicators of health care quality. CAHPS is also an actionable measure that helps health plans target interventions that will improve the quality and patient-centeredness of care.

         

        Review of the Evidence

        Prior research has identified several features of healthcare delivery structure, including plan characteristics and market-level characteristics that are associated with patient experiences.  Three major systematic reviews have examined the relationships among patient experience, clinical processes, and patient outcomes. A systematic review performed by researchers in the U.K. found that patient experience is favorably associated with adherence to recommended medications and treatments, preventive care such as screenings and immunizations, patient-reported health outcomes, clinical outcomes, reduced hospitalizations and primary care visits, and reduced adverse events (Doyle et al., 2013). Anhang Price et al. (2014) reviewed evidence on the association between patient experiences and other measures of health care quality in the U.S. They similarly found that better patient care experiences are associated with higher levels of adherence to recommended prevention and treatment processes, better clinical outcomes, and less health care utilization. Navarro et al. (2021) reviewed 9 studies and found that ratings of patient experience is related to the overall rating of health care and can influence clinical and quality outcomes.

         

        Structure

        Health plan type and market characteristics have been found to predict patient experience in several studies. Among managed care organizations (MCOs), for example, Medicaid enrollees had significantly less favorable CAHPS scores than commercial plan enrollees (Elliott, Farley, et al., 2005). For-profit and nationally affiliated health plans tended to receive worse patient experience scores, particularly on overall ratings of the health plan and composite measures on health plan customer service and access (Landon et al., 2021). The financial strength of health plans, as measured by their fiscal margins, was associated with more favorable CAHPS scores (Beauvais et al., 2007). Market-level factors such as HMO competition and penetration did not appear to affect patient experience (Scanlon, Swaminathan, et al., 2008), but Medicare beneficiaries in “higher-intensity” healthcare markets reported more problems getting care quickly than in markets with less healthcare consumption (Mittler, Landon, et al., 2010). For health plans, prior work highlights multi-level impacts on patient experience, which can occur at the system, care site, or physician level – with physicians accounting for the largest proportion of explainable variance (Rodriguez, Scoggins, at al., 2009). Improving the infrastructure supporting certain aspects of care may have broad effects because system changes can influence multiple outcomes (Cleary, 2016).

         

        Quality Improvement / Interventions

        Researchers have used the CAHPS survey to learn if accountable care organization (ACO) incentives to limit health care use and improve quality may enhance or hurt patients’ experiences with care. More specifically, using CAHPS survey data covering 3 years before and 1 year after the start of Medicare ACO contracts in 2012 as well as linked Medicare claims, McWilliams et al. (2014) compared patients’ experiences in a group of 32,334 fee-for-service beneficiaries attributed to ACOs (ACO group) with those in a group of 251,593 beneficiaries attributed to other providers (control group), before and after the start of ACO contracts. They found that, in the first year, ACO contracts were associated with meaningful improvements in some measures of patients’ experience and with unchanged performance in others. Lastly, studies have found that hospitals with more positive perceptions of patient safety culture tend to have more positive CAHPS scores from their patients (Sorra et al., 2012; Abrahamson et al., 2016). This finding suggests that improvements in patient safety culture may lead to improved patient experience with care.

         

        Outcomes

        Out of 40 evidence papers with outcome measures, Doyle’s (2013) meta- analysis found 29 studies that reported positive associations between patient experience and clinical outcomes, 11 with no associations, and none with negative associations. The lack of more evidence may be due to associations between a patient’s illness level, their level of care, and their likelihood for a poor outcome such as mortality, morbidity, or a readmission. Often, such associations have more than one plausible direction of causality. For example, clinicians may be especially attentive to the needs of sicker patients (Kahn et al., 2007) and patients near the end of life (Elliott, Haviland, et al., 2013). 

         

        Moreover, substantial evidence points to a positive association between various components of patient experience, such as good communication between clinicians and patients, and several important processes and outcomes. These include lower utilization of unnecessary healthcare services; better patient adherence to medical advice; better process of care measures for acute myocardial infarction (AMI), congestive heart failure, pneumonia and surgery; lower inpatient mortality among acute myocardial infarction (AMI) patients; lower infection rates (Anhang Price et al., 2014); and better clinician and staff perceptions of patient safety culture (Sorra et al., 2012). 

         

        Schneider and colleagues (2001) found that two HP CAHPS Survey composites were associated with several Healthcare Effectiveness Data and Information Set (HEDIS) clinical process measures among Medicare health plan enrollees. They found that experiences obtaining needed care and getting information and customer service from health plans were associated with mammography, eye examinations for diabetics, receipt of beta-blockers following myocardial infarction, LDL cholesterol testing following an acute cardiovascular event, and follow-up within 30 days following a hospitalization for mental illness (Schneider et al., 2001).

         

        Utilization

        Research suggests an association between better patient experiences and lower healthcare utilization. Platonova and Carnes (2023) found that better results on the measure Provider Communication composite measure led to 19% fewer ER visits for Medicaid patients in North Carolina. The items within the composite measure had strong relationships with ER visits, with provider treating the patient with respect associated with 37% fewer ER visits. 

         

        In another study, children with asthma were less likely to visit the emergency department, make urgent office visits, or be hospitalized if their physicians had reviewed a long-term therapeutic plan with their parents (Clark, Cabana, et al., 2008).  Among African Americans with Type 2 diabetes, those who reported that doctors or nurses usually listened carefully or spent enough time with them were significantly less likely to visit the emergency department in the 12 months following completion of a patient experience survey (Gary, Maiese, et al., 2005). Fenton et al. found that patients who rated their providers most highly had lower odds of visiting the emergency department but higher odds of being admitted to the hospital the following year (Fenton, Jerant, et al., 2012). Children whose parents report longer waits for primary care visits were more likely to visit the emergency department for non-urgent reasons than those who report waiting for less time (Brousseau, Bergholte, et al., 2004).

         

        Health-related Patient Behavior and Disease Management

        One composite of the HP CAHPS survey assesses patients’ perceptions of how well providers communicate with them.  Better patient-provider communication promotes healthcare-related patient behaviors (Fuertes, Boylan, et al., 2009). A 2009 meta-analysis of 127 studies assessing the link between patient treatment adherence and physician-patient communication found a 19% higher risk of non-adherence among patients whose physician communicated poorly (Zolnierek and Dimatteo, 2009). Doyle’s (2013) meta-analysis showed positive associations between the quality of clinician-patient communications and adherence to medical treatment in 125 of 127 studies analyzed. Studies using the CAHPS measure have found that better provider communication is positively associated with adherence to hypoglycemic medications among diabetics (Ratanawongsa, Karter, et al., 2013), adherence to tamoxifen among breast cancer patients (Liu, Malin, et al., 2013), and higher rates of colorectal cancer screening among adults in the US (Carcaise-Edinboro and Bradley, 2008).

         

        References

         

        Abrahamson K, Hass Z, Morgan K, Fulton B, Ramanujam R. (2016). The relationship between nurse-reported safety culture and the patient experience. J Nurs Adm. 46(12):662-668. doi: 10.1097/NNA.0000000000000423. PMID: 27851708.

         

        Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014) Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 71(5), 522-54.

         

        Beauvais, B, Wells, R., Vasey, J., and DelliFraine, J. (2007). Does money really matter? The effects of fiscal margin on quality of care in military treatment facilities. Hosp. Top. 85(3), 2-15.

         

        Brousseau, D. C., Bergholte, J.,et al. (2004). The effect of prior interactions with a primary care provider on nonurgent pediatric emergency department use. Archives of Pediatrics & Adolescent Medicine. 158(1), 78-82.

         

        Carcaise-Edinboro, P. and Bradley. CJ. (2008). Influence of patient-provider communication on colorectal cancer screening. Medical Care. 46(7), 738-745.

         

        Clark, NM., Cabana, MD. et al. (2008). The clinician-patient partnership paradigm: Outcomes associated with physician communication behavior. Clinical Pediatrics. 47(1), 49-57.

         

        Cleary, PD. (2016) Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law. 41(4), 675-696.

         

        Doyle, C., L. Lennox, et al. (2013). A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 3(1). http://bmjopen.bmj.com/content/3/1/e001570.full

         

        Elliott, MN., Farley, D., Hambarsoomians, K., and Hays, R.D. (2005). Do Medicaid and commercial CAHPS scores correlate within plans?: A New Jersey case study. Med Care. 43(10), 1027-1033.

         

        Elliott, MN., Haviland, AM., et al. (2013). Care experiences of managed care Medicare enrollees near the end of life. Journal of the American Geriatrics Society 61(3), 407-412.

         

        Fenton, JJ., Jerant, AF., et al. (2012). The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Archives of Internal Medicine. 172(5), 405-411.

         

        Fuertes, JN., Boylan, LS., et al. (2009). Behavioral indices in medical care outcome: The working alliance, adherence, and related factors. Journal of General Internal Medicine. 24(1), 80-85.

         

        Gary, TL., Maiese, EM., et al. (2005). Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Disease Management. 8(6), 361-371.

         

        Kahn, KL., Tisnado, DM., et al.  (2007).  Does ambulatory process of care predict health-related quality of life outcomes? Health Services Research. 42, 63-83.

         

        Landon, BE., Zaslavsky, AM., Beaulieu, ND., Shaul, JA., Cleary, PD. (2021). Health plan characteristics and consumers’ assessments of quality. Health Affairs, 20(2). 274-286.

         

        Liu, Y., Malin, JL., et al. (2013). Adherence to adjuvant hormone therapy in low-income women with breast cancer: The role of provider-patient communication. Breast Cancer Research and Treatment. 137(3), 829-836.

         

        McWilliams, JM, Landon, BE, Chernew, ME, Zaslavsky, AM. (2014). Changes in patients' experiences in Medicare Accountable Care Organizations. N Engl J Med. 371(18), 1715-24.

         

        Mittler, J., Landon, B., Fisher, E., Cleary, P., and Zaslavsky, A. (2010). Market variations in intensity of Medicare service use and beneficiary experiences with care. Health Serv Res 45(3), 647-669.

         

        Navarro, S., Ochoa, CY., Chan, E., Du, S., Farias, AJ. (2021). Will improvements in patient experience with care impact clinical and quality of care outcomes?: A systematic review. Medical Care. 59(9), 843-856, DOI: 10.1097/MLR.0000000000001598

         

        Platonova, EA, Carnes, KJ. (2023). Relationship between patient-centered primary care provider communication and emergency room visits in the Medicaid population in North Carolina, United States. J Prim Care Community Health. 14, doi: 10.1177/21501319231171430

         

        Ratanawongsa, N., Karter, AJ., et al. (2013). Communication and medication refill adherence: the Diabetes Study of Northern California. JAMA Internal Medicine. 173(3), 210-218.

         

        Rodriguez, HP, Scoggins, JF, von Glahn, T, Zaslavsky, AM, Safran, DG. (2009) Attributing sources of variation in patients' experiences of ambulatory care. Med Care. 47(8), 835-41.

         

        Scanlon, D., Swaminathan, S., Lee, W., and Chernew, M. (2008). Does competition improve health care quality? Health Serv Res. 43(6), 1931-1951.

         

        Schneider, EC, Zaslavsky, AM, et al. (2001). National quality monitoring of Medicare health plans: the relationship between enrollees' reports and the quality of clinical care. Medical Care. 39(12), 1313-1325.

         

        Sorra, J, Khanna, K, Dyer, N, Mardon, R, Famolaro, T. (2012) Exploring relationships between patient safety culture and patients’ assessments of hospital care. Journal of Patient Safety 8(3), 131–139.

         

        Zolnierek, KB. and Dimatteo, MR. (2009). Physician communication and patient adherence to treatment: a meta-analysis. Medical care. 47(8), 826-834.

         

        2.6 Meaningfulness to Target Population

        The family of CAHPS surveys measure aspects of patient-centered care that complement clinical process and outcome measures in consumer choice, quality improvement, public reporting, and pay-for-performance programs (Anhang Price et al, 2014). Published research indicates that individuals use information from patient experience measures to make decisions about their healthcare providers and plans. One study found that seeing publicly reported quality information was a determinant of choosing higher quality-rated health plans, although the weight given to quality information also depended on other features, such as cost and provider choice (Faber et al., 2009). A study of low-income parents in New York State found that parents choose separate CHIP managed care plans with higher CAHPS scores for their newly enrolled children (Liu et al., 2009). Additionally, a study of physician choice found that patients choosing a new primary care physician valued other patients’ reports of interpersonal quality and overall recommendations (Fanjiang et al., 2007).

         

        Patient experiences with health plans are also linked to their persistence in the plans. For example, one study found that the mean voluntary disenrollment rate among Medicare managed care enrollees is four times higher for plans in the lowest 10 percent of overall CAHPS Health Plan survey ratings than for those in the highest 10 percent (Lied et al., 2003). At the provider level, patients who reported the poorest-quality relationships with their physicians are three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships (Safran et al., 2001). 

         

        Racial and ethnic patient subgroups may value various aspects of the care experience differently.  CAHPS surveys have been used to measure these differences. For example, Collins et al. (2017) found that the CAHPS domains with the most importance to respondents varied across subgroups. These researchers conclude that tailoring quality improvement programs to the factors most important to the racial, ethnic, and language mix of the patient population of the health plan may help improve quality.

         

        References

         

        Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014) Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 71 (5): 522-54.

         

        Faber, M, Bosch, M., Wollersheim, H, Leatherman, S, and Grol, R. (2009). Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care. 47(1): 1-8.

         

        Collins, RL, Haas, A, Haviland, AM, Elliott, MN. (2017). What Matters Most to Whom: Racial, Ethnic, and Language Differences in the Health Care Experiences Most Important to Patients. Med Care. 55(11): 940-947.

         

        Fanjiang, G, von Glahn, T, Chang, H, Rogers, W, and Safran, D. (2007). Providing patients web-based data to inform physician choice: if you build it, will they come? J Gen Intern Med. 22(10): 1463-1466.

         

        Lied, TR, Sheingold, SH, Landon, BE, Shaul, JA, Cleary, PD. (2003). Beneficiary reported experience and reported voluntary disenrollment in Medicare managed care. Health Care Finance Rev. 25(1) :55–66.

         

        Liu, H, Phelps, C, Veazie, P, Dick, A, Klein, J, Shone, L, Noyes, K, and Szilagyi, P. (2009). Managed care quality of care and plan choice in New York SCHIP. Health Serv Res. 44(3): 843-861.

         

        Safran, DG, Montgomery, JE, Chang, H, Murphy, J, Rogers, WH. (2001). Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2): 130–6.

            Feasibility
            4.1a Data Structure and Availability

            The HP CAHPS Survey is a standardized instrument designed to assess patient experience of care.  As these patient-experience data are collected from patients, the structured data are not available in electronic sources outside of the data collection by the health plan or sponsoring organization. 

             

            The data are collected through a survey instrument that is administered directly to health plan enrollees, not during care delivery. Surveys are generally mailed to the sampled enrollees, and those survey results can be entered into structured databases (e.g., Excel, SPSS, SAS).  No proprietary platform is required to administer the survey. Though mixed-mode administration (i.e., mail and phone) is a viable strategy for the collection of CAHPS surveys, mail continues to be the most frequent mode for most CAHPS surveys. Users then create electronic databases of results after receipt of the completed hard copy survey through scanning or data entry. However, vendors may set up their database before data collection by populating the frame to assist in identifying nonresponse.

             

            Traditionally, the rationale for not using electronic sources more broadly is that mail and telephone are the best ways to obtain representative samples of patients based on the contact information that is available for sampling and data collection. Web/internet has been added as a mixed mode strategy for health plans for their enrollees. 

             

            Structured or unstructured fields.  All items are structured and on a 4 -item Likert-type response option scale (1-4) or for the rating items on a 0-10 scale.  All responses are numeric. 

             

            Electronic feasibility. HP CAHPS Survey users can offer a web survey to respondents to complete the survey though that option should not be the only option as it may exclude enrollees who have limited or no access to the web and/or who do not have an email address to send an electronic version of the survey.

             

            Missing data. Item level missing data is low on the HP CAHPS Survey however, some items will have fewer response than others due to gate or filter questions.  For example, if a respondent has not seen a personal doctor during the reference period (e.g., 6 months), they are skipped through items about their experiences with doctors. As a result, some CAHPS Health Plan Survey items have higher percentages of missing data overall, but when skip patterns are considered, the percentages of inappropriate missing data are much lower (< 10%). 

             

            Measure susceptibility to inaccuracies and ability to audit data: The HP CAHPS Survey is self-reported perceptions or experiences with the care received and therefore cannot be assessed to determine if the results are accurate.  The procedures for administering HP CAHPS Survey has been standardized by the Centers for Medicare & Medicaid Services (CMS) and NCQA for many years. Because NCQA-accredited health plans are required to submit HP CAHPS Survey results to the NCQA, those plans most often contract with an NCQA-certified survey vendor to accurately collect and report CAHPS survey results.  NCQA requires strict adherence to its standardized procedures and protocols for survey administration and collection. NCQA staff monitors each survey vendor’s work and provides ongoing technical support to survey vendors.  CMS contracts with vendors who are required to adhere to strict standards for survey administration and analysis.  Both the NCQA and CMS place a high value on aligning requirements to assist in streamlining CAHPS measurement for Health Plans and for those plan enrollees who are being surveyed. Further, data submitted to the AHRQ Database are reviewed to ensure there are no out of range values and that skip patterns are followed.

             

            Change to the Instrument.  Since the instrument was last endorsed, there was only one change to the survey and that was to add text to the instrument to allow for respondents to include care that was done virtually (i.e., phone or by video) to account for changes in care delivery due to the COVID-19 pandemic. For example, instructions now included “by phone, or by video”: “These questions ask about your own health care from a clinic, emergency room, or doctor’s office. This includes care you got in person, by phone, or by video. Do not include care you got when you stayed overnight in a hospital. Do not include the times you went for dental care visits.”  This change did not impact data structure or availability. 

             

            4.1b Implementation Costs and Burden

            Data Collection Burden

            For respondents: The survey takes approximately 15 minutes to complete, dependent on the individual.

            For states and health plans: Survey sampling uses administrative enrollment data that is maintained by all health plans and easily accessible to produce a sampling frame. Health plans generally hire a survey vendor to administer, track, and analyze their survey data resulting in lower burden for the health plans. To help reduce burden, states can use data submitted to the AHRQ CAHPS Databases to satisfy requirements for CMS Core Set Reporting (https://www.medicaid.gov/sites/default/files/2024-05/cahpsfactsheet_1.p…). 

             

            Cost Considerations

            The HP CAHPS Survey is freely available for use with no proprietary fees. 

            The cost to hire a vendor varies based on the size of the health plan and desired number of completed surveys. AHRQ provides guidance for hiring a vendor and resources for finding a certified vendor (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/hiring/in…). 

             

            Impact on Clinician Workflow

            The HP CAHPS Survey does not interfere with diagnostic thought processes or patient -physician interactions as the survey is retrospective after care has been given, not during the visit.

             

            Potential Barriers and Mitigation Strategies

            Achieving a desired response rate may be difficult for users. Phone is not optimal as the only mode of survey administration, but it is commonly used as a follow-up for CAHPS mail surveys. Phone follow-up can improve CAHPS response rates compared to mail-only (Burkhart et al., 2014; Fowler et al., 2002; Gallagher et al., 2005; Klein et al., 2011). A study of Medicare beneficiaries found that response rates continue to improve when up to 4 follow-up calls are made (Burkhart et al., 2014). In addition, phone follow-up calls help to achieve better representation of patients in terms of income, literacy/education, health status, age, gender, and race/ethnicity, above and beyond mail surveys alone (Tesler and Sorra, 2017). The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that are hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses. AHRQ also provided a webinar on how to achieve higher response rates (https://www.ahrq.gov/cahps/news-and-events/events/webinar-011124.html). 

             

            Analysis and Reporting: AHRQ makes available many resources to assist with analysis and reporting.  For instance, there is a free CAHPS Analysis Program which is written for SAS that enables survey users to conduct the analyses needed to produce valid comparisons of performance across similar health care organizations. Users can also review documentation on how to prepare data for analysis (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…).  Further, vendors usually conduct all analyses and reports. 

             

             

            References

            Burkhart, Q, Haviland, A, Kallaur, P, et al. (2014). How much do additional mailings and telephone calls contribute to response rates in a survey of Medicare beneficiaries. Field Methods. 27(4): 409-25.

             

            Fowler, FJ, Gallagher, PM, Stringfellow, VL, et al. (2002). Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Med Care. 40(3): 190-200.

             

            Gallagher, PM, Fowler, FJ, Stringfellow, VL. (2005). The nature of nonresponse in a Medicaid survey: causes and consequences. J Off Stat. 21(1) :73-87.

             

            Klein, DJ, Elliott, MN, Haviland, AM, et al. (2011). Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 51(6): 843-55.

             

            Tesler, R. and Sorra, J. CAHPS Survey Administration: What We Know and Potential Research Questions. (Prepared by Westat, Rockville, MD, under Contract No. HHSA 290201300003C). Rockville, MD: Agency for Healthcare Research and Quality: October 2017. AHRQ Publication No. 18-0002-EF. Accessible at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/about-cahps/rese….

            4.1c Confidentiality

            Most vendors have established methods for tracking the sample. The Consortium suggests setting up a system to track the returned surveys by the unique ID number that is assigned to each respondent in the sample. This ID number should be placed on every questionnaire that is mailed and/or on the call record of each telephone case.

             

            To maintain respondent confidentiality, the tracking system should not contain any of the survey responses. The survey responses should be entered in a separate data file linked to the sample file by the unique ID number. (This system will generate the weekly progress reports that should be reviewed closely.) Data should be stored securely—preferably on encrypted or password-protected systems—with access limited. If paper responses are used, they should be shredded following de-identified data entry.

             

            The HP CAHPS Survey data is therefore de-identified upon data collection with a focus on protecting the confidentiality of respondents. Vendors are trained on maintaining confidentiality and any data submitted to the AHRQ CAHPS Database is de-identified. Only the plan name is known, and that is not reported out by AHRQ. Every plan and respondent is assigned a de-identified ID. Results are only reported in aggregate form.  AHRQ does not report any results if there are fewer than 10 respondents and vendors have similar or more stringent rules.

            4.3 Feasibility Informed Final Measure

            The HP CAHPS Survey has a long history of use dating back to 1997. The HP CAHPS Survey has gone through four main revisions since that time, using field and psychometric testing conducted by multiple partners, including NCQA, CMS, and other stakeholders to increase the scientific rigor and relevance of the survey and the usability of the data.  All survey development has been conducted by the CAHPS Consortium, a public-private research collaborative. 

             

            Steps which have contributed to the content and design of the HP CAHPS Survey over time have included:

            • Literature review and review of existing measures
            • Development and consultation with technical expert panels
            • Focus groups with consumers
            • Cognitive testing of survey questions to ensure they will be understood by respondents
            • Field testing to assess the reliability of the survey results
            • Cognitive testing of measure labels to ensure that survey results are communicated clearly to providers and the public
            • Public comment
            • On-going collaboration and harmonization with key partners and stakeholders
            • Input from the NCQA Task Force and review and approval by the NCQA Committee on Performance Measurement to ensure harmonization with NCQA Health Plan accreditation requirements

            The Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that have been hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses.

             

            To address data collection efficiency and to improve response rates, the CAHPS Consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS Consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are in the survey sample. The follow-up to the entire sample is necessary to get a representative set of responses from a practice’s population, as not all patients may have e-mail.

            4.4 Proprietary Information
            Not a proprietary measure and no proprietary components
              Testing Data
              5.1.1 Data Used for Testing

              The data used for these analyses are from the 2024 AHRQ CAHPS Health Plan Survey Database which includes data from Adult and Child Medicaid populations. 

               

              AHRQ launched the development of the CAHPS Health Plan Survey in 1995 and released the first version for public use in 1997. The development included four field tests (Crofton et al., 1999). Over the years, the CAHPS team has conducted multiple field tests of the Health Plan Survey in geographically diverse sites, analyzed the field test data, and revised the instrument as needed based on the findings. 

               

              For evidence of performance gap demonstrating persistent gaps over time, we also include top box statistics on the 2023 HP CAHPS Survey data administered from July 2022 to June 2023. Data were included in the analysis if they had at least one reportable item from the HP CAHPS Survey.

               

              Unless noted otherwise, the top box scores presented are unadjusted since the results are not being used to compare entities, but rather for descriptive and scientific acceptability purposes. 

               

               

              Reference

              Crofton, C., Lubalin, JS., Darby, C. (1999). Foreword. Medical Care 37(3), MS1-MS9.

              5.1.1a Dates of Testing Data

              The surveys were collected between July 2023 and June 2024.

              5.1.2 Differences in Data

              None

              5.1.3 Characteristics of Measured Entities

              Health plan level survey results are calculated across the respondents within a health plan. All health plans submitted Adult Medicaid Version 5.1 (233 plans) and Child Medicaid Version 5.1 (234 plans).  Adult and child plans in this analysis each come from 49 states and the District of Columbia and Puerto Rico, as shown in Table 5.1.3a available in the Supplemental 7.1 zip file.

              5.1.4 Characteristics of Units of the Eligible Population

              A total of 69,505 respondents to the Adult survey and 111,833 respondents to the Child survey (completed by the child’s parent, relative, or legal guardian) are included in the analysis. The Adult survey had an average of 298 respondents per plan, ranging from 20 to 6,462 respondents per plan. The Child survey had an average of 478 respondents per plan, ranging from 23 to 5,234 respondents per plan.

               

              Tables 5.1.4a through g, available in the Supplemental 7.1 zip file, show descriptive characteristics of the respondents by the Adult and Child versions (sex, race/ethnicity, age, self-reported health status, education, survey mode, and survey language). Respondents were predominantly white and non-Hispanic (41%) and older than 54 (43%) for the Adult Survey and respondents were predominantly Hispanic or Latino (35%) for the Child Survey. 36% of respondents in the Adult Survey and 29% in the Child Survey had at least a GED or were a high school graduate. Most respondents completed the survey in English (85% for the Adult Survey and 72% for the Child Survey), and 12% completed it in Spanish for the Adult Survey, while 22% completed it in Spanish for the Child Survey. 

              5.2.1 Reliability Testing Conducted (instrument)
              Person or encounter level (i.e., data element) (e.g., inter–abstractor reliability)
              5.2.2 Method(s) of Reliability Testing

              We estimated internal consistency reliability using the Cronbach’s coefficient alpha for each composite measure. A reliability of at least 0.70 is considered acceptable for group-level comparisons (Nunnally and Bernstein, 1994). For composite measures with more than two items, we show the impact on Cronbach’s alpha of deleting one of the items from the composite measure. However, CAHPS scores are designed to evaluate care across units of care such as plans, or physician groups, not individual patients.

               

              The missing percents for all items were less than 10%. We ran the Cronbach’s alpha excluding all missing data as well as with listwise deletion and the results were the same.  Given the similarity of results, we have presented the Cronbach’s alpha values with the inclusion of cases with missing values (listwise deletion) in section 5.2.3.

               

              Reference

              Nunnally JC, Bernstein IH. (1994).  Psychometric Theory. New York: McGraw Hill.

              5.2.3 Reliability Testing Results

              Tables 5.2.3a1 and 5.2.3a2 (attached in 5.2.3a) show the Cronbach’s alpha for each composite measure in the Adult and Child surveys, respectively. For items within a composite measure consisting of 3 or more items, the Cronbach’s alpha if the item were deleted is provided to determine if there was room for improving coefficient alpha by dropping an item. The table also shows the standardized item to total correlations). 

              5.2.3a Attach Additional Reliability Testing Results
              5.2.4 Interpretation of Reliability Results

              For the Adult survey, two of the four composite measures have alphas higher than 0.70.  Two are below criterion at 0.68 (Health Plan Customer Service) and 0.66 (Getting Needed Care). As shown in Table 5.2.3a, removal of any questions in a composite measure would not result in a higher Cronbach's alpha. Further, all the item to total correlations were above 0.40. 

               

              For the Child survey, one composite measure had an alpha higher than 0.70 (How Well Doctors Communicate).  The other alphas ranged from 0.61 for Getting Needed Care to 0.68 for Health Plan Customer Service. As shown in table 5.2.3a, removal of any questions in a composite measure would not result in a higher Cronbach's alpha since these measures are two item measures.  Further, all the item to total correlations were above 0.40 for all items. 

               

              Cronbach’s alpha can be sensitive to the number of items in a scale, with more items often leading to higher reliability (Nunnally, 1978). All measures with reliability below 0.70 were two-item measures. While Cronbach’s alpha fell below the conventional threshold for several composite measures, it is not the most critical metric in this context. More important is the reliability at the unit level (e.g., plan-level reliability), which better reflects the measure’s utility for quality improvement. Nonetheless, internal consistency remains a relevant consideration in health care, and the Consortium will keep this in mind when implementing future revisions of the instrument. 

               

              Reference

               

              Nunnally, J.C. (1978), Psychometric Theory, 2nd ed. New York: McGraw–Hill

              5.3.1 Validity Testing Conducted (instrument)
              Person or encounter level (i.e., data element) (e.g., sensitivity and specificity)
              5.3.3 Method(s) of Validity Testing

              Several model fit indices were examined to determine how well the hypothesized factor structure, or composite measures, fit the data including chi-square divided by its degrees of freedom (𝝌𝟐/𝒅𝒇) (criteria: values less than 5.0; Schumacker & Lomax, 2004), comparative fit index (CFI) (criteria: values 0.95 or greater; Hu & Bentler, 1999), root mean square error of approximation (RMSEA) (criteria: values less than 0.06; Kline, 2005), and the standardized root mean square residual (SRMR) (criteria: values less than 0.08; Kenny, 2020). 

               

              We examined standardized factor loadings for each item on its respective composite measure. Factor loadings above 0.40 indicate that the item’s relationship to the composite measure is acceptable (Stevens, 2002). 

               

              References

               

              Hu, L., & Bentler, PM. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118

               

              Kenny, DA. (2020, June 5). Measuring model fit. Available at http://davidakenny.net/cm/fit.htm. Accessed October 2025.

               

              Kline, RB. (2005). Principles and practice of structural equation modeling (2nd ed.) New York: The Guilford Press.

               

              Schumacker R., & Lomax, R. (2004). A beginner’s guide to structural equation modeling 

              (2nd ed.). Lawrence Erlbaum.

               

              Stevens, JP. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum.

               

              5.3.4 Validity Testing Results

              Tables 5.3.4a and 5.3.4b (attached in section 5.3.4a), shows results for the model fit indices and standardized factor loadings for both adult and child datasets, respectively. 

              5.3.4a Attach Additional Validity Testing Results
              5.3.5 Interpretation of Validity Results

              The Confirmatory Factor Analysis results for both the Adult and Child HP CAHPS Surveys demonstrate strong model fit based on established criteria. For the Adult survey, the chi-square divided by degrees of freedom (χ²/df) was 12.52, and for the Child survey, it was 8.5, both exceeding the recommended threshold of less than 5.0, which is common in large samples due to the sensitivity of this index. However, the other fit indices all fall well within acceptable ranges: the Comparative Fit Index (CFI) was 0.99 for Adult and 0.98 for Child, surpassing the criterion of 0.95 or greater, indicating excellent model fit. The Root Mean Square Error of Approximation (RMSEA) was 0.05 for both surveys, meeting the standard of less than 0.06, and the Standardized Root Mean Square Residual (SRMR) was 0.03 for both, comfortably below the threshold of 0.08. 

               

              The estimates for each standardized factor loading on the items in the composite measures assess convergent validity. All standardized factor loadings are above 0.5, with the majority above 0.8, and all are statistically significant (p < 0.001), demonstrating the convergent validity of the measures.

               

              These results support the hypothesized factor structure for the measures in the surveys. 

               

                Use
                6.1.1 Current Status
                In use
                6.1.3 Program Details
                Name of the program and sponsor
                Office of Personal Management Federal Employees Health Benefits (FEHB) Health Plan Performance Assessment project.
                Purpose of the program

                OPM assesses the annual performance of health plans contracted under the FEHB program. Each year, FEHB plans send the adult version of the CAHPS® survey to a sample of plan members to evaluate their plan experiences. 

                Geographic area and percentage of accountable entities and patients included

                The FEHB Program provides private health insurance to about 8.3 million federal employees, retirees, and their dependents across the United States. There are approximately 180 health plan choices.

                Applicable level of analysis and care setting

                 Level of analysis – health plans

                Name of the program and sponsor
                NCQA Health Insurance Plan Ranking and Health Plan Accreditation
                Purpose of the program

                The purpose of publishing rankings and the accreditation program is to make quality information on health plans available to consumers and certifies that health plans meet basic requirements for consumer protection and quality improvement (adult and child measures).  

                Geographic area and percentage of accountable entities and patients included

                In 2024, NCQA rated over 1,000 plans across the United States. NCQA lists private (commercial), Medicare, and Medicaid health insurance plans based in part on their CAHPS® scores. Number of patients: Information not available. 

                Applicable level of analysis and care setting

                 Level of analysis – health plans

                Name of the program and sponsor
                CMS Medicare Advantage (MA) and Prescription Drug Plan (PDP) Program
                Purpose of the program

                CMS publicly reports plan-level CAHPS scores for consumers of Medicare Advantage Plans and Part D Prescription Drug Plans. The results from the Medicare CAHPS surveys (Adult version) are published in the Medicare & You handbook each Fall and on the Medicare Web site.

                Geographic area and percentage of accountable entities and patients included

                Approximately 600 health plans across in the United States. Number of patients: Information not available however estimates show approximately 32.8 million patients enrolled in Medicare Advantage. 

                Applicable level of analysis and care setting

                 Level of analysis- state level, health plans 

                Name of the program and sponsor
                Patient Protection and Affordable Care Act – CMS Exchange and Insurance Market Standards/ Quality Rating System
                Purpose of the program

                The Health Insurance Marketplace conducts the Qualified Health Plan (QHP) Survey, a version of the Adult CAHPS Health Plan Survey, to provide consumers with valuable information about health plan quality and help them make informed decisions when selecting a plan. 

                Geographic area and percentage of accountable entities and patients included

                All qualified health plans (QHPs). In 2025 there were 206 QHPs. For 2025, CMS requires that QHP issuers use a HEDIS® Compliance Auditor and follow the HEDIS® Compliance Audit standards to validate the QHP Enrollee Survey sample frame.  Number of patients: Information not available.

                Applicable level of analysis and care setting

                Level of analysis - health plans

                Name of the program and sponsor
                Agency for Healthcare Research and Quality CAHPS Database
                Purpose of the program

                To facilitate comparisons of CAHPS survey results by and among survey sponsors. This compilation of survey results from a large pool of survey users into a national database enables participants to compare their own results to relevant benchmarks. 

                Geographic area and percentage of accountable entities and patients included

                The 2024 CAHPS Database includes data from 233 Adult Medicaid Health Plans, 234 Child Medicaid Health Plans, and 48 Children's Health Insurance Programs (CHIP).  Number of patients:  Information is not available.  

                 

                Applicable level of analysis and care setting

                Level of analysis- individual aggregate and health plan

                Name of the program and sponsor
                CMS Core Measure Reporting
                Purpose of the program

                To promote the objectives of ensuring access to high-quality care through standardized set of measures to assess the quality of care provided to Medicaid and CHIP beneficiaries.

                Geographic area and percentage of accountable entities and patients included

                A subset of data submitted to the 2024 AHRQ CAHPS Database includes data from 42 states for Adult Medicaid, 47 states for Child Medicaid and CHIP.  Number of patients:  Information not available.  

                Applicable level of analysis and care setting

                Level of analysis- state

                6.2.1 Actions of Measured Entities to Improve Performance

                Actions to Improve Patient Experience 

                 

                CAHPS® surveys play an important role as a quality improvement (QI) tool for healthcare organizations that use the standardized data to:

                1. Identify relative strengths and weaknesses in their performance.
                2. Determine where they need to improve.
                3. Track their progress over time.

                 

                AHRQ has made available a CAHPS Ambulatory Care Improvement Guide which is a comprehensive resource for health plans, medical groups, and other providers seeking to improve their performance in the domains of patient experience measured by CAHPS surveys.  AHRQ also has created a short video to help improve patient experience https://www.ahrq.gov/cahps/quality-improvement/index.html#:~:text=CAHPS…;

                 

                The steps are:

                1. Compare CAHPS survey scores to other health care organizations to determine how the plan is doing in comparison to others.
                2. Examine how CAHPS scores are changing over time.
                3. Identify priorities based on these comparisons
                4. Confirm these priorities based on other sources of information (e.g., patient complaints, patient comments)
                5. Find out what is actually happening with patients and why.
                6. Brainstorm with staff to determine the best strategies for improvement. 

                In addition, AHRQ held a research meeting in 2020 to discuss how to improve patient experience and provided summaries of the presentations: https://www.ahrq.gov/cahps/news-and-events/events/2020-meeting-summary….

                 

                Difficulty in Increasing Response Rates

                 

                Users are also provided advice for improving response rates (AHRQ, 2008):

                1. Improve initial contact rates by making sure that addresses and phone numbers are current and accurate (e.g., identify sources of up-to-date sample information, run a sample file through a national change-of-address database, send a sample to a phone number look-up vendor).
                2. Use all available tracking methods (e.g., Lexis-Nexis, Internet database services and directories).
                3. Improve contact rates after data collection has begun (e.g., increase maximum number of calls, ensure that calls take place at different day and evening times over a period of days, mail second reminders, use experienced and well-trained interviewers).
                4. Consider using a mixed-mode protocol. In field tests, the combined approach was more likely to achieve a desired response rate than did one mode alone.
                5. Train interviewers on how to deal with gatekeepers.
                6. Train interviewers on refusal aversion/conversion techniques.
                6.2.2 Feedback on Measure Performance

                As part of CAHPS development and maintenance, the CAHPS Consortium has sought input from multiple users, including accreditors, health plans, and the public. Throughout the development process, the CAHPS Consortium has incorporated the data or input from these various sources in an incremental process of revision and refinement to develop measurement that is more precise and to produce survey data that would better meet the information needs of consumers and other stakeholders. The CAHPS Consortium hears user feedback during research studies and development. Users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. The CAHPS consortium also solicits feedback via focus groups with patients in developing survey content and design. We are not aware of any substantial problems experienced by health plans or respondents.

                6.2.3 Consideration of Measure Feedback

                The current 5.1 version of the instrument includes revisions based on feedback received since the development of the HP CAHPS Survey.  A few examples of feedback used to revise the instrument include changes made when creating version 5.0 which incorporated some minor changes into the wording of core items based on input gathered in consultation with stakeholders. For example, questions about access to urgent and non-urgent appointments were modified to ask respondents if they were able to get an appointment “as soon as they needed,” rather than as soon as “they thought” they needed for consistency across all CAHPS surveys. Similarly, the item about how often it was easy to get care was moved from the “Your Health Plan” section to the “Your Health Care” section because respondent feedback was that they had difficulty attributing this item to the health plan. For the 5.1 version, with the COVID-19 pandemic changing how some health care was delivered (e.g., video or phone rather than in-person), the instrument was updated again to change instructions and gate question wording to include these types of visits.  

                6.2.5 Unexpected Findings

                No unexpected findings.

                  Public Comments