The measure calculates the percentage of family or designated responsible party for long stay residents (i.e., residents living in the facility for 100 days or more), who are satisfied. This consumer reported outcome measure is based on the CoreQ: Long-Stay Family questionnaire that has three items.
Measure Specs
General Information
Collecting satisfaction information from skilled nursing facility (SNF) patients is more important now than ever. We have seen a philosophical change in healthcare that now includes the patient and their preferences as an integral part of the system of care. The Institute of Medicine (IOM) endorses this change by putting the patient as central to the care system (IOM, 2001). For this philosophical change to person-centered care to succeed, we have to be able to measure patient satisfaction for these three reasons:
(1) Measuring satisfaction is necessary to understand patient preferences.
(2) Measuring and reporting satisfaction with care helps patients and their families choose and trust a health care facility.
(3) Satisfaction information can help facilities improve the quality of care they provide.
The implementation of person-centered care in SNFs has already begun, but there is still room for improvement. The Centers for Medicare and Medicaid Services (CMS) demonstrated interest in consumers’ perspective on quality of care by supporting the development of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey for patients in nursing facilities (Sangl et al., 2007).
Further supporting person-centered care and resident satisfaction are ongoing organizational change initiatives. These include: the Advancing Excellence in America’s Nursing Homes campaign (2006), which lists person-centered care as one of its goals; Action Pact, Inc., which provides workshops and consultations with nursing facilities on how to be more person-centered through their physical environment and organizational structure; and Eden Alternative, which uses education, consultation, and outreach to further person-centered care in nursing facilities. All of these initiatives have identified the measurement of resident satisfaction as an essential part in making, evaluating, and sustaining effective clinical and organizational changes that ultimately result in a person- centered philosophy of care.
The importance of measuring resident satisfaction as part of quality improvement cannot be stressed enough. Quality improvement initiatives, such as total quality management (TQM) and continuous quality improvement (CQI), emphasize meeting or exceeding “customer” expectations. William Deming, one of the first proponents of quality improvement, noted that “one of the five hallmarks of a quality organization is knowing your customer’s needs and expectations and working to meet or exceed them” (Deming, 1986). Measuring resident satisfaction can help organizations identify deficiencies that other quality metrics may struggle to identify, such as communication between a patient and the provider.
As part of the U.S. Department of Commerce renowned Baldrige Criteria for organizational excellence, applicants are assessed on their ability to describe the links between their mission, key customers, and strategic position. Applicants are also required to show evidence of successful improvements resulting from their performance improvement system. An essential component of this process is the measurement of customer, or resident, satisfaction (Shook & Chenoweth, 2012). Bhattacharyya et al. (2022) note that “satisfaction is an integral part of nursing home (NH) quality of care.”
The CoreQ: Long-Stay Family questionnaire can strategically help nursing facilities achieve organizational excellence and provide high quality care by being a tool that targets a unique and growing patient population. Over the past several decades, care in nursing facilities has changed substantially. Statistics show that more than half of all elders cared for in nursing homes are now discharged home (approximately 1.6 million residents; CMS, 2009). Moreover, when satisfaction information from current residents (i.e., long stay residents) is compared with those of elders discharged home, substantial differences exist (Castle, 2007). This indicates that long stay and short stay residents are different populations with different needs in the nursing facilities. Thus, the CoreQ: Long-Stay Family questionnaire measure is needed to improve the care for long stay SNF patients.
Furthermore, improving the care for long stay nursing home patients is tenable. A review of the literature on satisfaction surveys in nursing facilities (Castle, 2007) concluded that substantial improvements in resident satisfaction could be made in many nursing facilities by improving care (i.e., changing either structural or process aspects of care). This was based on satisfaction scores ranging from 60 to 80% on average.
It is worth noting, few other generalizations could be made because existing instruments used to collect satisfaction information are not standardized. Thus, benchmarking scores and comparison scores (i.e., best in class) were difficult to establish. The CoreQ: Long-Stay Family measure has considerable relevance in establishing benchmarking scores and comparison scores.
This measure’s relevance is furthered by recent federal legislative actions. The Affordable Care Act of 2010 requires the Secretary of Health and Human Services (HHS) to implement a Quality Assurance & Performance Improvement Program (QAPI) within nursing facilities. This means all nursing facilities have increased accountability for continuous quality improvement efforts. In CMS’s “QAPI at a Glance” document there are references to customer-satisfaction surveys and organizations utilizing them to identify opportunities for improvement. Lastly, the new “Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities” proposed rule includes language purporting the importance of satisfaction and measuring satisfaction. CMS states “CMS is committed to strengthening and modernizing the nation’s health care system to provide access to high quality care and improved health at lower cost. This includes improving the patient experience of care, both quality and satisfaction, improving the health of populations, and reducing the per capita cost of health care.” There are also other references in the proposed rule speaking to improving resident satisfaction and increasing person-centered care (Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities, 2015). The CoreQ: Long-Stay Family measure has considerable applicability to both of these initiatives.
References
Bhattacharyya KK, Molinari V, Hyer K. Self-Reported Satisfaction of Older Adult Residents in Nursing Homes: Development of a Conceptual Framework. Gerontologist. 2022 Sep 7;62(8):e442-e456. doi:10.1093/geront/gnab061. PMID: 33979428.
Castle, N.G. (2007). A literature review of satisfaction instruments used in long-term care settings. Journal of Aging and Social Policy, 19(2), 9-42.
CMS (2009). Skilled Nursing Facilities Non Swing Bed - Medicare National Summary. http://www.cms.hhs.gov/MedicareFeeforSvcPartsAB/Downloads/NationalSum20…
CMS, University of Minnesota, and Stratis Health. QAPI at a Glance: A step by step guide to implementing quality assurance and performance improvement (QAPI) in your nursing home. https://www.cms.gov/Medicare/Provider-Enrollment-and- Certification/QAPI/Downloads/QAPIAtaGlance.pdf.
Deming, W.E. (1986). Out of the crisis. Cambridge, MA. Massachusetts Institute of Technology, Center for Advanced Engineering Study.
Institute of Medicine (2001). Improving the Quality of Long Term Care, National Academy Press, Washington, D.C., 2001. Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities; Department of Health and Human Services. 80 Fed. Reg. 136 (July 16, 2015) (to be codified at 42 CFR Parts 405, 431, 447, et al.).
MedPAC. (2015). Report to the Congress: Medicare Payment Policy. http://www.medpac.gov/documents/reports/mar2015_entirereport_revised.pd….
Sangl, J., Bernard, S., Buchanan, J., Keller, S., Mitchell, N., Castle, N.G., Cosenza, C., Brown, J., Sekscenski, E., and Larwood, D. (2007). The development of a CAHPS instrument for nursing home residents. Journal of Aging and Social Policy, 19(2), 63-82.
Shook, J., & Chenoweth, J. (2012, October). 100 Top Hospitals CEO Insights: Adoption Rates of Select Baldrige Award s and Processes. Truven Health Analytics. http://www.nist.gov/baldrige/upload/100-Top-Hosp-CEO-Insights-RB-final….
The collection instrument is the CoreQ: Long-Stay Family Satisfaction Questionnaire. The resident information comes from facility health information systems used by all SNFs such as billing systems. The exclusions are from the facility health information systems that all facilities have in place such as MDS.
Numerator
The numerator assesses the number of family or designated responsible party for long stay residents that are satisfied. Specifically, the numerator is the sum of the family or designated party responsible for long stay residents that have an average satisfaction score of =>3 for the three questions on the CoreQ: Long-Stay Family questionnaire.
The numerator includes all of the family or designated responsible party members for long stay residents that had an average response =>3 on the CoreQ: Long-Stay Family questionnaire.
We calculate the average satisfaction score for the individual family or designated responsible party member for long stay residents in the following manner:
- Respondents within the appropriate time window and who do not meet the exclusions are identified.
- A numeric score is associated with each response scale option on the CoreQ: Long-Stay Family questionnaire (that is, Poor=1, Average=2, Good=3, Very Good=4, and Excellent=5).
- The following formula is utilized to calculate the individual’s average satisfaction score: [Numeric Score Question 1 + Numeric Score Question 2 + Numeric Score Question 3]/3
- The number of respondents whose average satisfaction score >=3 are summed together and function as the numerator.
For respondents with one missing data point (from the 3 items included in the questionnaire) imputation will be used (representing the average value from the other two available questions). For respondents with more than one missing data point, they will be excluded from the analyses (i.e., no imputation will be used for these family members). Imputation details are described further below.
No risk-adjustment is used.
Denominator
The target population is family or designated responsible party members of a resident residing in a SNF for at least 100 days. The denominator includes all of the individuals in the target population who respond to the CoreQ: Long-Stay Family questionnaire within the two-month time window who do not meet the exclusion criteria.
The denominator includes all of the family or the designated responsible party members for residents that have been in the SNF for 100 days or more regardless of payer status; who received the CoreQ: Long-Stay Family questionnaire (e.g. people meeting exclusions do not receive the questionnaire), and who responded to the questionnaire within the two month time window.
The length-of-stay (of the resident of the family member or designated responsible party) will be identified from MDS nursing facility records (MDS item A1600 “Entry Date”).
Exclusions
The resident representative for each current resident is initially eligible regardless of their being a family member or not. Only one primary contact per resident should be selected.
Exclusions made at the time of sample selection are the following: (1) Residents who have poor cognition defined by the BIMS score; (2) residents receiving hospice; (3) residents with a legal court appointed guardian; and (4) residents who have lived in the SNF for less than 100 days.
Additionally, once the survey is administered, the following exclusions are applied: a) surveys received outside of the time window (two months after the administration date) b) surveys that have more than one questionnaire item missing c) surveys from residents who indicate that someone else answered the questions for the resident. (Note this does not include cases where the resident solely had help such as reading the questions or writing down their responses.)
Individuals are excluded based on information from the Minimum Data Set (MDS) 3.0 assessment. Representatives of residents with the following criteria will be excluded:
- Residents who have poor cognition: Then the Brief Interview for Mental Status (BIMS), a well validated dementia assessment tool is used. BIMS ranges are 0-7 (lowest); 8-12; and 13-15 (highest). Residents with a BIMS score of equal or less than 7 are excluded. (MDS Section C0200-C0500 items are used) (Saliba, et al., 2012).
- Patients receiving or having received any hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”).
- Patients with a court appointed legal guardian for all decisions will be identified from nursing facility health information system.
- Residents who have lived in the SNF for less than 100 days will be identified from the MDS. This is recorded in the MDS (Section A1600, Entry Date).
- Residents that respond after the 2-month response period.
- Residents whose responses were completed by someone other than the resident will be excluded. Identified from an additional question on the CoreQ: Long-Stay Resident questionnaire.
- Residents without usable data (defined as missing data for 2 or 3 of the survey questions).
Reference
Saliba D, Buchanan J, Edelen MO, Streim J, Ouslander J, Berlowitz D, Chodosh J. J Am Med Dir Assoc. 2012 Sep;13(7):611-7. doi: 10.1016/j.jamda.2012.06.004. Epub 2012 Jul 15.
Measure Calculation
1. Identify the representatives of residents that have been residing in the SNF for 100 days or more. Length of stay so far is the MDS target date (TRGT_DT) - MDS admission date (A1900).
2. Take the representatives of residents that have been residing in the SNF for >=100 days and exclude the following:
a. Representatives of residents on hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”).
b. Residents with Court appointed legal guardians for all decisions as identified from nursing facility health information system.
3. Exclude representatives of residents who reside in another country.
4. Administer the CoreQ: Long-Stay Family questionnaire to the representatives that do not meet these exclusion criteria.
Provide the family or designated responsible party member for the resident two months to respond to the survey.
a. Create a tracking sheet with the following columns:
i. Date Administered
ii. Date Response Received
iii. Time to Receive Response: ([Date Response Received – Date Administered])
b. Exclude any surveys where Time to Receive Response >60 days (2 months)
5.Combine the CoreQ: Long-Stay Family questionnaire items to calculate a resident’s representative satisfaction score. Responses for each item should be given the following scores:
a. Poor = 1,
b. Average = 2,
c. Good = 3,
d. Very good =4 and
e. Excellent = 5.
6.Impute missing data if only one of the three questions are missing data. Drop all survey response if 2 or more survey questions have missing data.
7.Calculate resident’s representative score from usable surveys.
a. Representative average score = (Score for Item 1 + Score for Item 2 + Score for Item 3) / 3.
b. Flag those representatives with a score equal to or greater than 3.0
I. For example, a representative of a resident rates their satisfaction on the three CoreQ questions as excellent = 5, very good = 4, and good = 3. The family member’s total score will be 5 + 4 + 3 for a total of 12. The representative of the long-stay resident total score (12) will then be divided by the number of questions (3), which equals 4.0. Thus, the representative’s average satisfaction rating is 4.0. Since this person’s average response is >3.0 they would be counted in the numerator. If it was <3.0 they would not be counted.
8.Calculate the facility’s CoreQ: Long-Stay Family Measure which represents the percent of respondents with average scores of 3.0 or above.
a. CoreQ: Long-Stay Family Measure = ([number of respondents with an average score of =3.0] / [total number of valid responses]) *100
9.No risk-adjustment is used.
Reference
Saliba, D., Buchanan, J., Edelen, M.O., Streim, J., Ouslander, J., Berlowitz, D, & Chodosh J. (2012). MDS 3.0: brief interview for mental status. Journal of the American Medical Directors Association, 13(7): 611-617.
The measure is not stratified.
1. Identify the representatives of residents that have been residing in the SNF for 100 days or more. This will be identified from MDS target date (TRGT_DT) - MDS admission date (A1900).
2.Take the representatives of residents that have been residing in the SNF for >=100 days and exclude the following:
a. Representatives of residents on hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”).
b. Residents with Court appointed legal guardian for all decisions as identified from nursing facility health information system.
3. Exclude representatives of residents who reside in another country.
4. Administer the CoreQ: Long-Stay Family questionnaire to family or designated responsible party members for long-stay residents.
5. Instruct representatives that they must respond to the survey within 2 months.
6. The response rate for a center is calculated by counting the number of usable surveys returned divided by the number of surveys administered.
a. Surveys returned as undeliverable are not counted as usable.
b. Surveys with missing responses for two or more questions are also not counted as usable.
c. A minimum response rate of 30% needs to be achieved for results to be reported for a SNF.
7. Regardless of response rate, SNFs must also achieve a minimum number of 20 usable questionnaires (e.g. denominator). If after 2 months, less than 20 usable questionnaires are received than a facility level satisfaction measure cannot be reported.
8. All the questionnaires that are received (other than those that satisfy the exclusion criteria seen in must be used in the calculations.
Reference
Saliba, D., Buchanan, J., Edelen, M.O., Streim, J., Ouslander, J., Berlowitz, D, & Chodosh J. (2012). MDS 3.0: brief interview for mental status. Journal of the American Medical Directors Association, 13(7): 611-617.
A minimum sample size of 20 and overall response rate of 30% is needed for the measure.
Supplemental Attachment
Measure Record
Point of Contact
Not applicable
Valerie Brandon
Washington, DC
United States
Nicholas Castle
University of West Virginia
Morgantown, WV
United States
Importance
Evidence
The definition of quality in a nursing facility has shifted from a focus on structure and process criteria to clinical outcomes, resident satisfaction, and quality of life. This shift was first supported by nursing home reform legislation included in the Omnibus Budget Reconciliation Act of 1987 (OBRA, 1987). Furthering the movement, the Institute of Medicine (IOM) put the patient as central to the care system (Castle, 2007; IOM, 2001) – necessitating the collection of satisfaction information. As mentioned previously (see 1b.1), a focus on person-centered care and satisfaction is also evident in the Quality Assurance & Performance Improvement Program (QAPI) for nursing facilities and proposed Reform Requirements for Long-Term Care Facilities (Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities, 2015).
Measuring and reporting satisfaction of nursing home care is important in many ways. First, residents are more likely to follow medical advice when they rate their care as satisfactory (Hall, Milburn, Roter, & Daltroy, 1998). Second, because resident satisfaction can influence the quality of care provided and the outcomes of treatment (Hudak and Wright 2000), satisfaction surveys can be used as measures of clinical and organizational accountability. Third, measuring and reporting resident satisfaction can help nursing facilities identify and improve aspects of quality. Furthermore, if publicly released, information on satisfaction with care can help elders and their families choose a nursing facility.
Several research efforts have concluded consumer satisfaction is an important indicator of quality of care in nursing homes (Gesell, 2001; Bangerter et al. 2016; Shippee et al 2015; Kajonius and Kazemi, 2016). In addition, other studies have concluded nursing resident satisfaction data provides information about quality of care that is different from clinician perspectives and clinical indicators (Berlowitz, Du, Kazis, & Lewis, 1993; Riccio 2000; Uman & Urman, 1997). This exemplifies the need for resident satisfaction data to achieve person-centered care. Only by hearing from the patient can we ensure the care provided is person-centered.
Researchers have recently studied the association of satisfaction scores and the star rating system. The resident’s perspective is imperative in selecting a nursing home where they intend to spend their lives. Nursing homes with higher satisfaction scores tend to have higher star ratings thus proving that the nursing home provides higher quality care (Kwon and Bowblis, 2024). The data shows that for every 1%-point increase in resident satisfaction score, the probability of being a 4- or 5-star nursing home increases by 0.7% points across all star ratings (Kwon and Bowblis, 2024). The addition of a publicly reported satisfaction measure in the star rating system or in Care Compare allows current and future consumers of nursing homes with the knowledge to make informed decisions about their long-term care. Bhattacharyya et al. (2022) note that “satisfaction is an integral part of nursing home (NH) quality of care.”
References
Bangerter, L.R., Heid, A.R., Abbott, K, & Van Haitsma, K. (2016). Honoring the Everyday Preferences of Nursing Home Residents: Perceived Choice and Satisfaction with Care. The Gerontologist. (Advance online publication): 1-8.
Berlowitz, D. R., Du, W., Kazis, L., & Lewis, S. (1995). Health-related quality of life of nursing home residents: Difference in patient and provider perceptions. Journal of the American Geriatric Society, 43, 799-802.
Bhattacharyya KK, Molinari V, Hyer K. Self-Reported Satisfaction of Older Adult Residents in Nursing Homes: Development of a Conceptual Framework. Gerontologist. 2022 Sep 7;62(8):e442-e456. doi: 10.1093/geront/gnab061. PMID: 33979428.
Castle, N.G. (2007). A literature review of satisfaction instruments used in long-term care settings. Journal of Aging and Social Policy, 19(2), 9-42.
CMS, University of Minnesota, and Stratis Health. QAPI at a Glance: A step by step guide to implementing quality assurance and performance improvement (QAPI) in your nursing home. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI…;
Gesell, S.B. (2001). A measure of satisfaction for the assisted-living industry. Journal for Healthcare Quality, 23(2), 16-25.
Hall J, Milburn M, Roter D, Daltroy L. Why are sicker patients less satisfied with their medical care? Tests of two explanatory models. Health Psychol. 1998;17(1):70–75.
Hudak, P. L. & J.G. Wright. (2000). The Characteristics of Patient Satisfaction Measures. Spine 25 (24): 3167-3177.
Institute of Medicine (2001). Improving the Quality of Long-Term Care, National Academy Press, Washington, D.C., 2001.
Kajonius, P. & Kazemi, A. (2016). Advancing the Big Five of user-oriented care and accounting for its variations. International Journal of Health Care Quality Assurance. 29(2): 162 – 176.
Kusmaul, N., Miller, R.J., Diehl, C., & Stockwell, I.A. (2024). A facility-level analysis of nursing home compare five star rating and Maryland’s family satisfaction with care survey. The Gerontologist, 64.
Kwon, J. H., & Bowblis, J. R. (2024). Association Between Nursing Home Five-Star Ratings and Consumer Satisfaction. Journal of the American Medical Directors Association, 25(12), 105322. https://doi.org/10.1016/j.jamda.2024.105322
Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities; Department of Health and Human Services. 80 Fed. Reg. 136 (July 16, 2015) (to be codified at 42 CFR Parts 405, 431, 447, et al.).
Omnibus Budget Reconciliation Act (OBRA) of 1987. (1987, December 22). Public Law 100-203. Subtitle C: Nursing Home Reform.
Riccio, P.A. (2000). Quality Evaluaiton of home nursing care: Perceptions of patients, physicians, and nurses. Nursing Administration Quarterly 24(3): 43-52.
Shippee, T.P., Henning-Smith, C., Kane, R.L, & Lewis, T. (2015). Resident- and Facility-Level Predictors of Quality of Life in Long-Term Care. The Gerontologist. 55(4):643-655.
Uman, C & Urman, H. (1997). Measuring consumer satisfaction in nursing home residents. Nutrition 13: 705-707.
Measure Impact
The consumer movement has fostered the notion that patient evaluations should be an integral component of health care. Patient satisfaction, which is one form of patient evaluation, became an essential outcome of health care widely advocated for use by researchers and policy makers. Managed care organizations, accreditation and certification agencies, and advocates of quality improvement initiatives, among others, now promote the use of satisfaction surveys. For example, satisfaction information is included in the Health Plan Employer Data Information Set (HEDIS), which is used as a report card for managed care organizations (NCQA, 2016).
Measuring and improving patient satisfaction is valuable to patients, because it is a way forward on improving the patient-provider relationship, which influences health care outcomes. A 2014 systematic review and meta-analysis of randomized controlled trials, in which the patient-provider relationship was systematically manipulated and tracked with health care outcomes, found a small but statistically significant positive effect of the patient-provider relationship on health care outcomes (Kelly et al., 2014). This finding aligns with other studies that show a link between patient satisfaction and the following health-related behaviors:
1. Keeping follow-up appointments (Hall, Milburn, Roter, & Daltroy, 1998);
2. Disenrollment from health plans (Allen & Rogers, 1997); and,
3. Litigation against providers (Penchansky & Macnee, 1994).
The positive effect of person-centered care and patient satisfaction is not precluded from skilled nursing facilities. A 2013 systematic review of studies on the effect of person-centered initiatives in nursing facilities, such as the Eden Alternative, found person-centered care associated with psychosocial benefits to residents and staff, notwithstanding variations and limitations in study designs (Brownie & Nancarrow, 2013).
As person-centered care (PCC) continues to evolve in long-term care, states are creating programs geared towards training and educating nursing home staff about PCC. The Promoting Excellence Alternatives in Kansas (Peak) 2.0 program incentivizes nursing homes in Kansas to implement PCC by empowering staff to ensure that residents have a choice about their care (Poey et.al, 2017). Resident satisfaction is a great way to determine the effectiveness of PCC but also, it’s a way to gain knowledge about nursing home quality. The program found the nursing homes that fully implemented PCC reports higher resident satisfaction. Programs that incorporate PCC benefits the residents but also have benefits for the nursing home by showing that it is a reputable location for older adults to age in place.
From the nursing facility and provider perspective, there are numerous ways to improve patient satisfaction. One study found conversations regarding end-of-life care options with family members improve overall satisfaction with care and increase use of advance directives (Reinhardt et al., 2014). Another found an association between improving symptom management of nursing home residents with dementia and higher satisfaction with care (Van Uden et al., 2013). Improvements in a nursing home food delivery system also were associated with higher overall satisfaction and improved resident health (Crogan et al., 2013). The advantage of the CoreQ: Long-Stay Family questionnaire is it is broad enough to capture patient dissatisfaction on various provided services and signal to providers to drill down and discover ways of improving the patient experience at their facility.
Specific to the CoreQ: Long-Stay Family questionnaire, the importance of the satisfaction areas assessed were examined with focus groups of residents and family members. The respondents were patients (N=40) in five nursing facilities in the Pittsburgh region. Table 1c.5 shows the score of the importance for question included in the CoreQ: Long-Stay Family questionnaire (See Section 7. Supplemental Information, Table 1c.5). The overall ranking used was 10=Most important and 1=Least important. The final four questions included in the measure had average scores ranging from 9.35 to 9.69; this clearly shows that the respondents value the items used in the CoreQ: Long-Stay Family measure.
Other more recent research publications have continued to show the importance of satisfaction and the strong link between satisfaction and patient outcomes (Kellogg et al., 2018, Kwon & Bowbli, 2024, Plaku-Alakbarova et al., 2019).
References
Allen HM, & Rogers WH. (1997). The Consumer Health Plan Value Survey: Round Two. Health Affairs. 1997;16(4):156–66.
Brownie, S. & Nancarrow, S. (2013). Effects of person-centered care on residents and staff in aged-care facilities: a systematic review. Clinical Interventions In Aging. 8:1-10.
Crogan, N.L., Dupler, A.E., Short, R., & Heaton, G. (2013). Food choice can improve nursing home resident meal service satisfaction and nutritional status. Journal of Gerontological Nursing. 39(5):38-45.
Hall J, Milburn M, Roter D, Daltroy L (1998). Why are sicker patients less satisfied with their medical care? Tests of two explanatory models. Health Psychol. 17(1):70–75.
Kelley J.M., Kraft-Todd G, Schapira L, Kossowsky J, & Riess H. (2014). The influence of the patient-clinician relationship on healthcare outcomes: a systematic review and metaanalysis of randomized controlled trials. PLoS One. 9(4): e94207.
Kellogg, C., Zhu, Y., Cardenas, V., Vazquez, K., Johari, K., Rahman, A., & Enguidanos, S. (2018). What consumers say about nursing homes in online reviews. The Gerontologist, 58(4), e273–e280.
Kwon, Jenny H. et al. (2024). Association Between Nursing Home Five-Star Ratings and Journal of the American Medical Directors Association, Volume 25, Issue 12.
Li, Y., Cai, X., Ye, Z., Glance, L.G., Harrington, C., & Mukamel, D.B. (2013). Satisfaction with Massachusetts nursing home care was generally high during 2005-09, with some variability across facilities. Health Affairs. 32(8):1416-25.
Lin, J., Hsiao, C.T., Glen, R., Pai, J.Y., & Zeng, S.H. (2014). Perceived service quality, perceived value, overall satisfaction and happiness of outlook for long-term care institution residents. Health Expectations. 17(3):311-20.
National Committee for Quality Assurance (NCQA) (2016). HEDIS Measures. http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures.aspx. Accessed March 2016.
Penchansky and Macnee, (1994). Initiation of medical malpractice suits: a conceptualization and test. Medical Care. 32(8): pp. 813–831.
Plaku-Alakbarova, B., Punnett, L., & Gore, R. J.; Procare Research Team. (2018). Nursing home employee and resident satisfaction and resident care outcomes. Safety and Health at Work, 9(4), 408–415.
Poey, J. L., Hermer, L., Cornelison, L., Kaup, M. L., Drake, P., Stone, R. I., & Doll, G. (2017). Does Person-Centered Care Improve Residents' Satisfaction With Nursing Home Quality?. Journal of the American Medical Directors Association, 18(11), 974–979. https://doi.org/10.1016/j.jamda.2017.06.007.
Reinhardt, J.P., Chichin, E., Posner, L., & Kassabian, S. (2014). Vital conversations with family in the nursing home: preparation for end-stage dementia care. Journal Of Social Work In End-Of-Life & Palliative Care. 10(2):112-26.
Van Uden, N., Van den Block, L., van der Steen, J.T., Onwuteaka-Philipsen, B.D., Vandervoort, A., Vander Stichele, R., & Deliens, L. (2013). Quality of dying of nursing home residents with dementia as judged by relatives. International Psychogeriatrics. 25(10):1697-707.
Performance Gap
The data provided here comes from LTC Trend Tracker, a data tool managed by American Health Care Association (AHCA) to allow nursing homes to benchmark and trend their performance on various metrics, including CoreQ. On a voluntary basis, nursing homes can upload data themselves or delegate a customer satisfaction vendor to upload on their behalf.
The data provided below reflects data from 2024q2-2025q1 representing 1,096 providers and 18,779 family member responses. With an average satisfaction rate of 73.3%, several of the top deciles (8, 9, and 10) are topped out at 100% satisfaction, but the bottom five deciles are below an average rating of 76% indicating there is still room for improvement.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 73.30 | 0.00 | 13.70 | 42.20 | 54.10 | 66.50 | 76.40 | 85.30 | 94.80 | 100.00 | 100.00 | 100.00 | 100.00 |
| N of Entities | 1096 | 39 | 109 | 110 | 110 | 109 | 110 | 110 | 110 | 110 | 110 | 109 | 348 |
| N of Persons / Encounters / Episodes | 18779 | 133 | 2305 | 2151 | 1725 | 2060 | 2302 | 2540 | 2948 | 1077 | 985 | 686 | 2870 |
Performance Scores by Decile, 2024q2-2025q1
| Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 73.30% | 0.00% | 13.70% | 42.20% | 54.10% | 66.50% | 76.40% | 85.30% | 94.80% | 100.00% | 100.00% | 100.00% | 100.00% |
| N of Entities | 1096 | 39 | 109 | 110 | 110 | 109 | 110 | 110 | 109 | 110 | 110 | 109 | 348 |
| N of Persons / Encounters / Episodes | 18779 | 133 | 2305 | 2151 | 1725 | 2060 | 2302 | 2540 | 2948 | 1077 | 985 | 686 | 2870 |
Care Gaps
Closing Care Gaps
Multiple studies in the past twenty years have examined racial disparities in the care of nursing facility residents and have consistently found poorer care in facilities with high minority populations (Fennell et al., 2000; Mor et al., 2004; Smith et al., 2007). Work on racial disparities in nursing facilities’ quality of care between elderly white and black residents within nursing facility has shown clearly that nursing homes remain relatively segregated and that specifically nursing home care can be described as a tiered system in which blacks are concentrated in marginal-quality homes (Li, Ye, Glance & Temkin-Greener, 2014; Fennell, Feng, Clark & Mor, 2010; Li, Yin, Cai, Temkin-Greener, Mukamel, 2011; Chisholm, Weech-Maldonado, Laberge, Lin, & Hyer, 2013; Mor et al., 2004; Smith et al., 2007). Such homes tend to have serious deficiencies in staffing ratios, performance, and are more financially vulnerable (Smith et al, 2007; Chisholm et al., 2013). Based on a review of the nursing facility disparities literature, Konetzka and Werner concluded that disparities in care are likely related to this racial and socioeconomic segregation as opposed to within-provider discrimination (Konetzka and Werner 2009). This conclusion is supported, for example, by Grunier and colleagues who found that as the proportion of black residents in the nursing home increased the risk of hospitalization among all residents, regardless of race, also increased (Grunier et al., 2008). Thus, adjusting for racial status has the unintended effect of adjusting for poor quality providers not to differences due to racial status and not within-provider discrimination.
Therefore, lower satisfaction scores for both Caucasian and Blacks and other ethnicities are likely to increase as the proportion of black residents increases in a SNF, indicating that the best measure of racial disparities in satisfaction rates is one that measures scores at the facility level. That is, ethnic and social economic status differences are related to inter-facility differences not to intra-facility differences in care. Therefore, the literature suggests that racial status should not be risk adjusted otherwise one is adjusting for the poor quality of the SNFs rather than differences due to racial status.
The CoreQ information was collected from nursing homes and assisted living facilities during 2021. These facilities were primarily located in MA, PA, and NJ. These facilities were a sample of convenience, in that they had voluntarily participated in collecting CoreQ information during 2020 and continued doing so in 2021. In addition to the CoreQ items, in 2021 demographic questions including asking for the respondents’ race were also included.
In these facilities, discharged residents and family members were sent a CoreQ questionnaire, a letter describing the questionnaire, and postage paid return envelope. The questionnaires were anonymous, but they did include the facility name as part of the survey. For long-stay residents, social workers in the facilities collected CoreQ information. The questionnaires used were also anonymous. In all cases, the standard data collection exclusions for collecting CoreQ were used.
The proportion of black residents in nursing homes is not uniform, and some facilities have higher concentrations of black residents than others. This has led to investigations of both within-nursing home care examining if differences exist by race within the same facility, and between-nursing home care examining if differences exist across facilities. Both within- and between-nursing home differences were examined.
For all three samples (discharge, resident, and family), overall scores for black residents are lower than those for white residents. However, we know that black residents are disproportionately cared for in lower quality facilities (described above). This may influence the overall scores. Indeed, when we examine nursing homes with >20% black residents, the resulting scores for black vs. white residents are almost identical.
References
Chisholm L, Weech-Maldonado R, Laberge A, Lin FC, Hyer K. (2013). Nursing home quality and financial performance: does the racial composition of residents matter? Health Serv Res;48(6 Pt 1):2060–2080.
Fennell ML, Feng Z, Clark MA, Mor V. (2010). Elderly Hispanics more likely to reside in poor-quality nursing homes. Health Aff (Millwood);29(1):65–73.
Grabowski, D.C. (2004). The admission of Blacks to high-deficiency nursing homes. Medical Care 42(5): 456-464.
Gruneir, A., Miller, S. C., Feng, Z., Intrator, O., & Mor, V. (2008). Relationship between state Medicaid policies, nursing home racial composition, and the risk of hospitalization for black and white residents. Health Services Research, 43(3), 869-881.
Konetzka, R. T., & Werner, R. M. (2009). Review: Disparities in long-term care building equity into market-based reforms. Medical Care Research and Review, 66(5), 491-521.
Li Y, Yin J, Cai X, Temkin-Greener J, Mukamel DB. (2011). Association of race and sites of care with pressure ulcers in high-risk nursing home residents. JAMA;306(2):179–186.
Li Y, Ye Zhiqiu, Glance, Laurent & Temkin-Greener, Helena. (2014). Trends in family rating experience with care and racial disparities among Maryland nursing homes. Med Care, 52(7): 641-648.
Mor, V., Zinn, J., Angelelli, J., Teno, J. M., & Miller, S. C. (2004). Driven to tiers: socioeconomic and racial disparities in the quality of nursing home care. Milbank Quarterly, 82(2), 227-256.
Smith, D. B., Feng, Z., Fennell, M. L., Zinn, J. S., & Mor, V. (2007). Separate and unequal: racial segregation and disparities in quality across US nursing homes. Health Affairs, 26(5): 1448-1458.
Feasibility
Feasibility
The data elements are routinely generated through the satisfaction survey. All data elements are in defined fields in a combination of electronic sources. In an effort to keep administrative burden low to encourage collection of satisfaction data, which is important in the field, there are no efforts to develop a plan for electronic collection.
Facilities have no data entry burden. However, they do have data collection burden. In work we have done with CMS for a different CoreQ survey (NH discharge survey) the cost burden for the facility was calculated to be $2.80 per respondent. This calculation was based on requiring more than 20 data elements; whereas, here only 4 are needed. The cost will likely be less than $2.80. Because CoreQ can be added to the beginning of existing surveys, the cost and burden of adoption can be minimal.
No barriers were encountered with the measure specifications. The measure calculation was sometimes confused with an average score. The CoreQ measure is not an average. This is explained on reports produced and in the technical manual.
All of the patient surveys are anonymous. In addition, scores are only calculated with 20 or more survey returns. Thus, patient confidentiality is protected.
This is a maintenance application. As detailed above we have continued to collect CoreQ data to examine any changes in scores and implementation issues. No adjustment to the measure has occurred.
Proprietary Information
There are no fees, licensing, or other requirements for this measure. The Long-Stay Family survey/questionnaire and the methodology are proprietary. The CoreQ name is trademarked by the American Health Care Association/National Center for Assisted Living.
Scientific Acceptability
Testing Data
Data utilized for testing came from CoreQ: Long-Stay Family questionnaire from CY2025. To validate the measure; we also utilized CASPER Quality Indicators and data from Nursing Home Compare from CY2025 from a national sample of facilities.
Additionally, Updated testing based on the Pilot CoreQ: Long-Stay Family questionnaire was completed for CY2025 from a national sample of facilities.
Performance gap data was collected from LTC Trend Tracker, a data tool managed by the American Health Care Association (AHCA) to allow nursing homes to benchmark and trend their performance on various metrics, including CoreQ. The data provided below reflects data from 2024q2-2025q1.
Reliability and Validity data are derived from the CoreQ: Long-Stay Family questionnaire from CY2025. Also, CASPER Quality Indicators, CMS Nursing Home Compare, Five-Star, and Payroll-Based Journal data from CY2025 were used.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
The dates of the data used for testing are:
- Centers for Medicare and Medicaid Services (CMS) Nursing Home Compare, CASPER Quality indicators, Five-Star, and Payroll-Based Journal data from CY2025 (January-December 2025).
- Pilot CoreQ: Long-Stay Family questionnaire data from CY2025 (January-December 2025).
- CoreQ: Long-Stay Family questionnaire data from CY2025 (January-December 2025).
- Performance gap data was collected from LTC Trend Tracker for 2024q2-2025q1.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
We conducted two levels of testing in the development of the CoreQ: Long-Stay Family measure. The first focused on testing (e.g., reliability, validity, exclusions) of the CoreQ: Long-Stay Family questionnaire. The first source of data (pilot data) was utilized in developing and choosing the items to be included in the CoreQ: Long-Stay Family questionnaire. This included using a questionnaire with 22 items. Below we call this the Pilot CoreQ: Long-Stay Family questionnaire.
Once the CoreQ: Long-Stay Family questionnaire was developed, a second source of data was used to test the validity of the CoreQ: Long-Stay Family measure (i.e., facility and summary score validity).
For this E&M similar analyses were conducted using CY2025 data. This was to check the continued validity of the CoreQ: Long-Stay Family measure.
Reliability Testing: CoreQ: Long-Stay Family Questionnaire data from CY2025 (January 2025-December 2025) a national sample (n=536, 16,021 family members or resident representatives) at the data element, person/questionnaire, and measure (facility) level.
Validity Testing: CoreQ: Long-Stay Family Questionnaire data from CY2025 (January 2025-December 2025) a national sample (n=536, 16,021 family members or resident representatives). To test the validity of the measure, CASPER Quality Indicators, CMS Nursing Home Compare, Five-Star, and Payroll-Based Journal data from CY2025 (January 2025- December 2025) was used.
The testing and analysis included several data sources, one of which had additional variables collected for a subset of respondents:
1. The Pilot CoreQ: Long-Stay Family questionnaire was examined using responses from 1,324 Family members or resident representatives from a national sample of nursing facilities (Data Source #1, see table 5.1.3 below).
a. In addition, Family-level sociodemographic (SDS) variables were examined using this same sample of 1,324 Family members or resident representatives (#1 below) in nursing facilities across the US. (Data Source #1, see Table 5.1.3).
2. Validity testing of the Pilot CoreQ: Long-Stay Family questionnaire was examined using responses from 100 Family members or resident representatives from the Pittsburgh area. (Data Source #2, see Table 5.1.3).
3. CoreQ: Long-Stay Family measure was examined using 221 facilities and included responses from 6,192 Family members or resident representatives. These nursing facilities were located in multiple states across the US. (Data Source #3, see Table 5.1.3).
4. Repeat data came from a national sample of facilities collected in 2025. This included 536 facilities and included responses from 16,021 family members or resident representatives. These nursing facilities were located in multiple states across the US.
Some basic descriptive characteristics of these facilities (data sources) are provided below.
Table 5.1.3: Descriptive Statistics of Centers Included in the Analysis
Data Source | Average Number of Licensed Beds | Average Daily Census | Sample Size of Family members (N) |
Source 1 | 136 | 122 | 1,324 |
Source 2 | 202 | 188 | 100 |
Source 3 | 142 | 131 | 6,192 |
Source 4 | 136 | 122 | 16,021 |
Data Source: CoreQ: Long-stay Family Questionnaire (CY2014 (Pilot Data: Sources 1-3) and CY2025, Source 4)
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
Data was used from the CoreQ: Long-Stay Family questionnaire. The questionnaire was administered to all eligible long-stay family. The testing and analysis included:
1. The Pilot CoreQ: Long-Stay Family questionnaire was examined using responses from 1,324 family members or resident representatives from a national sample of nursing facilities. (Data Source #1 above, Table 5.1.3)
a. In addition, Family-level sociodemographic (SDS) variables were examined using this same sample of 1,324 family members (Data Source #1 above,Table 5.1.3) in nursing facilities across the US.
2. Validity testing of the Pilot CoreQ: Long-Stay Family questionnaire was examined using responses from 100 family members from the Pittsburgh area. (Data Source #2 above, Table 5.1.3)
3. CoreQ: Long-Stay Family questionnaire measure was examined using 221 facilities and included responses from 6,192 family members or resident representatives. These nursing facilities were located in multiple states across the US. (Data Source #3 above, Table 5.1.3)
The descriptive characteristics of the family members are given in the following table that includes information from all of the data used (the education level and race information comes only from the sample described above with 1,324 respondents, as this data was not collected for the other samples).
Table 5.1.4: Respondent Demographics
| DEMOGRAPHICS |
| Percent (CY2014, Samples 1-3 pooled) | Percent (CY2025 data) |
| Are you male or female? | Male | 30% | 28% |
Female | 70% | 72% | |
| What year were you born? | Average | 1946 | 1954 |
| What is the highest grade or level of school that you have completed? | Some HS | 7% | 4% |
HS or GED | 32% | 26% | |
Some College/ 2yr Degree | 32% | 31% | |
4yr College Degree | 15% | 21% | |
>4yr College Degree | 15% | 18% | |
| What is your race? | White | 92% | 87% |
Black | 7% | 11% | |
Asian | 1% | 1% | |
Native Hawaiian | 0% | <1% | |
American Indian | 0% | <1%
|
Data Source: CoreQ: Long-Stay Family Questionnaire (CY2014 (Pilot Data) and CY2025)
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
Reliability
We measured reliability at the: (1) data element level; (2) the person/questionnaire level; (3) at the measure (i.e., facility) level and, (4) accountable entity-level. More detail of each analysis follows.
1. Data Element Level
To determine if the CoreQ: Long-Stay Family questionnaire items were repeatable, producing the same results a high proportion of the time when assessed in the same population in the same time period, we re-administered the questionnaire to family members 1 month after their first survey. The Pilot CoreQ: Long-Stay Family questionnaire had responses from 100 family members; we re-administered the survey to 50 of these same family members. The re-administered sample was a sample of convenience as they represented family members from the Pittsburgh area (the location of the team testing the questionnaire). To measure the agreement, we calculated first the distribution of responses by question in the original round of surveys, and then again in the follow-up surveys (they should be distributed similarly); and second, calculated the correlations between the original and follow-up responses by question (they should be highly correlated). Repeat data came from a national sample of facilities collected in 2025. This included 536 facilities and included responses from 16,021 family members or resident representatives. These nursing facilities were located in multiple states across the US.
2. Person/Questionnaire Level
Having tested whether the data elements matched between the pilot responses and the re-administered responses, we then examined whether the person-level results matched between the Pilot CoreQ: Long-Stay Family questionnaire responses and their corresponding re-administered responses. In particular, we calculated the percent of time that there was agreement between whether or not the pilot response was poor, average, good, very good or excellent, and whether or not the re-administered response was poor, average, good, very good or excellent. Repeat data came from a national sample of facilities collected in 2025. This included 536 facilities and included responses from 16,021 family members or resident representatives. These nursing facilities were located in multiple states across the US.
3. Measure (Facility) Level
Last, we measured stability of the facility-level measure when the facility’s score is calculated using multiple “draws” from the same population. This measures how stable the facility’s score would be if the underlying family members are from the same population but are subject to the kind of natural sample variation that occurs over time. We did this by bootstrap with 10,000 repetitions of the facility score calculation and present the percent of facility resamples where the facility score is within 1 percentage point, 3 percentage points, 5 percentage points, and 10 percentage points of the original score calculated on the Pilot CoreQ: Long-Stay Family questionnaire sample.
4. Accountable Entity-Level Reliability
Entities were ranked by mean performance score and grouped into deciles of approximately equal size. Reliability was estimated using a signal-to-noise framework in which reliability represents the proportion of total variance attributable to true between-entity differences rather than measurement error. For each decile, we summarized the mean performance score, number of entities, and total number of persons/encounters/episodes contributing to the estimates. Reliability values ≥0.70 were considered acceptable and values ≥0.80 were considered good.
Data Source: CoreQ: Long-Stay Family Questionnaire for CY2025.
1. Data Element Level
Table 5.2.3a shows the three CoreQ: Long-Stay Family questionnaire items, and the response per item for both the pilot survey of 100 family members and the re-administered survey of 50 family members. The responses in the pilot survey are not statistically significant from the re-administered survey. This shows that the data elements were highly repeatable and produced the same results a high proportion of the time when assessing the same population in the same time period. Similar findings are shown from the repeat 2025 data.
2. Person/Questionnaire Level
Table 5.2.3c shows the CoreQ: Long-Stay Family questionnaire items, and the agreement in response per item for both the pilot survey of 100 family members compared with the re-administered survey of 50 family members. The person-level responses in the pilot survey are not statistically significant from the re-administered survey. This shows that a high percent of time there was agreement between whether or not the pilot response was poor, average, good, very good or excellent, and whether or not the re-administered response was poor, average, good, very good or excellent. Table 5.2.3d shows the agreement between the pilot and re-administered responses. In summary, 97% or more of the re-administered responses agreed with their corresponding pilot responses, in terms of whether or not they were rated in the categories of poor or average or good, very good or excellent. Similar findings are shown from the repeat 2025 data.
3. Measure (Facility) Level
After having performed 10,000-repitition bootstrap 10.6% of bootstrap repetition scores were within 1 percentage point of the score in the original sample, 38.5% were within 3 percentage points, 64.8% were within 5 percentage points, and 43.6% were within 10 percentage points.
*The tables mentioned above are in the Additional Reliability Testing Results section.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
In summary, the measure displays a high degree of element-level, questionnaire-level, and measure (facility)-level reliability. First, the CoreQ: Long-Stay Family questionnaire data elements were highly repeatable, with pilot and re-administered responses agreeing between 97% and 99% of the time depending on the question. That is, this produced the same results a high proportion of the time when assessed in the same population in the same time period. Second, the questionnaire level scores were also highly repeatable, with pilot and re-administered responses agreeing 98% of the time (or more). Third, a facility drawing family members from the same underlying population will only vary modestly. The 10,000-repetition bootstrap results show that the CoreQ: Long-Stay Family measure scores from the same facility are moderately stable given the minimum sample size of 20 was set for this measure; and the maximum sample size was 95.
The performance measure demonstrated good overall reliability (0.798). Reliability increased progressively across performance deciles, ranging from 0.45 among the lowest-performing entities to 0.93 among the highest-performing entities, indicating stronger measurement stability among higher-performing groups.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | |||||||||||||
| Mean Performance Score | 73.30 | 0.00 | 13.70 | 42.20 | 54.10 | 66.50 | 76.40 | 85.30 | 94.80 | 100.00 | 100.00 | 100.00 | 100.00 |
| N of Entities | 1096 | 39 | 109 | 110 | 110 | 109 | 110 | 110 | 109 | 110 | 110 | 109 | 348 |
| N of Persons / Encounters / Episodes | 18779 | 133 | 2305 | 2151 | 1725 | 2060 | 2302 | 2540 | 2948 | 1077 | 985 | 686 | 2870 |
Accountability Entity Level Reliability Testing Results by Denominator, January - December 2025
| Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max |
Reliability | 0.798 | 0.45 | 0.45 | 0.76 | 0.79 | 0.84 | 0.88 | 0.75 | 0.88 | 0.84 | 0.86 | 0.93 | 0.93 |
Mean Performance Score | 83% | 34% | 41% | 50% | 57% | 64% | 70% | 78% | 84% | 87% | 94% | 99% | 100% |
N of Entities | 620 | 5 | 62 | 62 | 62 | 62 | 62 | 62 | 62 | 62 | 62 | 62 | 18 |
N of Persons / Encounters / Episodes | 18830 | 156 | 1860 | 1911 | 1845 | 1934 | 1809 | 1867 | 1922 | 1870 | 1823 | 1989 | 935 |
Note: Table 2a used Cronbach's alpha to show computations per decile.
Accountable Entity Level Reliability Testing Results by Reliability Score, January - December 2025
| Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max |
Reliability | 0.798 | 0.45 | 0.45 | 0.76 | 0.79 | 0.84 | 0.88 | 0.75 | 0.88 | 0.84 | 0.86 | 0.93 | 0.93 |
Note: Table 2b used Cronbach's alpha to show computations per decile.
Validity
In the development of the CoreQ: Long-Stay Family questionnaire, several sources of data were used to perform three levels of validity testing. These are described above in Section 1.5.
The first source of data (See Table 5.1.3) (data from a sample of convenience collected near the researchers developing the questionnaire in Pittsburgh) was used in developing and choosing the format to be utilized in the CoreQ: Long-Stay Family questionnaire (i.e., response scale).
The second source of data (See Table 5.1.3) was pilot data collected from a national sample of 1,324 family members. This data was used in choosing the items to be used in the CoreQ: Long-Stay Family questionnaire (i.e., questionnaire items). This data was also used in examining Family-level sociodemographic (SDS) variables.
The third source of data (collected from 221 facilities) (See Table 5.1.3) was used examine the validity of the CoreQ: Long-Stay Family measure (i.e., facility and summary score validity). These family members / nursing facilities were from multiple states across the U.S.
Thus, the following sections describe this validity testing:
1. Validity Testing of the questionnaire format used in the CoreQ: Long-Stay Family questionnaire (using data source 1, from above); (See Table 5.1.3).
2. Testing the items for the CoreQ: Long-Stay Family questionnaire (using data source 2, from above;(See Table 5.1.3));
3. Testing to determine if a sub-set of items could reliably be used to produce an overall indicator of satisfaction (Core Q: Long-Stay Family measure) (using data source 3, from above);
4. Validity testing for the CoreQ: Long-Stay Family measure (also using data source 1, from above) (See Table 5.1.3).
1. Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Family Questionnaire
A. The face validity of the domains used in the CoreQ: Long-Stay Family questionnaire was evaluated via a literature review. The literature review was conducted to examine important areas of satisfaction for LTC family. Specifically, the research team examined 12 commonly used satisfaction surveys and reports to determine the most valued domains when looking at satisfaction. These surveys were identified by completing internet searches in PubMed and Google. Key terms that were searched included: Family satisfaction, long-term care satisfaction, and elderly satisfaction.
B. The face validity of the domains was also examined using a focus group of family members. The overall ranking used was 1=Most important and 22=Least important. That is family members were asked to rank the domains from most important to least important. The respondents were family members (N=40) of residents in five nursing facilities in the Pittsburgh region.
C. The face validity of the Pilot CoreQ: Long-Stay Family questionnaire response scale was also examined. The respondents were family members (N=40) with residents in five nursing facilities in the Pittsburgh region The percent of respondents that stated they “fully understood” how the response scale worked, could complete the scale, AND in cognitive testing understood the scale was used.
D. The Flesch-Kinkaid scale was used to determine if respondent correctly understood the questions being asked (Streiner & Norman, 1995).
Streiner, D. L. & Norman, G.R. (1995). Health measurement scales: A practical guide to their development and use. 2nd ed. New York: Oxford.
2. Testing the Items for the CoreQ: Long-Stay Family Questionnaire
The second series of validity testing was used to further identify items that should be included in the CoreQ: Long-Stay Family questionnaire. This analysis was important, as all items in a satisfaction measure should have adequate psychometric properties (such as low basement or ceiling effects). For this testing, (1) A pilot group of 40 family members was first used in focus groups; (2) a Pilot version of the CoreQ: Long-Stay Family questionnaire survey was administered consisting of 18 items (N= 1,324 family members). The testing consisted of:
A. Family members were asked to rate the 18 different satisfaction questions related to their experience in SNFs. This was conducted with a pilot group of 40 family members in focus groups.
B. The Pilot CoreQ: Long-Stay Family questionnaire items performance with respect to the distribution of the response scale and with respect to missing responses. (Using 1,324 family members described above)
C. The intent of the Pilot instrument was to have items that represented the most important areas of satisfaction (as identified above) in a parsimonious manner. Additional analyses such as exploratory factor analysis (EFA) were used to eliminate items in the Pilot instrument. This was an iterative process that included using Eigenvalues from the principal factors (unrotated) and correlation analysis of the individual items. (using 1,324 family members described above)
3. To determine if a Sub-Set of Items could be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Family Measure).
The CoreQ: Long-Stay Family measure under development was meant to represent overall satisfaction with as few items as possible. The testing given below describes how this was achieved.
A. To support the construct validity that the idea that the CoreQ items measured a single concept of “satisfaction” – we performed a correlation analysis using all items in the instrument.
B. In addition, using all items in the instruments a factor analysis was conducted. Using the global items Q1 (“How satisfied are you with the facility?”) the Cronbach’s Alpha of adding the “best” additional item was examined.
4. Validity Testing for the Core Q: Long-Stay Family Measure.
A. To determine if the 3 items in the CoreQ: Long-Stay Family questionnaire were a reliable indicator of satisfaction, the correlation between these three items (the “CoreQ: Long-Stay Family Measure”) and ALL of the items on the Pilot CoreQ instrument was conducted.
B. We performed additional validity testing of the facility-level CoreQ: Long-Stay Family measure by examining the correlations between the CoreQ: Long-Stay Family individual item scores within the survey and i) measures of regulatory compliance and other quality metrics from the Certification and Survey Provider Enhanced Reporting (CASPER) data, and ii) several other quality metrics from Nursing Home Compare. If the CoreQ Long Stay Family scores correlate negatively with the measures that decrease as they get better, and positively with the measures that increase as they get better, then this supports the validity of the CoreQ Long Stay Family measure.
Repeat data came from a national sample of facilities collected in 2025. This included 536 facilities and included responses from 16,021 family members or resident representatives. These nursing facilities were located in multiple states across the US.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
1. Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Family Questionnaire
A. The face validity of the domains used in the CoreQ: Long-Stay Family questionnaire was evaluated via a literature review (described above). The research team examined the surveys and reports to identify the different domains that were included. The research team scored the domains by simply counting if an instrument included the domain. Table 5.3.4a gives the domains that were found throughout the search, as well as a score. An example is the domain clinical care, this was used in 10 out of the 12 surveys identified in the literature. An interpretation of this finding would be that items addressing clinical care are extremely important in satisfaction surveys. These domains were used in developing the pilot CoreQ: Long-Stay Family questionnaire items. A recent scoping review by Li et al. (2023) provides very similar results.
Reference
Li, X., Mpofu, E., Collins, S., Yin, C., & Shaw, T. (2023). Resident satisfaction indicators in long term care settings in the United States: A scoping review. Aging and Health Research, 3.
B. The face validity of the domains was also examined using family members. The following abbreviated table shows the rank of importance for each group of domains. The overall ranking used was 1=Most important and 22=Least important. The ranking of the 3 areas used in the CoreQ: Long-Stay Family questionnaire are shown. Note, the food domain was ranked third – but was excluded from the CORE Q based on additional analyses showing that it was highly correlated with the overall domain; thus, it added little to the measure.
C. The face validity of the pilot CoreQ: Long-Stay Family questionnaire response scale was also examined. Table 5.3.4c gives the percent of respondents that stated they “fully understood” how the response scale worked, could complete the scale, AND in cognitive testing understood the scale.
D. The CoreQ: Long-Stay Family questionnaire was purposefully written using simple language. No a priori goal for reading level was set, however a Flesch-Kinkaid scale score of six, or lower, is achieved for all questions.
2. Testing the Items for the CoreQ: Long-Stay Family Questionnaire
A. Each family member was asked to rate on a scale of 1 to 10 (with 10 as the best) how important they thought the question was for evaluating the experience with SNF care. The three questions included in the COREQ were highly rated out of all the questions and in analysis of family member’s responses to 18 questions. That is, these three items were shown to provide unique information to distinguish satisfaction with SNFs. Specifically, “In recommending this facility to your friends and family, how would you rate it overall?” had an average score of 9.69; “Overall, how would you rate the staff?” had an average score of 9.6; and, “How would you rate the care you receive?” had an average score of 9.5. This shows a very pervasive influence of the satisfaction items with the experience of SNF care. See Table 1c.5 in Section 7. Supplemental Information.
B. The pilot CoreQ: Long-Stay Family questionnaire items are shown in Table 5.3.4d. This shows that the items performed well with respect to the distribution of the response scale and with respect to missing responses.
C. Using all items in the instruments (excluding the global item Q1 (“How would you rate the facility?”)) exploratory factor analysis (EFA) was used to evaluate the construct validity of the measure. The Eigenvalues from the principal factors (unrotated) are presented in the Table below. In this analysis, the first Eigenvalue is overwhelmingly greater than the second Eigenvalue, this supports the proposition that the CoreQ instrument is measuring a single global concept of customer satisfaction – rather than a number of sub-concepts of customer satisfaction. Sensitivity analyses using principal factors and rotating provide highly similar findings.
3. To determine if a Sub-Set of Items could Reliably be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Family measure).
A. To support the construct validity that the idea that the CoreQ items measured a single concept of “satisfaction” – we performed a correlation analysis using all items in the instrument. The analysis identifies the pairs of CoreQ items with the highest correlations. The highest correlations are shown in the Table 5.3.4f. Items with the highest correlation are potentially providing similar satisfaction information. Because items with the highest correlation were potentially providing similar satisfaction information they could be eliminated from the instrument. Note, the table provides 7 sets of correlations, however the analysis was conducted examining all possible correlations between items.
B. In addition, using all items in the instrument a factor analysis was conducted. Using the global items Q1 (“How satisfied are you with the facility?”) the Cronbach’s Alpha of adding the “best” additional item is shown in the table below. Cronbach’s alpha measures the internal consistency of the values entered into the factor analysis; a value of 0.7 or higher is generally considered acceptably high. The additional item(s) is considered best in the sense that it is most highly correlated with the existing item and therefore provides little additional information about the same construct. So, this analysis was also used to eliminate items. Note, table 5.3.4g again provides 7 sets of correlations, however the analysis was conducted examining all possible correlations between items. Thus, using the correlation information and factor analysis 3 items representing the CoreQ: Long-Stay Family questionnaire were identified.
4. Validity Testing for the Core Q: Long-Stay Family Measure.
The overall intent of the analyses described above was to identify if a sub-set of items could reliably be used to produce an overall indicator of satisfaction, the CoreQ: Long-Stay Family questionnaire.
A. The items were all scored according to the rules identified elsewhere. The same scoring was used in creating the 3 item CoreQ: Long-Stay Family questionnaire summary score and the satisfaction score using the Pilot CoreQ: Long-Stay Family questionnaire. The correlation was identified as having a value of 0.90.
That is, the correlation score between actual the “CoreQ: Long-Stay Family Measure” and all of the 18 items used in the Pilot instrument indicates that the satisfaction information is approximately the same if we had included either the 3 items (much less burdensome, and therefore likely to yield a higher response rate) or the 18 item Pilot instrument. Thus, we only included the three measures as additional measures did not provide additional information for a quality measure to assess a facilities satisfaction score. Additional questions may help with quality improvement efforts to identify specific areas of satisfaction or dissatisfaction.
B. We performed additional validity testing of the facility-level CoreQ: Long-Stay Family measure by measuring the correlations between the CoreQ: Long-Stay Family measure scores and A) measures of regulatory compliance and other quality metrics from the Certification and Survey Provider Enhanced Reporting (CASPER) data, B) several other quality metrics from Nursing Home Compare, C) risk-adjusted discharge to community measure, and D) risk adjusted PointRight® Pro 30™ Rehospitalizations.
CoreQ: Long-Stay Family measure is the percentage of family members of residents discharged from the facility within 100 days of admission from a hospital to the nursing facility who, on average for the three CoreQ items included in the measure, rated the facility >= 3. We measured satisfaction using family’s responses to the three items from the CoreQ: Long-Stay Family questionnaire (see Table 5.3.4a).
The summary score from the 3 CoreQ: Long-Stay Family questionnaire items is calculated in the following way: Respondents answering poor are given a score of 1, average = 2, good =3, very good =4 and excellent =5. For the 3 questionnaire items the average score for the Family is calculated. The facility score represents the percent of family members with average scores of 3 or above. This score should be associated with quality. Therefore, for each facility in the sample the correlation with other quality indicators was examined.
(i) Relationship with CASPER Quality Indicators
Certification and Survey Provider Enhanced Reporting (CASPER) contains data collected as part of state/federal nursing home inspections. In short, nursing facilities that accept residents with Medicare and/or Medicaid payments are surveyed. This includes most (i.e., 97% [16,000 facilities]) nursing homes in the U.S. The survey process occurs approximately yearly and includes the recording of many quality characteristics of the nursing home. These include restraint use; pressure ulcers; catheter use; antipsychotic use; antidepressant use; antianxiety use; and use of hypnotics. These are commonly used quality indicators used for examining the quality of nursing homes.
In addition, when a nursing home is determined not to meet a certification minimum standard a deficiency citation is issued. These deficiency citations are also commonly used in the analyses of the quality of nursing homes. Approximately 180 deficiency citations exist and are grouped into 16 categories. These 16 categories group like areas together. They were developed by CMS and have considerable face validity; although, one limitation of using these categories is that they were not defined using empirical estimation (such as factor analysis). One category groups together 25 “quality of care” deficiency citations. In addition, for all deficiency citations a determination of the scope and severity of the problem(s) identified is also made. One of 12 categories is used which are labeled "A" through "L," with L having the highest severity and scope. The most severe (i.e., JKL) are used in this analysis. Thus, we would expect a negative correlation between family satisfaction and the number and severity of deficiencies cited by the State Survey agency.
(ii) Relationship with Nursing Home Compare (NHC) Quality Indicators, Five Star ratings, and staffing levels
Nursing Home Compare (NHC) is a nursing home report card. After several years of pilot testing, the Centers for Medicare and Medicaid Services (CMS) released this report card on the world-wide web in November of 2002. Briefly, Nursing Home Compare provides information for facility location, structural factors (such as ownership), and staffing characteristics (such as registered nurse [RN] staffing levels). Most significantly, standardized quality information is presented in what are called Quality Measures (QMs). These are calculated from MDS information.
At the time period of for this study CMS reported on 19 measures – these are called the core Quality Measures. The Quality Measures address specific areas of resident care, 5 are for short-stay residents and 14 are for long-stay residents. Long-stay measures are for those residents staying at a facility 3 months or more and short-stay measures are for residents staying at a facility less than 3 months. The long-stay measures are most pertinent to the CoreQ: Long-Stay Family questionnaire; therefore, these were used in the analyses.
Nursing Home Compare also uses a five-star rating for facilities. This is based on information from the health inspection, direct care staffing, and the MDS quality measures. A five-star facility is the highest score and a 1-star facility the lowest score. With respect to staffing, two measures are used: 1) RN hours per family day; and 2) total staffing hours (RN+ LPN+ nurse aide hours) per family day.
(iii) Relationship with the risk-adjusted Discharge to Community Measure
The Discharge to Community measure determines the percentage of all new admissions from a hospital who are discharged back to the community within 100 days and remain out of any skilled nursing center for the next 30 days. The measure, referring to a rolling year of MDS entries, is calculated each quarter and includes all new admissions to a SNF regardless of payor source. Unsuccessful discharges will result in the resident becoming a long stay resident, which we hypothesize would increase family member dissatisfaction in SNFs with poor discharge to community rates.
The results of testing for correlation between Risk-adjusted discharge to community measure and the CoreQ: Long-Stay Family questionnaire are provided in Table 5.3.4i.
(iv) Relationship with the risk adjusted PointRight® Pro 30™ Rehospitalizations
PointRight® Pro 30™ is an all-cause, risk adjusted rehospitalization measure. It provides the rate at which all patients (regardless of payer status or diagnosis) who enter skilled nursing facilities from acute hospitals and are subsequently re-hospitalized during their SNF stay, within 30 days from their admission to the SNF. Individuals who are re-hospitalized after admission are much more likely to become a long stay residents. We hypothesize family members would therefore be more dissatisfied on average in SNFs with high short stay resident rehospitalization rates. The results of testing for correlation between Risk-adjusted PointRight® Pro 30™ Rehospitalizations measure and the CoreQ: Long-Stay Family questionnaire are provided in Table 5.3.4j.
Repeat data came from a national sample of facilities collected in 2025. This included 536 facilities and included responses from 16,021 family members or resident representatives. These nursing facilities were located in multiple states across the US.
Summary
All staffing and quality indicators were significantly associated with the outcome (p < 0.001), with agency staffing and staff turnover demonstrating the strongest associations, suggesting meaningful relationships between staffing structure and the measured outcome.
*All tables are included in the Additional Validity Testing Results.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
1. Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Family Questionnaire
A. The literature review shows that domains used in the Pilot CoreQ: Long-Stay Family questionnaire items have a high degree of both face validity and content validity.
B. Family’s overall rankings, show the general “domain” areas used indicates a high degree of both face validity and content validity.
C. The results show that 100% of Family’s are able to complete the response format used. This testing indicates a high degree of both face validity and content validity.
D. The Flesch-Kinkaid scale score achieved for all questions indicates that respondents have a high degree of understanding of the item.
2. Testing the Items for the CoreQ: Long-Stay Family Questionnaire
A. The percent of missing responses for the items is very low. The distribution of the summary score is wide. This is important for quality improvement purposes, as nursing facilities can use benchmarks etc.
B. EFA shows that one factor explains the common variance of the items. A single factor can be interpreted as the only “concept” being measured by those variables. This means that the instrument measures the global concept of satisfaction and not multiple areas of satisfaction. This supports the validity of the CoreQ instrument as measuring a single concept of “customer satisfaction”. This testing indicates a high degree of criterion validity.
3. Testing to Determine if a Sub-Set of Items could Reliably be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Family measure)
A. Using the correlation information of the Core Q: Long-Stay Family questionnaire (18 items) and the 3 items representing the CoreQ: Long-Stay Family questionnaire a high degree of correlation was identified. This testing indicates a high degree of criterion validity.
B. EFA shows that one factor explains the common variance of the items. A single factor can be interpreted as the only “concept” being measured by those variables. This means that the instrument measures the global concept of satisfaction and not multiple areas of satisfaction. This supports the validity of the CoreQ instrument as measuring a single concept of “customer satisfaction”. This testing indicates a high degree of criterion validity.
4. Validity Testing for the Core Q: Long-Stay Family Measure
A. The correlation of the 3 item CoreQ: Long-Stay Family measure summary score (identified elsewhere in this document) with the overall satisfaction score (scored using all data and the same scoring metric) gave a value of 0.90.
That is, the correlation score between actual the “CoreQ: Long-Stay Family Measure” and all of the 18 items used in the Pilot instrument indicates that the satisfaction information is approximately the same if we had included either the 3 items or the 18 item Pilot questions.
This indicates that the CoreQ: Long-Stay Family measure score adequately represents the overall satisfaction of the facility. This testing indicates a high degree of criterion validity.
(i) Relationship with CASPER Quality Indicators
The CASPER Quality Indicators all had negative correlation with the CoreQ: Long-Stay Family measure as expected (higher satisfaction is associated with better quality). These correlations range from ± 0.03 to 0.28. The CoreQ: Long-Stay Family measure is associated with these quality indicators. This testing indicates a reasonable degree of construct validity and convergent validity.
(ii) Relationship with Nursing Home Compare (NHC) Quality Indicators, Five Star ratings, and staffing levels
The Nursing Home Compare (NHC) Quality Indicators, Five Star ratings, and staffing levels had a moderate to high level of correlation with the CoreQ: Long-Stay Family measure. These correlations range from ± 0.11 to 0.45. The CoreQ: Long-Stay Family measure is associated with these quality indicators, and always in the hypothesized direction (good correlates with good). In particular, as emphasized in the structure-process-outcome framework of the evidence section, the link between staffing and customer satisfaction is particularly high, as confirmed by the correlation coefficients 0.45 for RN hours per resident-day and 0.42 for total staffing hours per resident day. This testing indicates a reasonable degree of construct validity and convergent validity.
We repeated some of this testing with 2025 data. This also included additional staffing data such as turnover, stability, and agency use. These factors were used as previous studies have shown strong associations with quality in nursing homes. We find a similar relationship with CoreQ. This updated testing indicates a high degree of construct validity and convergent validity.
(iii) Relationship with the risk-adjusted Discharge to Community Measure
The risk-adjusted Discharge to community measure was negatively correlated to the CoreQ: Long-Stay Family measure. The correlations range from -0.03 to -0.06, all of which are not statistically significant at the p-value of 0.05. This was not as hypothesized which may be related to some SNFs that specialize in long stay, have very low discharge to community rates as admissions do not have a plan to go home.
(iv) Relationship with the risk adjusted PointRight® Pro 30™ Rehospitalizations
The risk-adjusted PointRight® Pro 30™ Rehospitalizations was negatively correlated to the CoreQ: Long-Stay Family measure. The correlations range from -0.18 to -0.21, and all of them were statistically significant at the p-value of 0.05. This is expected because lower rehospitalization rates (an indicator of high quality) are associated with higher satisfaction scores. This was as hypothesized. This testing indicates a reasonable degree of construct validity and convergent validity.
In summary, moderate correlations were identified between the quality measures. As noted by Mor and associates (2003, p.41) “there is only a low level of correlation among the various measures of quality.” Castle and Ferguson (2010) also show the pattern of findings of quality indicators in nursing facilities is consistently moderate with respect to the correlations identified. The magnitude of correlations of the CoreQ with quality metrics are consistent with other findings in this setting. Thus, it is not surprising that “very high” levels of correlations were not identified. Nevertheless, moderate correlations were identified and with a larger 2025 sample and more reliable data the correlations were consistent.
References
Castle N. G., Ferguson J. C. (2010). What is nursing home quality and how is it measured? The Gerontologist, 50(4), 426–442.
Mor V., Berg K., Angelelli J., Gifford D., Morris J., Moore T. (2003). The quality of quality measurement in U.S. nursing homes. The Gerontologist, 43(Special Issue II), 37-46.
Risk Adjustment
Risk adjustment is noted in some recent publications examining long-term care satisfaction. For example, one recent report noted “As of the date of this report, risk adjustment for geography (Twin Cities Metro vs. Other) is recommended. The U of MN suggests evaluating risk adjustment for size once there is sufficient data from small facilities (<20 residents), to conduct meaningful analysis.” (Shippee, Woodhouse, & Skarpho, 2023). The rationale for this has some merits, as this same report notes this “allows for more fair comparisons between providers who may serve residents with different needs and have different resources.”
However, this does come with some complications. First, it is not clear what risk adjustment methodology should be used. A simple stratification here would seem appropriate. Second, the notion of fair comparisons comes from a provider perspective. An alternative view is that a consumer should know the quality (i.e., satisfaction level) of a facility based on a standard and uniform metric. So risk adjustment for CoreQ using facility metrics could be misleading.
A memo regarding the Home Health Care CAHPS (HHCAHPS) Survey patient-mix adjustments (MUC2024, Adjustment Factors) noted that in 2022 HHCAHPS field test were used to determine which patient characteristics (patient mix) affected patients’ assessment of the home health care they received. As with prior HHCAHPS field tests, the results of this showed differences in responses attributable to patient mix characteristics. Patient mix characteristics that were important included: Age, Education, Self-reported health status, and Mental/emotional status. Some have noted issues with HHCAHPS risk-adjustment (Lines eta l., 2020).
In long-term care hospitals (LTACs) Zuckerbraun and associates (2020) identified the need for the adjustment of a CHAPS like survey. Patient risk factors included: age, gender, education, ethnicity, race, marital staus, and overall health.
Kwon and Bowblis (2024) included the percent of residents with dementia, psychiatric illness and depression in their risk adjustment. For testing of CoreQ, a similar approach was not possible as the surveys are not linked directly to patient characteristics.
For CoreQ, an overall score is calculated for a facility (and as described elsewhere is not an average score of the items). Thus, the CoreQ score cannot be calculated using specific characteristics (such as age). So in our testing we used the average score for one question item (In recommending this facility to your friends and family, how would you rate it overall?). We acknowledge that one limitation of this is that the risk adjustment (if any) may be different for the other question items in CoreQ. Also, no interaction effects are used (e.g., male and <65). Factors such as SES and other factors not included may have an influence. And, lastly the theoretical reasons for the potential differences in scores is not well explained in the literature.
CoreQ Long-Stay Family
| Age | Mean Score |
| <65 | 3.61 |
| 65-75 | 3.69 |
| >75 | 3.63 |
| Sex | |
| Male | 3.58 |
| Female | 3.62 |
| Education | |
| Some High School | 3.60 |
| High School Graduate/GED | 3.57 |
| College Graduate or More | 3.59 |
Testing summary. Conducted in 2025 with 4,733 family members. The facilities used were geographically diverse, and representative of national long-term care characteristics for size, payor mix, and ownership). The score distribution is 1 (low) to 5 (high). The average scores do vary by the patient mix. However, the influence is small and given the numerous unknowns in this area (described above) risk-adjustment is not recommended at this time.
To date, results from satisfaction surveys have mostly not used risk adjustment. The CoreQ measures overall satisfaction. Providing the scores across entities without any adjustment does provide a fair comparison. In addition, the data elements that could be used for any adjustment are not collected as part of the CoreQ (the surveys are anonymous).
References
Kwon, J. & Bowblis, J.R. (2024). Association between nursing home five-star ratings and consumer satisfaction. JAMDA, 25, 105322.
Lines, L.M., Anderson, W.L., Gordek, Ha., & Kenyoun, A.E. (2020). Risk adjustment in home health care CAHPS. AJMC, 25(2), 58-59.
MUC2024. Patient-Mix Adjustment Factors for Home Health Care CAHPS Survey Field Test Results.
Shippee T., Woodhouse, M., & Skarpho, T. (2023). Building Resident Quality of Life and Family Satisfaction Measures for the Minnesota Assisted Living Report Card. A Report to the Minnesota Department of Human Services.
Zuckerbraun, S.M., Deutsch, A., Eicheldinger C., et al. (2020). Risk adjustment, mode adjustment, and nonresponse bias analysis on quality measures from a long-term care hospital experience of care survey. Archives of Physical Medicine and Rehabilitation, 101, 841-51.
Use & Usability
Use
The Quality Incentive Payment Program (QIPP) gives nursing facilities in New Jersey the opportunity to earn bonus payments if they achieve specific quality and performance goals that are essential to providing appropriate resident care. Each facility is eligible to earn additional payments per resident per day on top of their normal rate for residents that are Medicaid members based on the number of quality benchmarks that the facility achieves. The program is a collaboration between the Division of Aging and the Division of Medical Assistance and Health Service (DMAHS), which administers the state’s Medicaid program (NJ FamilyCare), within the Department of Human Services.
New Jersey: 348 SNFs and 126,240 patients
Facility-level; SNFs
TennCare value-based purchasing initiative to promote the delivery of high quality Long Term Services and Supports (LTSS), focusing on the performance measures that are most important to people who receive LTSS and their families. These changes will reward providers that improve the member’s experience of care and promote a person centered care delivery model. Nursing Facility payment will be based in part onresident’s level of need and on nursing facility performance on quality metrics set forth in the QuILTSS Quality Framework.
Tennessee: 350 SNFs and 83,185 patients
Facility-level; SNFs
The AHCA/NCAL National Quality Awards Program is a progressive program that is based on the Baldrige Criteria for Performance Excellence. This nationally recognized approach to performance excellence focuses on systems-based quality improvement to create.
The geographic area is the nation. The AHCA/NCAL National Quality Awards Program is used across the nation. Over 1,700 entities have received an award to date.
The level of analysis is the facility-level. The care settings are skilled nursing and assisted living facilities.
The program allows skilled nursing and assisted living organizations to benchmark personal metrics to those of their peers and examine ongoing quality improvement efforts.
Skilled Nursing and Assisted living facilities across the United States utilize LTC Trend Tracker. About 15,266 Skilled Nursing Facilities and 9,280 Assisted Living Facilities use the program.
The level of analysis is the facility-level. The care settings are skilled nursing and assisted living facilities.
Measuring and improving patient satisfaction is valuable to patients, because it is a way forward on improving the patient-provider relationship, which influences health care outcomes. The target population is all residents in long-term care, that includes Medicare, Medicaid and private pay. Functional long-term care accountability programs often include a satisfaction component. These programs often prioritize the following components:
• Quality Assurance and Performance Improvement (QAPI) system in place,
• Person-centered care,
• Data-driven monitoring of performance metrics,
• Strong Leadership,
• And ethical compliance.
Together, all these components create a framework for quality improvement.
Usability
Improving performance relies on the testing of change and benchmarking. Frequently collecting data is a necessary step to enhance and maximize quality improvement. Data collected during tests provides critical insight that is needed to determine the best path forward. Benchmarking is a process used to measure the quality and performance of your organization. Benchmarking plays a significant role in identifying patterns, providing context, and then guiding decision-making processes.
The CoreQ Long-Stay Family Satisfaction measure allows skilled nursing facilities to measure the impact of tests of change and benchmark their performance relative to other facilities. Specifically, facilities can increase the number of staff and/or improve staff training and measure the impact using CoreQ. Similarly, improvements in reduced adverse events, such as falls and hospitalizations, increase the family rating of care received and increase satisfaction. Finally, facilities can understand and address the needs and wants of families and residents, like certain activities or food, to increase their willingness to recommend the facility and CoreQ performance.
The actions needed to improve performance are not difficult once a process or plan for improvement is developed (e.g. Quality Assurance/Performance Improvement (QAPI)). Measured entities can overcome difficulties by monitoring data and results. Monitoring data often ensures you preserve the advances of the quality improvement effort. Developing a feedback and monitoring system to sustain continuous improvement helps providers preserve the advances of the quality improvement effort.
The CoreQ measure for families of skilled nursing residents has elevated the resident and family voice as well as help guide consumer choices as another way for potential residents to review the quality of a care facility. Specifically, the CoreQ measure has been independently tested as a valid and reliable measure of customer satisfaction. The CoreQ is a short survey with three to four questions which reduces response burden on residents and allows organizations to benchmark their results with consistent questions and response scale. Satisfaction vendors and providers have particularly appreciated how easy it is to integrate the CoreQ questions to their satisfaction surveys. They believe the short length relative to other survey tools, like HCAHPS, helps increase and maintain high response rates.
AHCA/NCAL developed LTC Trend Tracker, a web-based tool that enables long term and post-acute care providers, including assisted living, to access key information that can help their organization succeed. The CoreQ report and upload feature within LTC Trend Tracker includes an API (application programming interface) for vendors performing the survey on behalf of SNFs to upload data, so that the aggregate CoreQ results will be available to providers. Given that LTC Trend Tracker is the leading method for AHCA SNF members to profile their quality and other data, the incorporation of CoreQ into LTC Trend Tracker means it will immediately become the de facto standard for customer satisfaction surveys for the SNF industry. AHCA/NCAL continues to work with customer satisfaction vendors to promote CoreQ and receives requests for vendors to be added to the list of those incorporating CoreQ. Currently, there are over 40 vendors across the nation who can administer the CoreQ survey.
We also are working with states who require satisfaction measurement to incorporate CoreQ into their process. AHCA/NCAL has a presence in each state, and our state affiliates continue to promote the use of the CoreQ.
Feedback is continuously obtained through meetings with facility operators and vendors serving on AHCA/NCAL’s Customer Experience Committee and the CoreQ Vendors’ Workgroup. The purpose of the Customer Experience Committee is to champion the importance of meeting customer expectations now and in the future. This includes defining quality from the consumer’s perspective. Key areas of focus include collecting, analyzing, and using data to drive performance improvement, and the application of successful practices. The CoreQ Vendors’ Workgroup was created to help improve CoreQ usage and discuss ways to best support the CoreQ Vendors’ who administer the surveys.
AHCA/NCAL developed LTC Trend Tracker, a web-based tool that enables long term and post-acute care providers, including assisted living, to access key information that can help their organization succeed. The CoreQ report and upload feature within LTC Trend Tracker includes an API for vendors performing the survey on behalf of SNFs to upload data, so that the aggregate CoreQ results will be available to providers. Given that LTC Trend Tracker is the leading method for AHCA SNF members to profile their quality and other data, the incorporation of CoreQ into LTC Trend Tracker means it will immediately become the de facto standard for customer satisfaction surveys for the SNF industry. AHCA/NCAL continues to work with customer satisfaction vendors to promote CoreQ and receives requests for vendors to be added to the list of those incorporating CoreQ.
Among providers and vendors, we receive feedback during committee and workgroup meetings. For feedback on LTC Trend Tracker, we scope out the cost and feasibility of suggested enhancements. For example, we added a more graphical user interface option for the API, in addition to the original command line interface that was more technical, based on feedback from vendors.
For some of the feedback we receive, we use it as an opportunity to educate about best practices in survey collection and administration. For example, some vendors and providers inquire about administering CoreQ over the phone or other mixed modes of collection. In this instance, we caution vendors and providers about possible response or interviewer bias and recommend using written surveys as the primary method because it has been tested and shown to be reliable and valid.
The impact of COVID pandemic and the gradual recovery shows in the performance trend. Prior to the pandemic, the average satisfaction rate in LTC Trend Tracker was 83.5% (n=1,747). Over the next two years, the average rate dropped to a low of 68.3% (n=1,396) as the industry faced a massive staffing crisis, limited visitation, and had intense isolation procedures in place to limit potential outbreaks. Since, the average rate has been on the rise. The latest average rate is 75.9% (n=605). We suspect the increase in industry workforce as reported by the U.S. Bureau of Labor Statistics and the return to more normal operations related to visitors, activities, and patient interactions have helped increase satisfaction rates.
There were no negative consequences to individuals or populations identified during testing or evidence of unintended negative consequences to individuals or populations reported since the implementation of the CoreQ: Long-Stay Family questionnaire or the measure that is calculated using this questionnaire.
This is consistent with satisfaction surveys in general in nursing facilities. Many other satisfaction surveys are used in nursing facilities with no reported unintended consequences to patients or their families.
There are no potentially serious physical, psychological, social, legal, or other risks for patients. However, in some cases the satisfaction questionnaire can highlight poor care for some dissatisfied patients, and this may make those patients further dissatisfied.
Comments
Staff Preliminary Assessment
CBE #2616 Staff Assessment
Importance
Strengths:
- A clear logic model is provided, depicting the relationships between inputs (e.g., competency of staff, responsiveness of management), activities (e.g., discharge instructions, RN assessments), and desired outcomes (e.g., ratings of care and short stay discharge satisfaction). This model demonstrates how the measure's implementation will lead to the anticipated outcomes.
The problem this measure addresses aligns with past initiatives to improve nursing home care, and more recent evidence demonstrates that residents’ satisfaction with care is associated with measures of nursing home quality.
Data from quarter 2 of 2024 to quarter 1 of 2025 of the LTC Trend Tracker show a performance gap, with decile ranges from 0.00% to 100%, with an average 73.3%, indicating variation in measure performance and less than optimal performance across the target population. The top three deciles are topped out at 100%, while the bottom five have an average rating below 76%.Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Patient input was obtained through literature review and focus group of residents and family members. However, the literature evidence provided was older and no timeframe for when patient input was obtained was provided.
Limitations:
- The literature review mainly includes studies that are more than 10 years old, including their review of nursing home quality initiatives. The submission could be strengthened by discussion of the quality, quantity, and consistency of the older evidence, and providing updated references, if possible.
- The literature provided for importance to patients is older and no timeframe for when patient input was obtained is provided. The submission could be strengthened by discussion of the quality, quantity, and consistency of the older evidence, and providing updated references, if possible.
Rationale:
- This maintenance measure meets all criteria for 'Met' for importance due to the significance of the problem it addresses its robust evidence base, a documented performance gap, and well-articulated logic model, making it essential for addressing patient satisfaction. There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient outcomes, and justify use of the measure.
Closing Care Gaps
Strengths:
- None identified.
Limitations:
- The developer referenced analyses performed using CoreQ survey data collected in 2021. The developer stated they found that average measures scores for Black residents were lower than for white residents, and that this difference may have been due to between-facility differences in care rather than within-facility differences in care. However, the developer did not describe the statistical method used or report statistical results to support their statements. The developer did not provide recommended actions entities can take to close care gaps, such as targeted quality improvement initiatives or policy changes, if relevant.
Rationale:
- The rating for Closing Care Gaps is 'Not Met' due to insufficient information provided. While the developer attempted to assess gaps in care across race subgroups, statistical methods and results for their analyses were not reported.
Feasibility Assessment
Strengths:
- All required data elements are routinely generated during care delivery, and required elements are available from digital or electronic sources.
- The developer indicated there have been no changes to the measure specifications. The developer stated that no feasibility issues were found requiring adjustment of the final measure specifications.
- The developer described the costs and burden associated with data collection and data entry, validation, and analysis. They discussed current barriers that could be encountered in implementing or reporting the measure, which include collection burden and cost burden. They noted that the cost burden is very minimal and that CoreQ can be added to the beginning of existing surveys to reduce adoption burden.
- The developer described how all required data elements can be collected without risk to patient confidentiality because surveys are anonymous and scores are only calculated with 20 or more survey returns.
- Any fees, licensing, or other requirements to use any aspect of the measure (e.g., value/code set, risk model, programming code, algorithm) are clearly described and justified.
Limitations:
- There is a small cost and collection burden, but these are minimal and mitigations are addressed by the developer.
Rationale:
- This maintenance measure meets all criteria for 'Met' for feasibility due to its well-documented feasibility assessment, clear and implementable data collection strategy, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world health care setting.
Scientific Acceptability
Strengths:
- The developer describes signal-to-noise reliability estimation which is needed for this maintenance measure.
Limitations:
- The developer did not provide sufficient details about the data used for accountable entity-level reliability testing. Based on the submission, the developer seems to have estimated signal-to-noise reliability for each decile of entities ranked by performance score rather than estimate signal-to-noise reliability for each entity. The percentage of entities which meet the expected threshold of 0.6 for signal-to-noise reliability cannot be determined from this submission.
Rationale:
- This maintenance measure is rated as ‘Not Met But Addressable’ for reliability because the description of the data used for accountable entity-level reliability was insufficient and the reliability metrics provided by the developer do not allow for determining whether established thresholds were met. However, the identified limitations are deemed addressable, as the developer may consider providing details about the data used for accountable entity-level reliability and applying the signal-to-noise reliability test method to each entity and then updating Tables 2a and 2b with the mean reliability of entities within each decile. By addressing these issues, there is potential to meet the reliability requirements.
Strengths:
- The developer performed the required validity testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) validity testing at the level for which the measure is specified. The data used for the validity analysis was collected from a national sample of 536 facilities (n=16,021 family members and caregivers of long stay residents) in January to December 2025, and included the CoreQ Short Stay Questionnaire, CMS Nursing Home Compare measures and the CMS Five-star rating, and selected Certification and Survey Provider Enhanced Reporting (CASPER) quality indicators. Note that testing results based on data collected before 2020 cannot be considered in the rating.
- The developer hypothesized moderate positive correlations between the CoreQ Short Stay measure and the five other measures for which 2025 results are reported in Table 5.4.3b in the "Additional Validity Testing" attachment, including "Staff Stability" (r=0.28, p<0.001), "Agency Staffing" (r=0.45, p<0.001), "CNA hours per resident day" (r=0.33, p<0.001); "Staff Turnover" (r=0.41, p<0.001); and Five-Star rating (r=0.34, p<0.001).
- A thorough, well-developed logic model supports an inference of validity for this measure.
Limitations:
- The developer reported only the range of estimates for the correlation analyses of the measure with CASPER quality indicators, and the specific CASPER indicators included in the analysis were not named or described. In addition, hypotheses and supporting background regarding the expected direction and strength of each correlation observed is required to understand the meaning of any results.
- Literature was cited to support expectations for "moderate correlations" between measures, but references provided are from 2003 and 2010, and the specific measures the cited studies reported on are not provided.
- Note:
- The developer referenced two other measures (community discharge, rehospitalization) and provided correlation results in the text, but these results use data collected in 2014 and are not considered in the rating.
- Several tables referenced in the submission that provide results using 2014 data, including 5.3.4a, 5.3.4c-g, 5.3.4i, and 5.3.4j, are not included in the attached validity testing document.
- The face validity testing described appears to refer to psychometric testing of the instrument, rather than accountable-entity level testing of the measure score itself. Person- or encounter-level testing of the measure is not required for a maintenance submission, and is not considered in the validity rating.
- The developer did not conduct risk or case-mix adjustment or stratification. The developer provided the rationale that a risk adjustment approach at the patient-level was not possible, as surveys are not linked directly to patient characteristics, and use of facility-level metrics could be misleading. The developer examined average survey scores for one question by appropriate case-mix factors and noted minimal variation in measure score by case-mix factor.
Rationale:
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on quality and can distinguish good from poor performance to a limited extent.
- The developer did not conduct risk adjustment or stratification, but provided a reasonable rationale for why and supported the rationale with literature and basic analysis.
Use and Usability
Strengths:
- The measure is currently used in the Quality Incentive Payment Program; New Jersey’s Division of Aging and the Division of Medical Assistance and Health Services, the Division of TennCare’s Quality Improvements in long Term Services and Supports, LTC Trend Tracker, and American Health Care Association and the National Center for Assisted Living (AHCA/NCAL) National Quality Award Program.
- Attributes of a suitable program for this measure are described, and these include a target population is all residents in long-term care, that includes Medicare, Medicaid and private pay.
- The developer provided a summary of how accountable entities can use the measure results to improve performance. Specifically, benchmarking and monitoring data to measure family needs and wants, satisfaction, and staff competency.
- Feedback is gathered via committee and workgroup meetings and scoped for feasibility and cost. Some feedback is used as an opportunity to educate vendors about best practices in survey collection and administration. Feedback has also been used to make updates to the interface for the API.
- Pre-pandemic, the mean performance score was 83.5%, which dipped to 68.3% during the early pandemic years and post pandemic, has risen to 75.9%. The developer provided a clear rationale for these changes, noting that the pandemic drove significant reductions in staffing and implementation of strict isolation protocols.
- The developer reported no unexpected findings. The developer noted that the satisfaction questionnaire can highlight poor care for some dissatisfied patients, and this may make those patients further dissatisfied.
Limitations:
- No specific timeframe for performance data was given. The most recent number of reporting facilities is less than half of the number in earlier reporting periods, which could potentially affect mean ratings through selection bias. The submission could be strengthened with a discussion of any potential effects.
Rationale:
- This maintenance measure is rated ‘Met’ for use and usability because it is actively used in at least one accountability application, with a systematic feedback approach that allows for continuous updates based on stakeholder feedback. The measure also demonstrates a positive trend in performance results, affirming its ongoing usability. The developer reported no unexpected findings.
Committee Independent Review
Generally Concur with Staff Assessment
Importance
Literature is primarily older, as noted by PQM staff. In addition, citations focus almost exclusively on resident satisfaction, rather than family satisfaction. Would be helpful to speak directly to how families and others value this outcome and also share any findings about the relationship between the resident and family ratings. Also, lacking date for 40-person focus group on measure importance.
Closing Care Gaps
Concur with staff that submission only partially illustrates care gaps and does not address how to close them. Note that this domain is not required.
Feasibility Assessment
Concur with staff assessment.
Scientific Acceptability
Lack of information on dates associated with specific testing activities is challenging for review - suggest developing a timeline or other exhibit that ties together the various cited testing activities. This incudes clearly distinguishing test/re-test and re-administration efforts.
Concur with staff and note comment above regarding dates/timeline. Also, risk adjustment discussion focuses on resident characteristics and does not address respondent characteristics for this measure that could impact measure results.
Use and Usability
Concur with staff assessment. Submission would benefit from specific examples of how data have been used in QAPI or other quality improvement efforts.
Summary
Generally concur with staff assessment but suggest more/different evidence that reflects who is providing the rating (i.e., family/representatives and not residents). This is not a proxy measure but a different construct than 2615.
Not Met but addressable
Importance
This is an important metric, as family members are often the spokepersons for residents in long term care facilities. I think the logic model is sufficient, but as with measure 2614, I think it is basic and could be more robust. There is more to satsifaction with the facility and the staff than staff training and staff competency. There is evidence of a care gap in lower deciles. The top 3 deciles are all 100%, however the bottom 4 deciles are below mean, demonstrating continued room for improvement.
Closing Care Gaps
This is not required but I do think the developer provided a thoguhgtful reply with supporting data. They made a case for why risk adjustment based on race might have unintended consequences
Feasibility Assessment
The survey is only 3 items, all surveys are anonymous and patient confidentiality is protected. The cost of survey admisntration is provided, however it is not clear to me who adminsters the survey - the facility or a vendor or both.
Scientific Acceptability
defer to staff preliminary assessment
defer to staff preliminary assessment
Use and Usability
The survey is currently in use in NJ and Tennesse, as well as by the AHCA/NCAL National Quality Program. The survey results are easy to interpret and the developers report that the survey results are used for internal QI. I think that a more devleoped logic model may help clarify thestructures and processes that facilities can target for QI.
Summary
Overall I support this measure; all comments are constructive in nature
Public Comments
No public comments
No public comments