Satisfaction with Quality of Care - Toolkit

A 1988 report from the Office of Technology Assessment noted that outcomes of quality medical care include both changes in health status and satisfaction (U.S. Congress 1988). Donabedian stated that “achieving and producing health and satisfaction, as defined for its individual members…is the ultimate validator of the quality of care” (Donabedian 1966). Quite surprisingly, there has been limited research to date in examining satisfaction among terminally ill patients. For some dying patients, if not the majority, satisfaction may be the most important outcome variable for themselves and their families (Teno 1996).

A literature review of the Medline database was conducted. Given the scarcity of instruments found specific to measuring satisfaction among terminally ill patients, we reviewed available measures of satisfaction with ambulatory care and hospitalization to see if existing instruments would be applicable to a dying population.

Based on a review of these 153 articles, we found 20 potential instruments for consideration which are listed below:

A self-completion satisfaction questionnaire evaluating the standard of care of HIV outpatient services (Beck et al 1999). This instrument has 28-items consisting of multiple choice questions, questions on demographics and some open-ended questions.

Bereaved family members perceptions of end of life care (Hanson et al. 1997) - this instrument uses open-ended questions to assess informant's perceptions of treatment decisions, treatment intensity, and the effectiveness of symptom relief. (This instrument has not been tested for reliability or validity).

Cancer patient satisfaction with care (Wiggers 1990) – a 60 item scale which measures technical competence, communication skills, interpersonal skills, accessibility of care, continuity of care, non-medical care, finances, family care and hospital/clinic care.

CASC Comprehensive Assessment of Satisfaction with Care (Bredart et al. 1998) - is a self-administered questionnaire that assesses technical quality, interpersonal manner, communication skills, coordination, waiting time, continuity availability, access, and physical environment. (This has not been tested for validity and reliability).

FAMCARE (Kristjanson 1986, 1989, 1993) – a scale developed based on qualitative research that asked family members to list indicators of quality of palliative care from their perspective and the patient’s perspective. This list was shortened to a 20 item scale based on a Q-sort and examination of the psychometric properties.

Medial Interview Satisfaction Scale (Wolf 1978) – this scale was used in the evaluation of the National Hospice Study.

National Hospice Organization Family Satisfaction Survey (NHO 1996) – an 11 item survey that asks about satisfaction with aspects of hospice care.

Need satisfaction scale (Dawson 1991) – this scale consists of nine items based on Hampe’s research on the needs of a grieving family (Hampe 1975).

NY satisfaction of hospice survey (Hannan 1984) - a questionnaire developed to assess the primary caregiver’s satisfaction with the patient’s emotional and physical quality of life, and the support and assistance offered to the caregiver.

Patient Judgment System (Nelson 1989) – a 68 item scale that examines 11 aspects of hospital care.

Picker-Commonwealth Survey of Patient Centered Care – (Cleary 1991) a 61 item scale that identifies the patient’s experience of hospitalization based on patient specific reports (e.g., “Were you told of the purpose of your medications in way that you could understand?”). This scale has yet to be used in a terminally ill population. The scale has been used in a national survey in which about one in five patients reported their health to be poor. A modified hospice care-specific version (Tierney et al. 1998) has been used in a study which looked at the relationships between symptom relief, quality of life, and satisfaction with hospice care.

PMH/PSQ-MD Princess Margaret Hospital Patient Satisfaction with Doctor Questionnaire (Loblaw et al., 1999) – is a 29-item self-administered questionnaire that measures information exchange, interpersonal skills, empathy and quality of time.

Postal questionnaire to examine carer satisfaction with palliative care (Jacoby et al., 1999) – is an 89-item questionnaire that assesses information received from health professionals, community care and GP care, hospice and hospital care (including symptom control), provision of equipment, decedent's attitudes/preferences over place of death, carer’s post-bereavement health and support, as well demographic details of the deceased.

QPP Quality of Care from the Patient's Perspective (Larsson et al. 1998) - is a 64-item instrument measuring medical-technical competence, physical-technical conditions, the identity-orientated approach, and socio-cultural atmosphere. This instrument is a revised version of the original QPP (Wilde et al 1994) which contained 54 items.

Quote-CNSLD instrument (Van Campen et al. 1997) – is a 32-item instrument that identifies indicators of quality of health care for patients with chronic nonspecific lung diseases. (A version for elderly patients has been submitted.)

Satisfaction of families of children with cancer (Barbarin 1984) - a measure of seven dimensions of staff behavior: information transmission, clarity and honesty of communication, acceptance of parental efficacy, resolution of conflicts, personal contact with parents, empathy with the child, and staff competence.

Satisfaction Survey from Ware and colleagues (Davies 1988, Ware 1978, Tarlov 1989). Since the early 1970s, John Ware and colleagues have measured patient satisfaction in terms of the physical environment, access to care, technical aspects of care, and interpersonal care. These scales have been modified by McCusker (1984), Kane and colleagues (1984), and the SUPPORT Investigators (Baker 1996) to examine satisfaction in terminally and seriously ill patients. Westra modified it to examine satisfaction with home care (Home Care Client Satisfaction Instrument, Westra 1995).

Toolkit After-death Bereaved Family Member Interview (Teno et al. 2001) - a retrospective telephone-administered survey with bereaved family members. The instrument includes seven domains of care (physical comfort and emotional support, inform and promote shared decision-making, encourage advance care planning, focus on individual, attend to emotional and spiritual needs of family, provide coordination of care, and promote self-efficacy of family caregivers). The instrument's design emphasized face validity for the purposes of quality improvement.

Unmet needs of patients with cancer (Houts 1988) -a scale which asks family members about unmet needs in 14 different domains.

Based on review of these 20 instruments, I have identified four for further consideration.

i. Conceptual and Measurement model (Does the scale represent a single domain or do model scales measure distinct domains? Is the variability of the scale reported? If so, please document it. What is the intended level of measurement (i.e., ordinal, interval, ratio or category?)

This is a self-administered questionnaire (also can be interviewer administered) that measures various aspects of satisfaction based on the theory that satisfaction is influenced by the structure of the organization, accessibility and availability of care, the technical quality of care, communication, financial arrangements, time spent with providers and interpersonal aspects of care. It is an ordinal scale with ranges reported on various populations.

ii. Reliability (Did they address internal consistency? Did they address reproducibility?)

There are conflicting results among patients with a terminal illness (McCusker 1984; Wales 1983). Wales and colleagues found a satisfactory internal reliability of .72 and higher. On the contrary, McCusker’s modified version of the scales had a reliability between .40-.69 among a terminally ill patient population.

In a seriously ill population, Marilyn Bergner modified these scales to 21 items that measure technical aspects of care, satisfaction with interpersonal aspects of care, and privacy. This was later modified for a second phase of this study because of missing data and factor analyses performed on the phase I data. An investigator suggested there were three factors (personal communication factor, nurse factor, and physician factor), composed of ten items. However, this was not confirmed by a factor analysis on the phase II data. Rather, one factor was indicated as the best solution to explain 66% of the variance for patients and 65% of the variance for surrogates. These 10 items were used in Phase II of the SUPPORT study as a safety check. The internal consistency was high (.81-.94) among a sample of seriously ill patients with the entire scale having a Crohnbach’s alpha of .94.

iii. Validity – (How did they address content related validity? Any information on construct related validity? Any information on criterion validity?)

McCusker (1984) tested discriminant and convergent validity. She found that availability of care had poor discriminant validity, yet convergent validity was satisfactory. Otherwise, her adapted versions of continuity of care and physician competence performed satisfactorily. She does recommend combining personal qualities of the physician and communication with the physician into one scale.

In an ambulatory population, Ware and colleagues (1983) have assessed the construct related validity.

iv. Responsiveness (Any information? Has the scale ever been used as an outcome measure? If so, what populations?)

It has been used as an outcome measure in Kane and colleagues’ (1985) study of hospice. Satisfaction with interpersonal care was one of the few outcome variables that revealed a benefit from hospice.

v. Interpretability (What populations has it been applied to? Is the score translated into a clinically relevant event? Does the score predict outcome events?)

One of the important limitations of satisfaction scales is that scores have not been translated into clinically relevant events. For the most part, patient responses are skewed with most patients responding either excellent or very good.

vi. Burden (Any information on cost or time to administer? Does the instrument impact on the respondent? How long does a survey take to complete? Response rates? Any problems with missing data?)

Depending on number of subscales, this is a self-administered instrument that can be easily completed.

vii. Alternative forms (What are the modes of administration? Alternatives? If alternative exist, provide what is known for each of the above categories)

i. Conceptual and Measurement model (Does the scale represent a single domain or do model scales measure distinct domains? Is the variability of the scale reported? What is the intended level of measurement (i.e. ordinal, interval, ratio, or category?)

The TIME Surveys (including the Afterdeath Bereaved Family Member Interview) are based on the conceptual model of patient focused, family centered medical care. Patient focused, family centered medical care includes four key components: providing the desired physical comfort and emotional support for the patient, promoting shared medical decision-making, treating each person as an individual by understanding his/her needs and expectations, and attending to the needs of those that care for and love the dying person. As tools for quality improvement, the TIME surveys provide health care providers with "problem scores" (which indicate opportunities to improve quality of care) as well as scaled ratings.

The Afterdeath Bereaved Family Member Survey was designed to measure the family member’s perspective regarding the extent to which a health care institution met the needs and expectations of a dying person and his/her family members. The instrument contains seven domains (physical comfort and emotional support, inform and promote shared decision-making, encourage advance care planning, focus on individual, attend to the emotional and spiritual needs of the family, provide coordination of care, and support the self-efficacy of the family) and a set of overall rating questions. As a result of the several different types of questions included in the tool, there are several types of response categories, including:

ii. Reliability (Did they address internal consistency? Did they address reproducibility?)

Internal consistency of each of the survey’s domains was tested and found to be acceptable based on psychometric testing. Two problem scores that demonstrates borderline acceptability. Each of these problem scores contains only three or four items. According to the authors, the choice to use so few questions for these problem scores was justified in order to minimize response burden and maintain patient centered reports as the question structure.

Domain/Dimension	Number of Items	Cronbach’s Alpha
Informing & Decision-making	8	.77
Advance Care Planning	3	.58
Coordination of Care	4	.68
Control and Respect	6	.80
Physical Comfort	5	Not available*
Surrogate Emotional Support	4	.58
Self-Efficacy of Family	3	.74
Overall Ratings	5	.87

Test-retest reliability has been addressed by collecting data from 30 family members approximately one month after the initial testing of the survey. The Kappa and ICC statistics revealed evidence of stability of the reported responses.

iii. Validity (How did they address content related validity? Any information on construct related validity? Any information on criterion validity?)

Content validity is supported by the process of generating domains and questions. Domains were identified from a review of expert guidelines and an analysis of focus groups conducted with bereaved family members from different geographic regions in the U.S. and from various settings of care. Survey questions underwent cognitive testing and review by an expert panel.

Construct validity was evaluated by examining the correlation between a problem score (higher score meaning more opportunity to improve) and its corresponding overall rating measure (higher score meaning better care). Criterion validity was evaluated by examining the correlation between a problem score and the overall rating of satisfaction with care (higher scores for both meaning more opportunity to improve).

Domain/Dimension	Construct Validity	Criterion Validity
Informing & Decision-making	-.59	.51
Advance Care Planning	-.37	.24
Coordination of Care	-.42	.46
Control and Respect	-.45	.47
Physical Comfort*	Not available*	Not available*
Surrogate Emotional Support	-.59	.45
Self-Efficacy of Family	N/A	-.45
Overall Ratings	N/A	-.52

iv. Responsiveness (Any information? Has the scale ever been used as an outcome measure? If so, in what populations?)

Information is not yet available.

v. Interpretability (What populations has it been applied to? Is the score translated into a clinically relevant event? Does the score predict outcome events?)

Problem scores translate into a clinically relevant event in that they quantify opportunities for health care institutions to improve. Higher scores indicate more opportunity to improve. National norms for the problem scores will be available in by Winter, 2001.

The instrument has been used with bereaved family members from three sites of care: hospitals, outpatient hospice services, and nursing homes. Preliminary psychometric testing data was based on family members from the New England area. The instrument is currently being used with a nationally representative sample, and data from this study will be available by Winter, 2001.

vi. Burden (Any information on cost or time to administer? Does the instrument impact on the respondent? How long does a survey take to complete? Response rates? Any problems with missing data?)

The full instrument contains 133 questions, but the instrument has been shortened and tailored for specific services and sites of care (hospice, hospital, nursing home). In addition, health care institutions can further shorten the instrument by choosing specific domains upon which to focus.

The full instrument was used for validity and reliability testing. For this survey, the average length of time to complete the survey was 28.5 minutes (median: 25.8 minutes; range: 14-70 minutes). The response rate was 63.4%, when those potential respondents we were unable to contact or locate were excluded from the denominator. The majority of interview respondents had either a positive (60.9%) or very positive (14.7%) response to being interviewed, with only 7 respondents having a negative experience according to the interviewer. No problems with missing data were reported.

vii. Alternative Forms (What are the modes of administration? Alternatives? If alternatives exist, provide what is known for each of the above categories?)

An interviewer administers the instrument, either by telephone or in-person. A Scoring Packet is available from the author; the Packet includes a pre-formatted Microsoft Excel Worksheet and an accompanying Scoring Guide. In addition, a computerized survey program for use with a palm pilot and an accompanying web-based interface for analysis are being developed.

viii. Cultural and Language Adaptations (Any information?)

The instrument has been developed and tested among English-speaking families in the U.S.

The After-death Bereaved Family Member Interview is a valid and reliable survey tool. The problem scores clearly indicate areas in which a health care institution has opportunities to improve. As a result, problem scores are actionable and, therefore, facilitate quality improvement efforts.

The theoretical framework for this scale is a model of describing constructs, concepts, and measurement tools created by Gibbs (1972). The original items were based on qualitative research with families of patients with terminal illness who were admitted to a palliative care service. In phase II, they performed a Q-sort (a ranking tool) of the 74 items regarding care of the patient and 77 items concerning care of the family. Families were recruited from a palliative care unit, home care, and an acute care hospital. Only family members of patients with cancer were interviewed. Based on this ranking, the author came up with 20 items for the FAMCARE scale.

Factor analysis suggested that there were four separate subscales – availability of care, physical patient care, psychosocial, and information giving.

ii. Reliability (Did they address internal consistency? Did they address reproducibility?)

Internal consistency has been found to be high (.93) among a population of cancer patients and test-retest agreement was high, as well (.91).

iii. Validity – (How did they address content related validity? Any information on construct related validity? Any information on criterion validity?)

The correlation with the McCusker (1984) instruments was found to be .80 and .77.

iv. Responsiveness (Any information? Has the scale ever been used as an outcome measure? If so, what populations?)

The scale was used as an outcome measure in Jarvis and colleague’s study of 84 hospice patients.

v. Interpretability (What populations has it been applied too? Is the score translated into a clinically relevant event? Does the score predict outcome events?)

vi. Burden (Any information on cost or time to administer? Does the instrument impact on the respondent? How long does a survey take to complete? Response rates? Any problems with missing data?)

The scale is a short 20 item questionnaire that is interviewer administered with a mean data collection time of 22 minutes.

vii. Alternative forms (What are the modes of administration? Alternatives? If alternative exist, provide what is known for each of the above categories)

Concerns over the validity of patient reports of satisfaction led Cleary and colleagues (Cleary 1991) to create a patient-centered report of the experience of hospitalization. This 61 item self-administered questionnaire aims to collect data on specific events in way such that personal relationships and expectations do not influence the measure of satisfaction.

ii. Reliability (Did they address internal consistency? Did they address reproducibility?)

No information was available in the reviewed articles on internal consistency, but the authors stated that questions measured 7 specific domains.

ii. Validity – (How did they address content related validity? Any information on construct related validity? Any information on criterion validity?)

The count of the number of problems (they averaged a problem score in each of these 7 areas) was the strongest predictor of patient evaluation of care (a scale modified from existing instruments).

iii. Responsiveness (Any information? Has the scale ever been used as an outcome measure? If so, what populations?)

This is ongoing at this time. At one hospital, a PCA program was instituted in response to patient reports that pain was inadequately controlled, and subsequent patient surveys showed that the program was successful (Cleary 1991).

iv. Interpretability (What populations has it been applied to? Is the score translated into a clinically relevant event? Does the score predict outcome events?)

This instrument, which identifies specific problem areas, has a strong face validity from a clinical standpoint. It has yet to be used in a dying population.

v. Burden (Any information on cost or time to administer? Does the instrument impact on the respondent? How long does a survey take to complete? Response rates? Any problems with missing data?)

This is a self-administered questionnaire that takes about 15-25 minutes on average to complete.

vi. Alternative forms (What are the modes of administration? Alternatives? If alternative exist, provide what is known for each of the above categories)

There was no one scale that emerged as a final recommendation. If researchers were going to use an instrument off the shelf, I would recommend the FAMCARE scale. However, I have some reservations about this recommendation. First, the response scale of ‘very satisfied’ to ‘very dissatisfied’ has been criticized by Ware and colleagues (1978). Second, several items may not be applicable or as relevant to a dying population (e.g., the time to make a diagnosis). Third, there have been important concerns with ranking that are applicable with this scale in that the means are equivalent to the majority of patients stating they were “very satisfied.” Fourth, the FAMCARE has been used only among cancer patients. An alternative is to consider using subscales of a satisfaction instrument developed by Rand and the Health Care Institute. I would recommend the subscale that measures interpersonal care. The problem is that answers are skewed as with the FAMCARE scale. Additionally, there are important measures such as comfort and decision making which should be addressed that were found to be problematic in the SUPPORT study (Baker 1996). Questions used in the SUPPORT project about decision making included:

If the doctors and nurses had told us more about (PATIENT'S) condition during the hospitalization in (MONTH OF INDEX HOSPITALIZATION), we would have made different decisions about (his/her) care.

The decisions we made about (PATIENT'S) care were carried out by the (NAME OF HOSPITAL’S) staff as much as possible.

(Note: these priorities are based on an article that I wrote for the Institute of Medicine).

Recommendation No. 1 - Make better use of family member reports and ratings of medical care. A family member is not only a proxy for the patient. Rather, we care for the patient in the context of both a formal and informal care giving network. Measures must be developed which examine family members’ perspectives on medical care.

Perhaps as many as one in two persons age 85 and older will have a substantial cognitive deficit. Such impairment will often limit their ability to report on and rate their medical care. In interviewing the seriously ill, we found that nearly one half of the respondents were not able to be interviewed about their medical care (Teno 1995; SUPPORT Investigators 1995). Those patients unable to be interviewed had a greater disease burden and were more likely to have decisions made about life sustaining treatment. If we had relied on patient interviews, we would have had a biased picture of medical decision making for seriously ill dying patients. The use of proxy data is important. However, the proxy cannot accurately report the patient’s experience (of course, the patient may not accurately report on it, either). Highly subjective patient experiences (e.g., report of pain) bring this concern into sharper focus. Various strategies have been suggested, such as calibrating the surrogate responses.

However, it is my and others’ argument that the family report and ratings about medical care should be an independent outcome of interest (Gerteis 1993); one that is not solely meant to be calibrated to estimate a patient response and certainly not one to be ignored. A family member who states that a loved one was in severe pain may be responding to physical signs which are not indicative of severe pain. However, I am somewhat troubled with the response that this perception should be merely cast away as inaccurate reporting, or be calibrated. A patient’s death is the last memory of her loved one. The memory that the patient died in severe pain is problematic, and might be remedied with good communication and education about dying. We should respond with concern to such reports. Of course, we need to understand their meaning. Research is needed to understand how both the patient and family member view the illness experience. Even if there is low correlation of patient and family views, the family members’ reports and ratings of care are important quality outcome measures. Geriatric and palliative care medicine aim not only to treat a patient, but to provide compassionate care for the patient and their family (WHO 1989).

Recommendation No. 2 – Future research should reconsider the meaning of patient item responses (i.e., whether the patient responded “excellent” or “very good”) and total aggregate scale scores. For the purpose of continuous quality improvement, we may need to focus on individual item responses and not solely on the total aggregate scale score.

Ware and colleagues noted that patients differ in their reported satisfaction with medical care based on whether the response scale was “poor to excellent” or “very dissatisfied to very satisfied” (Ware 1983). Indeed, he and colleagues took the lead in recommending the use of the “poor to excellent.” The impetus for the use of this 5-item response was based on the experience of the airline and hotel industry. Distributions on such scales are often skewed to the very good and excellent response. Many reports lump these two categories together and note that patient satisfaction was high. A fundamental assumption is that the difference between excellent and very good responses is known and small. As noted by Fowler, we might be underestimating the difference (or the large gulf) between very good and excellent.

Nelson and colleagues have reported the correlation between satisfaction and whether or not the patient would recommend that health care institution or return to the same facility (Nelson 1989). It would be of interest to examine how many of those who responded “excellent” would return compared to how many of those responded “very good.” “Very good” may well indicate that the patient is dissatisfied. Indeed, we may need to view satisfaction responses as the “tip of the iceberg” – getting a “very good” response may be an early warning sign and an opportunity to improve. A “good” response indicates a major problem that should be addressed with further work. In a study of seriously ill patients, we found that nearly 70% of family members were satisfied (i.e., they responded agree or disagree to all items in the correct direction regarding statements about their satisfaction with the patient’s comfort and their involvement in medical decision making). Is the glass half full or half empty? Some have noted that these results indicate that the majority of family members were satisfied with patient comfort and decision making in the last week of life (JAMA Contempo 1996). However, we found that nearly one in three were dissatisfied with one or more aspects of care. This offers an important target for improvement.

Already, cognitive research or “cognitive testing” of survey respondents is being undertaken to better understand the process in which patients respond to questions about the rating of medical care (e.g., see Research Triangle Institute final report to AHCPR Contract No. 282- 92-0045, delivery order No. 9, deliverable No. 7). Such research is important. Equally important is that we expand and further develop conceptual models of the cognitive process that one undertakes in determining whether one is satisfied with health care or not. To date, researchers have drawn on job satisfaction and marketing research models to help develop these models. We need to expand on this work to better understand the cognitive process of determining satisfaction with health care.

Additionally, we need to better understand the degree to which measures (or total satisfaction scale scores) differentiate among various health care plans. Ultimately, one use for satisfaction scores would be to allow consumers to understand the average medical care experience in various health care plans, thereby enabling an informed choice of a health care plan. In buying a dog, a multimedia program allows the user to rank several factors including shedding, obedience, how the dog behaved with children, etc. and then the program provides the breeds that most closely match stated preferences. Can we develop a similar program for health care that would let me choose between health care plans that behave as a “pit bull” compared to my preferred breed of a reliable and ever so friendly “labrador retriever”? Consumers are going to need to have such decision aids to help them sort through the data and make choices consistent with their own values and preferences (it should be noted that work needs to be done on stability of such preferences and whether such a match of scores and preferences does indeed improve and enhance the consumer experience-- i.e. is it really a good decisional aid?).

Having meaningful scale cut-points is important to differentiate between health care plans. For the purpose of CQI, we should examine individual items (or sub-scale scores) and not just rely on the total scale score. Similar to the 30 point Mini-Mental Status Exam, both the overall score specific deficits are important in understanding and addressing a patients’ cognitive deficits. Similarly, in the year 2006, we will need to be able to track both the total score and monitor individual items to ensure that quality care is being given as well as to improve care. Data will need to be displayed graphically over time so that health care providers can appropriately track performance and respond to the need for early intervention. Perhaps the wise health care plan will utilize a screening instrument and, when needed, employ both focus groups and a more comprehensive survey when a problem or slip in performance is noted. The ultimate goal will to be better understand of and prompt response to patient concerns.

Recommendation No. 3 – Both patient (and family) reports and ratings of care are important. Strong consideration should be given to use of both types of measures in assessing the quality of medical care.

Two of the most extensively developed and tested instruments to measure satisfaction rely on patient ratings of their medical care (Davies 1988, Rubin 1993). Concerns with patient ratings about their medical care led Cleary and colleagues to develop a 61 item survey about the patient’s experience of hospital care that was based solely on patient report of specific events (e.g., “Were you told of the purpose of your medications in a way that you could understand?[Cleary 1991]). The authors argued that they framed questions “to be as specific as possible, to minimize the influence of confounding factors such as the patients’ expectations, personal relationships, gratitude, or response tendencies related to gender, class, or ethnicity” (Cleary 1991). This scale, which examines hospital care from the perspective of seven domains (ranging from respect for patients’ values, preferences, and expressed needs, coordination and integration of care, physical comfort, to transitions and continuity of care) was developed based on focus groups and expert panel review. From the perspective of a clinician, these measures strike accord and have strong face validity. However, I still must argue that we need both patient rankings and reports. Consider the following conversations about preferences for continued use of life sustaining treatment:

Physician A: Let me tell you if we remove the breathing tube, you will die! Do you want to die?

Physician B: Mrs. M, there are several options that we have. Your views are important in deciding the best treatment decision about continuing on the ventilator, the machine that is helping you breath now. OK… Let me tell you about your condition and then we will discuss what the options are…

A patient or family member should report quite factually that in both cases the physician discussed whether life sustaining treatment should be utilized. Yet, the patient’s or family member’s degree of satisfaction with those conversations may be quite different (it should be noted that I am making an assumption which should be questioned and tested -- that patients actually prefer a shared decision making approach). An important question is whether patient’s or family member’s ratings of a provider’s interpersonal and communication skills can accurately discriminate between an overly coercive communication style and a more egalitarian approach. Research should take on this issue. We may again need to borrow from the hotel industry and examine new response scales regarding whether or not the communication exceeded the patient’s expectations. It is quite possible that patients simply have low expectations. We may need to intensely “listen” to a random sample of patients on a routine basis about how treatment decisions were made.

Delbanco and colleagues used a similar approach in developing an instrument which measures patient centered hospital care (Delbanco 1992). A survey response is an efficient means to measure a construct across large groups of persons. However, the patient’s own words provide valuable information to guide and reform health care. I suspect that successful health care institutions will learn to regularly “listen” to patients and incorporate insights from patient stories in creating innovations and better health care outcomes.

If none of these are proven valuable, I would argue that strong consideration should be given to development of other measures, such as simulated patients or routine recordings of physician-patient (-family) conversations to measure whether patients (and/or families) are sensitively and skillfully communicated with about their options in making decisions about important treatment decisions.

My example above involved a situation in which the communication and interpersonal skills of the provider had an important and immediate effect on the outcomes of care. An important research question is whether to do sampling for those situations where communication about treatment options is especially important to the outcomes of care (e.g., treatment of prostate cancer in older persons or breast cancer treatment options). It is my recommendation that we consider routinely targeting such populations. With such targeting, there is a real downside in that the survey costs will increase depending on how hard it is to locate such individuals. However, I would argue that the most important measures of the quality of medical care need to be based on those patients who need medical care the most. One such example is the work reported by Fowler and Bin, in conjunction with the Prostate Cancer PORT. They have developed a survey which examines both patient report and rating about their medical care (Fowler, In Press). Further work should be undertaken for other diseases and a key question is whether such measures are sensitive and discriminating enough to guide quality improvement and differentiate between interventions to enhance the care of these patients.

Decades of work have gone into the development of patient ratings and reports of their medical care. Many are urging the standardization of satisfaction measures. To a certain extent, I would agree. My main reservation is that such standardization does not stop the necessary innovation needed for CQI. Berwick recently characterized this as the need for “real time science” (Berwick 1996). In the future, we must be ready to change and adapt measures to the ever changing health care market. Also, we must periodically step back and openly question the fundamental premises of medical care. For example, modern bioethics has assumed that patients want to have control or participate in decision making. Recent research has questioned whether this principle holds true.

Recommendation No. 4 – Consideration should be given to the development of situation specific and disease specific measures (i.e., ratings and reports).

In part, I have already argued that we need to develop disease specific measures. We do need standardization of measures for the purpose of regulation and for consumers to compare health care plans. Yet, I believe that CQI demands the ability to develop measures that examine specific situations and the patient’s experience with a given disease. The work of developing reports and ratings about medical care for patients with prostate cancer is an important example. For older persons and their families, two other important areas-- the experience of the dying and the medical care of patients with dementia-- demand special attention. In a changing health care market, dying patients and those with dementia are extremely vulnerable.

Traditionally, health service research has relied on measures of mortality to rank quality of care. The implicit assumption is that a low mortality rate is the best possible care. However, over the past three decades, we have attempted to strike a balance in the use of life sustaining technology because for some patients life sustaining treatment prolongs dying and suffering. Dying is a profound event that impacts on the patient and stays with family members for the rest of their lives. Petrarch said, “ A good death does justice to an whole life.” With an aging population now slowly dying over a period of years, our health care system is going to have to become increasingly attentive to how we manage transitions in goals of medicine. Fundamental to both geriatric and palliative care medicine is that practitioners learn to skillfully negotiate a transition in goals of care, going from a major emphasis on extending life to medical care that focuses on patient comfort, even if one’s life is shortened. Based on research with colleagues in the Study to Understand Prognosis and Risks of Treatment, I suspect that we are currently timing this transition much too late in the disease course. Callahan has referred to this pattern of decision making as “technical brinkmanship” (Callahan 1990), in that we believe that we can master the timing of withdrawal so that treatment is only stopped when there is absolutely no hope. Based on findings among these seriously hospitalized ill, this pattern of decision making is not without important impact on the quality of care that patients receive while dying (SUPPORT, In Press). I would caution that changing the timing of this transition is not without significant and important risks. Moreover, as managed care organizations provide care for more older patients, there is a real risk that the timing of this transition will occur much too soon.

To date, several researchers have proposed both reports and ratings of the dying experience (Greer 1984; McCusker 1984; Kane 1984). Examining patient and family ratings and reports has been fundamental to Dame Cicely Suanders’ hospice program at St. Christopher’s Hospice. For example, Parkes reported that over a ten year period at St. Christopher’s Hospice, reports of severe discomfort have decreased and, at the same time, a smaller number of patients were reported by surviving spouses to have been very confused or unconscious in the last days of their lives (Parkes 1979). Tracking such information is important to ensure that we deliver quality compassionate care to dying patients and their families (or significant others). Yet, it is not now routine to collect such information in hospice, long term care, or among hospitals in the United States. Keay and colleagues proposed chart based quality indicators of medical care for terminally ill patients in long term care facilities (Keay 1994). Part of the inherent difficulty in establishing such indicators is defining the term “terminally ill.” Indeed, I suspect that a major indicator may be merely recognizing that the patient has a limited life expectancy and making appropriate plans. The authors establish their recommended domains based on a comprehensive literature review. I would urge that such measures or quality indicators be further developed, based on selective interviewing of seriously ill patients and family reports of the dying experience. Prior to settling on a set of domains, research utilizing focus groups, in-depth interviews, and expert panels will need to determine the domains that are important from the patient’s perspective. Such instruments should consider the use of patient and family-specific reports similar to those proposed by Cleary and colleagues (Cleary 1991). Additionally, such instruments should have patient and family rankings that provide information on interpersonal relationship of the patient, family members, and health care provider.

Dementia is a prevalent disease which has a devastating impact on patients and their families. As managed care enrolls more patients with Medicare, providers in managed care will need to develop systems of care for these patients. Dementia patients may not (and probably will not) fit easily into ten minute office visits. Health plans which do not appropriately provide families with assistance in caring for demented patients in the community most likely will inappropriately “discharge” such patients into long term care. With dementia patients, measures which examine the family caregivers experiences are important. A healthy caregiver is often one of the most important determinants of being able to maintain such a patient at home. With the changing health care market, an important danger is that patients and their families may not get appropriate social services from inexperienced or over-extended case managers. Such complicated patients can easily overwhelm the system and they may be encouraged to disenroll from the health plan. From the standpoint of regulation and CQI, it will be important to examine reasons for disenrollment and the family experience in caring for patients with dementia.

At this point in time, the number of patients with dementia and those dying in the majority of managed care organizations is probably small. For example, I am currently assessing an 80,000-person managed care organization in Washington, DC. Only about 80 persons die in a given year. From a standpoint of prudent business, it may not make sense that we should focus on such a small number of cases. This plan has yet to undertake a Medicare risk contract. I suspect that this rate will increase, but the dying will still represent a small constituency of the covered lives in managed care organizations. However, dying is such a profound human experience. Health care delivery systems should examine and achieve quality medical care at the end of life.

I should note that many make a strong argument that we will not be able to develop disease specific measures. Rather, we may need to adapt current satisfaction measures for “high utilizers”. Instead, I strongly urge that we target at least those persons who are seriously ill, those diagnosed with disease which implies a limited life expectancy, and that we develop family rating and report regarding the dying experience.

Recommendation No. 5 – Maybe we need to educate patients to expect more from their health care providers and health plans prior to the completion of surveys.

An important concern with patient ratings with medical care is the degree of discord with reports about care and rankings of satisfaction. In seriously ill patients, it has been noted that patients will report severe pain, and high levels of satisfaction with pain control. Why does this occur? Would the patients be satisfied after an educational intervention informed them that, in the majority of situations, patients need not suffer severe pain while dying? We need to consider educating patients about the health care system’s mission on quality medical care prior to their completion of satisfaction surveys. This recommendation may be best tested in focus groups and probably should not be targeted to domains in which outcomes are not well understood.

There are several issues (e.g., response rates, modes of survey administration, case mix adjustment etc.) that I have not highlighted. This omission should not be taken as a sign that they are not important. For the most part, my remarks have remained at the larger, conceptual level. However, these painstaking details, if unattended, will also have dire consequences for the use of patient reports and rankings of medical care.

Let me finally return to the claim for the older person measures of satisfaction may be a final common pathway for all health care outcomes. Over a lifetime, patient expectations of health care may change quite dramatically. A patient in the early decades of life may place more emphasis on technical competence. An elderly person with multiple ADL impairments probably has quite different values. For her, the degree to which her personal needs are addressed, that medical care emphasizes her comfort, and that she is treated with respect are of the most concern. For this patient, quality medical care is not indicated by current measures, like whether a screening mammogram was done. For her, outcomes measures that deal with comfort, dignity, and supportive services will be of paramount importance. Of course, older persons are quite heterogeneous. The current cut-off of defining one as “elderly” at the age of 65 is quite arbitrary (at least from a physiological standpoint). Many healthy 75 year olds have a primary emphasis and goals that are quite similar to a 40 year old patient. However, each of us probably reaches a point in an illness, or perhaps at a certain age, when what is important changes. In the coming decade, health care providers and plans need to learn how to best acknowledge and adept to these transitions in the goals of care. Successful health care plans must listen intensely and partner with patients to enhance and improve the care of older persons.

Baker R, Teno JM, Wu A, et al. Family satisfaction with end of life care. Manuscript under review. 1997.

Barbarin OA, Chesler MA. Relationships with the medical staff and aspects of satisfaction with care expressed by parents of children with cancer. J Community Health 1984;9:302-313.

Beck EJ, Griffith R, Fitzpatrick R, et al. Patient satisfaction with HIV service provision in NPMS hospitals: the development of a standard satisfaction questionnaire. AIDS Care 1999;11:331-343.

Beck-Friis B, Stang P. The family in hospital-based home care with special reference to terminally ill cancer patients. J Palliat Care 1993;9:5-13.

Berwick, DM. Harvesting knowledge from improvement. Editorial. JAMA 1996;275:877-878.

Bredart A, Razavi D, Delvaux N, et al. A comprehensive assessment of satisfaction with care for cancer patients. Support Care Cancer 1998;6:518-523.

Callahan, D. What kind of life: the limits of medical progress. New York: Simon and Schuster 1990.

Cleary PD, Edgman-Levitan S, Roberts M, et al. Patients evaluate their hospital care: a national survey. Health Affairs Winter 1991;254-267.

Creek, LJ. A homecare hospice profile: description, evaluation, and cost analysis. J Fam Pract 1982;14:53-58.

Davies AR, Ware JE. GHAA’s Consumer Satisfaction Survey and User’s Manual. Group Health Association of America: Washington, DC, 1988.

Dawson, NJ. Need satisfaction in terminal care settings. Soc Sci Med 1991;32:83-87.

Delbanco, TL. Enriching the doctor-patient relationship by inviting the patient’s perspective. Ann Intern Med 1992;116:414-418.

Donabedian, A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly: Health and Society 1966;44.

Fowler FJ Jr., Barry MJ, Lu-Yao G, et. al. Outcomes of External Beam Radiation Therapy for Prostate Cancer: a Study of Medicare Beneficiaries on Three Seer Areas. J Clin Oncol, In Press.

Gerteis M, Edgeman-Levitan S, Daly J, et al. Through the patient’s eyes: understanding and promoting patient-centered care. Josey-Bass Inc: San Francisco, 1993.

Greer DS, et al. Final Report of the National Hospice Study. Providence, RI: Brown University Program in Medicine, Center for Health Care Research, November 1984.

Hampe S. Needs of the grieving spouse in a hospital setting. Nurs Res 1975;24:113-120.

Hannan EL, O’Donnell JF. An evaluation of hospices in the New York State hospice demonstration program. Inquiry 1984;21:338-348.

Hanson LC, Danis M, Garrett J. What is wrong with end-of-life care? Opinions of bereaved family members. J Am Geriatr Soc 1997;45:1339-1344.

Houts PS, Yasko JM, Harvey HA, et.al. Unmet needs of persons with cancer in Pennsylvania during the period of terminal care. Cancer 1988;62:627-634.

Jacoby A, Lecouturier J, Bradshaw C, et al. Feasibility of using postal questionnaires to examine carer satisfaction with palliative care: a methodological assessment. Palliat Med 1999;13:285-298.

Kane RL, Bernstein L, Wales J et.al. A randomized controlled trial of hospice care. Lancet 1984;April:890-894.

Kane RL, Klein SJ, Bernstein L, et. al. Hospice role in alleviating the emotional stress of terminal patients and their families. Med Care 1985;23:189-197.

Keay TJ, Fredman L, Taler GL, et.al. Indicators of quality medical care for the terminally ill in nursing homes. JAGS 1994;42:853-860.

Kristjanson, LJ. Indicators of quality of palliative care from a family perspective. J Palliat Care 1986;2:7-19.

Kristjanson, LJ. Quality of terminal care: salient indicators identified by families. J Palliat Care 1989;5:21-30.

Kristjanson, LJ. Validity and reliability testing of the FAMCARE scale: measuring family satisfaction with advanced cancer care. Soc Sci Med 1993;36:693-701.

Larsson G, Wilde-Larsson B, Munck IME. Refinement of the questionnaire 'Quality of Care from the Patients' Perspective' using structural equation modelling. Scand J Caring Sci 1998;12:111-118.

Letters to the Editor. The SUPPORT Project and Improving Care for Seriously Ill Patients. JAMA 1996;275:1227-1232.

Loblaw DA, Bezjak A, Bunston T. Development and testing of a visit-specific patient satisfaction questionnaire: the Princess Margaret Hospital Satisfaction with Doctor Questionnaire. J Clin Oncol 1999;17:1931-1938.

Lynn J, Teno JM, Phillips RS, et.al. Dying experience of older and seriously ill patients: findings from the SUPPORT and HELP projects. Ann Intern Med, In Press.

McCusker, J. Development of scales to measure satisfaction and preferences regarding long-term and terminal care. Med Care 1984;22:476-493.

Nelson EC, Hays RD, Larson C, et al. The patient judgement system: reliability and validity. QRB 1989;June:185-191.

Parkes, CM. Terminal Care: evaluation of in-patient service at St. Christopher’s Hospice. Part I. Views of Surviving Spouse on Effects of the Service on the Patient. Postgrad Med J 1979;55:517- 522.

Rubin HR, Gandek D, Rogers WH, et al. Patient’s ratings of outpatient visits in different practice settings: results from Medical Outcomes Studies. JAMA 1993;37:835-840.

Rubin, HR, Ware JE, Nelson, EC, Meterko M. The Patient Judgement of Hospital Quality Questionaire. Med Care 1990: 28, No. 9, Special Supplement.

SUPPORT Principal Investigators. A controlled trial to improve care for seriously ill hospitalized patients: The Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT). JAMA 1995;274:1591-1598.

Tarlov AR, Ware JE, Greenfield S, et al. The medical outcomes study: an application of methods for monitoring the results of medical care. JAMA 1989;262:925.

Teno JM, Hakim RB, Knaus WA, et al. Preferences for cardiopulmonary resuscitation: physician- patient agreement and hospital resource. J Gen Intern Med 1995;10:179-186.

Teno, JM. Consumer reports and ratings of medical care: will we ever get satisfaction? Institute of Medicine Conference: Evaluating Health Outcomes for Elderly People in a Changing Health Care Marketpalce, Briefing Book, 1996.

Tierney RM, Horton SM, Hannan TJ, Tierney WM. Relationships between symptom relief, quality of life, and satisfaction with hospice care. Palliat Med 1998;12:333-344.

United States Congress, Office of Technology. The quality of medical care: Information for customers, OTA-H-386. Government Printing Office: Washington, DC. June, 1988.

van Campen C, Sixma HJ, Kerssens JJ, Peters L. Assessing noninstitutionalized asthma and COPD patients' priorities and perceptions of quality of health care: the development of the QUOTE-CNSLD instrument. J Asthma 1997;34:531-538.

Wales J, Kane R, Robbins S, et al. UCLA hospice evaluation study: methodology and instrumentation. Med Care 1983;21:734-744.

Ware JE, Hays RD. Methods for measuring patient satisfaction with specific medical encounters. Med Care 1983;21:886-891.

Ware, JE. Effects of acquiescent response set on patient satisfaction ratings. Med Care 1978;16:327.

Westra BL, Cullen L, Brody D et al. Development of the Home Care Client Satisfaction Instrument. Pub Health Nursing 1995;12:393-399.

Wiggers JH, Donovan KO, Redman S, Sanson-Fisher RW. Cancer patient satisfaction with care. Cancer 1990;1:610-616.

Wilde B, Larsson G, Larsson M, Starrin B. Quality of care from the elderly person's perspective: subjective importance and perceived reality. Aging Clin Exp Res 1995;7:140-149.

Wolf MH, Putnam SM, James SA, et al. The medical interview satisfaction scale: Development of a scale to measure patient perceptions of physician behavior. J Behav Med 1978;1:391.

World Health Organization: Cancer pain relief and palliative care. Geneva:WHO 1989.