Choosing an Instrument - Toolikit (TIME)

Demings has been quoted as saying, “if you don't measure, you can't improve it!”. Conducting an audit (i.e., a systematic critical analyses of the quality of care) is often a first step in identifying opportunities to improve and enhance the care of dying persons and their loved ones. Measurement of the processes and outcomes of care take on a significant role in this initiative. An initial survey of family members after the patient's death and/or review of medical records is an important first step in observing medical practice at your health care institution. Audits through review of the medical record or interviews help to identify unmet needs and the opportunities to improve medical care. Audit results create the needed tension, through highlighting the difference between current guidelines or standards of care and your local medical practice, that leads to substantive change. Such tension creates the awareness for health care providers to improve and enhance their current practice.

A second important role of measurement is to assess the “end results” or outcomes of the proposed program. Outcome measures examine the effect of your proposed service or intervention on the quality of care and the quality of life of the patients and/or family members. Careful consideration should be given to the choice of outcome variables. Selected outcomes should be patient focused and family centered reflecting the goals of palliative care medicine as stated by the World Health Organization in 1990: “..The goal of palliative care is achievement of the best quality of life for patients and their families.” Additionally, outcome measures selection should be guided by what key stakeholders believe is important evidence of effectiveness to justify the continued financial support of your program.

The difference between process and outcome measurement is an important distinction. A process measure examines what the proposed service or intervention does for patient and families (e.g., Are the patients pain levels assessed?) while an outcome measures the benefit of services to patients and their loved ones (e.g., Did the care you received through (name of organization) provide you with the desired level of pain control?). Ideally, a program should consider measuring both the processes of care and measurable outcomes. However, the success of your program must be judged by outcome variables. Changing the process of care is necessary step, but may not be sufficient enough to improve patient care. For example, several research efforts have shown that merely getting patients to write advance directives does not lead to improve patient outcomes. It is important that your intervention attend to changing the processes of care, but the success or failure of the intervention should be based on outcomes or benefit of your intervention to patient and their families.

Quality of care and quality of life change substantially for those with a serious chronic illness and nearing the end of their lives. As one dies, life takes on new shape – values change and things once ignored become more important. Existing quality of care measures do not attend to the changes in priorities or to dimensions that acquire new significance (e.g., spirituality and transcendence). Many of the most prominent current instruments make seriously flawed assumptions for patients at the end of life (e.g., that objective functional capacity is a large determinant of quality of life). Dying can be conceptualized as an inevitable stage of human life which holds the potential for suffering as well as important discernible opportunities for personal growth. It is a time period when medical care ought to be patient focused and family centered. Patient focused care in part reflects that (short of utter medical futility) informed, patient preferences ought to shape the timing of transitions in the goals of care (i.e., from treatment that pursues life extension to acceptance that one is dying or terminally ill). Family centered care acknowledges the important role played by that loved ones in a patient's final days. Additionally, an important goal of palliative care is addressing bereavement. Dying profoundly impacts on family members as noted by Dame Cicely Saunders, the founder of the modern hospice movement, “How people die remains in the memories of those who live on.”

In 1996 together with colleagues at the Center to Improve Care of the Dying, the first Tool Kit of Instruments to Measure End of life care (TIME) conference was held at the National Academy of Science compound at Woods Hole, MA. This conference focused on what are the goals of the tool kit, existing conceptual models of quality of care and quality of life for patient and their families, and identifying what are the available measures to examine the quality of care and quality of life for dying patients and their families. An important contribution to this conference was a paper authored by Anita Stewart and colleagues [2]. This paper outlines a conceptual model of quality of care for dying patients and their loved ones (see figure 1 and 2 of the attached article on the enclosed diskette). This work build upon the deliberations of Ethics Committee of the American Geriatric Society. [3] The latter group proposed 10 domains to measure the quality of care. Participants at the TIME conference conducted literature reviews in each of these domains to identify existing measurement tools which have been used in a population of seriously ill and/or dying patients. For each domain, we selected a few key instruments for in depth review using the frame work proposed by Medical Outcomes Trust. Listed on this WWW page are the actual bibliographies.

Choice of domains as outcome measures should be guided by the goals of your program (ask yourself what do you hope to accomplish?). For example, let us consider an intervention whose aim is to improve patient and family outcomes through facilitating comprehensive advance care planning. You believe that the intervention will impact on the process of communication through helping patients formulate informed preferences with their physicians, that physicians as well as other health care providers formulate a plan of care that ensures honoring those preferences and that this will lead to improve quality of the dying experience. Potential hypothesis include the following:

To what extent were these wishes or preferences followed in the medical treatment (he/she) received from the time of (his/her) last hospitalization to (his/her) death?

62. How severe was the pain? Was it... not at all severe 1 moderately severe 2 extremely severe 3

63. How much did the pain distress or bother (him/her)? Would you say... not at all 1 a little bit 2 somewhat 3 quite a bit 4 very much 5

The choice of outcome measure should be guided by the goals of your project. Table 1 lists some key domains and example questions drawn from existing instruments.

There are three important things to highlight from the proceeding example: the use of family members as a source of information; patient centered reports are an important outcome variable to examine the quality of care; and we are still at an early stage in development of measurement tools. The TIME conference has chosen to emphasize the use of interviews with the family or surrogate decision maker after the patient's death. Many patients, perhaps higher than one in three, are unable to be interviewed in the last week of life. Furthermore, identifying prospectively who is in their last week of life is difficult task. Without data collected from a similar time period, it is difficult to compare interventions with control groups (e.g., pain worsens as a patient gets closer to death; thus, comparing the pain ratings 3 weeks prior to death of the intervention patient to one week prior to death of the controls may lead to false conclusions). We have referred to this as the “denominator problem” (see enclosed draft paper from the Second TIME conference by Teno and Coppola). One strategy for pain and other symptoms is to make pain (or other symptom) as a fifth vital sign that is prospectively routinely assessed.

A second strategy is utilizing post death family interviews. Who is in the denominator and when is the last week of life clearly defined by this strategy . Family members can provide important information about the patient's physical, psychological, and spiritual needs at the end of life, and how these needs were met or unmet by the health care team. Focusing on medical needs, family members can indicate if they perceived that the patient's wishes were understood and followed, and if the patient died without significant pain. Additionally, families can report on the quality of care from their unique perspective. This type of information can be easily collected and used to guide quality improvement efforts. [5] Yet, there is still important research that needs to be conducted to understand the validity of family reports of the quality of care after the patients death. Research needs to examine if family members' views of dying change with the process of bereavement. In order for family members to move on with their lives (and to make peace with the experience) it is quite possible that they replace memories of a painful or disturbing death of their loved one with less painful memories . Alternatively, family members may be angry or harbor resentments from their loss, which may promote memories that are reported as overly negative. A second important concern is the accuracy in which a family member can report on certain domains such as the patient experience of pain and other symptoms. Pain is such a personal experience which another can't accurately perceive. However, family members can report on what patients told them about their pain and how the patient appeared to them.

As shown in Table 1 modified from the Institute of Medicine report on care at the end of life, we have exerted that patient centered reports and ranking of the quality of care are an important outcome variable to measure the quality of care. Over a lifetime, patient expectations of health care may change quite dramatically. A patient in the early decades of life may place more emphasis on technical competence. An elderly person with multiple ADL impairments may have different health values, but health values such as preferences for life sustaining treatment are not predicted solely on patients age or functional status. Given this heterogeneity of patient preferences, it is paramount that medical care is patient focused and the success of an intervention is not measured solely on timing of withdrawal of life sustaining treatment. Rather, medical care must attend to a process of communication such that it helps patients formulate informed preferences and ensures that care at the end of life is consistent with patient preferences. For dying patients, being appropriately informed of their treatment options in a way that they can understand, spoken to in a compassionate manner, and treated with compassion and dignity may be more important than the amount of resuscitation attempts in the final day of life.

Typically, such perceptions of medical care have been measured by satisfaction measures. Some might argue that our current state of measuring satisfaction is best summed up by the 1965 Rolling Stones hit, “(I can't get no) Satisfaction.” It has been argued that patients are not accurate judges of the technical competence of health care providers and institutions. Patient report of technical quality may be overly based on interpersonal skills of the provider. Highly empathic skills may mask poor medical judgments. Furthermore, a patient is vulnerable and dependent on medical care providers. Thus, many, if not the majority, of dissatisfied patients are either afraid to or do not choose to voice concerns to their physician, health care institution, health plan, or even to an anonymous interviewer on the telephone. [6]

These and other important methodological concerns have been raised about satisfaction measures. [7] Many have been struck by the inconsitency between how the patient and family respond to standardized surveys about satisfaction with medical care and how they would answer the very same questions verbally. For example, a 80 y/o patient from a nursing home who had completed an AD and stated that she did not want “heroics” developed a GI bleed. Her colon was perforated during the colonoscopy. This resulted in an 81 day hospital stay on a ventilator. The patient was then returned to the nursing home with a feeding tube and is now dependent in all her basic activities of daily living. Her nephew, the designated proxy in a durable power of attorney, was interviewed in the first, second, and six month of this illness. After sixth months, he still recalled and responded to an open text question that the hospital staff ignored her advance directive even though he told them that the she had an advance directive. Yet, his rating of satisfaction with medical care was between good and excellent.

Concerns with patient satisfaction (or ratings) about their medical care led Cleary and colleagues to develop a 61 item survey about the patient's experience of hospital care that was based solely on patient report of specific events (e.g., “Were you told of the purpose of your medications in a way that you could understand?” [8]). The authors argued that they framed questions “to be as specific as possible, to minimize the influence of confounding factors such as the patients' expectations, personal relationships, gratitude, or response tendencies related to gender, class, or ethnicity.” [9] Based on this work, we have written new questions to measure key processes in end of life care. These questions have been included in the draft Tool kit instruments.

However, we still need both patient ranking and reports. Consider the two following conversations about preferences for continued use of life sustaining treatment:

Physician B: Mrs. M, there are several options that we have.. Your views are important in deciding the best treatment and whether or not to continue on the ventilator, the machine that is helping you breath now. OK… Let me tell you about your condition and then we will discuss what the options are…

A patient or family member in both instances should report quite factually that in both cases the physician discussed whether life sustaining treatment should be utilized. Yet, the patient's or family member's degree of satisfaction or ranking of the quality of those conversations may be quite different. It is our recommendations that we rely on both reports and rating of quality of care. To improve the discriminate validity of patient and family ranking of quality of care, we are currently working on whether informing respondents of the organization mission improves the discrimination of rankings of the quality of care. For example, here is the current example of a question that we are testing:

In the next set of questions, I am going to read you a statement about the mission of Hospice Care of Rhode Island and then ask a question on whether we achieved that goal.

The mission of Hospice Care of Rhode Island is to provide excellent medical care by the following:

Communicating with the patient and family so that they understand the patient's illness and treatment plans.

How well do you think the Hospice Care of Rhode Island did in achieving this goal? Choose a number on the scale 0 to 10, where 0 is the communication failed to meet that expectation and 10 is greatly exceeded that expectation.

This website contains both validated and draft versions of the Toolkit instruments as resource for you. The validated instruments include an Afterdeath Bereaved Family Member Interview and a Prospective Patient Interview. Many of the questions in the Toolkit instrument have been adapted or modified from existing instruments; however, new questions have been written as outlined above. The current Toolkit is framed towards adults; it has yet to be adapted for pediatric patients. Depending on the age of the patient, parents may play an even more important role in assessing the quality of care. As in cognitively impaired adults, assessment of pain in young children is a difficult challenge, with adequate instruments yet to be developed.

At this time, we are still far from the ideal or perfect instrument; important gaps remains. Tremendous progress has been made in measurement of health related quality of life. It our hope that a similar effort can be undertaken for dying patients and their families. We hope that Tool kit can serve as resource in your effort to improve and enhance the care of the dying.

Overall Quality of Life

Considering all parts of my life-physical, emotional, social and spiritual, and financial—my quality of life in the past two days was..

very bad to excellent on 10 point scale

see reference 3

Spirituality/ Transcendence

It is important to me to feel that my life has meaning (5 points from agree to disagree).

(From the Missoula-Vitas quality of life instrument)

Did members of the hospice team really listen to you and (PATIENT) about your hopes, fears, and beliefs as much as you wanted?

Did someone talk with you and/or (PATIENT) about your religious or spiritual beliefs in a sensitive manner?(From the Tool kit of instruments)

Symptoms

INTERVIEWER: FOR QUESTION 2, RECORD THE SYMPTOMS NOTED BY THE RESPONDENT. FOR EACH SYMPTOM, ASK QUESTIONS 2B- 2E. FILL IN RESPONSES TO QUESTIONS IN THE TABLE BELOW.

2A. What were the two most bothersome symptoms for (PATIENT)in the last week of life?

2B. How often did (PATIENT) have (SYMPTOM)? Would you say..

(Respondent chooses between: occasionally, about half of the time, most of the time, all of the time)

2C. How severe was the (SYMPTOM)? Was it...

(Respondent chooses between: not at all severe, moderately severe, and extremely severe)

2D. How much did (SYMPTOM) distress or bother (PATIENT)? Would you say...

(Respondent chooses between: not at all, a little bit, somewhat, quite a bit, and very much)

2E. Did (PATIENT) tell you directly about his/her) (SYMPTOM)?

Questions from Tool Kit adapted from Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments, Memorial Sloan Kettering Symptom Assessment Score, and McGuill Quality of Life Instrument

Advance Care Planning

Have you and your doctor made plans that makes sure you are not in pain or discomfort while dying? (Tool Kit question)

During the last week of (the patient's) life, did he/she prefer a course of treatment that focused on extending life as much as possible, even if it meant more pain and discomfort, or on a plan of care that focused on relieving pain and discomfort as much as possible, even if that meant no living as long?

To what extent were these wishes or preferences followed in the medical treatment he/she received during the last week of life?

Patient Centered Reports and Ranking

Did you feel that the members of the hospice team were as helpful as possible in explaining (PATIENT'S) condition?

Do you feel that the hospice team provided you with enough information so that there were no surprises or unplanned medical events?

Did a doctor talk with you or (PATIENT), in a way that was easily understandable, about the chances that (he/she) would survive the last hospitalization?

(from Tool kit instruments)

Footnotes

[1] This is summary is based on the executive summary of the Tool kit of Instruments to Measure End of life care (TIME), a paper by Joan Teno prepared for the Institute of Medicine on measuring satisfaction, and a paper by Joan Teno and Kristen Coppola, For Every Numerator, You need a Denominator: A Simple Statement but Key to Measuring the Quality of Care of the “Dying”.

[2] Assessing the Quality of Care of Seriously Ill Dying Patients and Their Families in Terms of Quality of Life Anita L. Stewart, Ph.D., Joan Teno, M.D., Donald L. Patrick, Ph.D., M.S.P.H., and Joanne Lynn, M.D. (manuscript has been included in your packet for your review on 31/2 diskette)

[3] Lynn JL. Measuring quality of care at the end of life: A statement of principles. 1997; 45:526-527.

[4] Cohen, SR, Mount, BM, Strobe, MG, et. al., The McGill Quality of Life Questionnaire: A Measure of Quality of Life for People with Advanced Disease. Palliative Medicine 9:207-219; 1995.

[5] Teno JM, Landrum K, Kreling BA, Boekoloo BA. Breaking through the Glass Ceiling of Hospice Referral: Is it Enough? (manuscript under review); Addington-Hall, J and McCarthy M. Audit Methods. Views of Family Members After Death in Clinical Audits in Palliative Care, I Higginson, ed. New York, Radicliffe Medical Press, 1993.

[6] Vouri, H. Patient Satisfaction- An Attribute or Indicator of the Quality of Care? Editorial. Quality Review Bulletin 1987;13:106-8.

[7] Ware, J. E. Effects of Acquiescent Response Set on Patient Satisfaction Ratings. Medical Care 1978;26:327-336.; Linn, L. S., and S. Greenfield. Patient Suffering and Patient Satisfaction among the Chronically Ill. Medical Care 20:425-431. Rubin, H. R. Can Patients Evaluate the Quality of Hospital Care? Medical Care Review 1990;47:267-336.; Walker, A. H., and J. D. Restuccia. Obtaining Information on Patient Satisfaction with Hospital Care: Mail versus Telephone. Health Services Research 1984;19:291-396. ; Ley, P., J. Kinsely, and S. T. Atherton. Increasing Patients' Satisfaction with Communication. British Journal of Social and Clinical Psychology 1976;15:403-413. ; Fowler, F. J., P. D. Cleary, J. Magaziner, D. L. Patrick, and K. L. Benjamin. Methodological Issues in Measuring Patient-Reported Outcomes: The Agenda of the Work Group on Outcomes Assessment. Medical Care 1994;32:JS65-JS76.

[8] Cleary, P. D., S. Edgman-Levitan, M. Roberts, T. W. Moloney, W. McMullen, J. D. Walker, and T. L. Delbanco. Patients Evaluate Their Hospital Care: A National Survey. Health Affairs Winter 1991:254-267.

[10] Modified from Table 5.4 of Approaching Death: Improving Care at the End of Life, edited by Mairlyn J Field and Chirstine K. Cassel, 1997 Naltional Academy of Science, Washington, DC.