Frequently Asked Questions about Joining the Learning Collaborative:
- What measurement points do we use?
- How reliable are the instruments?
- How will these instruments work with our Electronic Medical Record?
- Do you need an electronic medical record?
- How easy is it for clinicians to enter the data and to use the data?
- How often is the data collected?
- What is your definition of recovery?
- Do I have to use all of the instruments at once?
1) What measurement points do we use?
Reaching Recovery measures recovery from multiple perspectives and multiple dimensions over time to understand more about recovery.
- The Recovery Markers Inventory, which is the staff’s rating of the consumer’s progress, measures eight (8) dimensions that “tend to correlate” with an individual’s recovery.
- The Consumer Recovery Measure measures the consumer’s perception of their recovery. It is very useful to understand whether what clinicians’ observe matches how the consumer is feeling. For example, a person may stay at home because they are an introvert OR because they might have paranoia symptoms.
- The Recovery Needs Level assigns the right level of service to the consumers. The basic assumption being that consumers recover and their needs change over time. The Recovery Needs Level measures criteria for service needs in 17 areas such as: GAF, Hospitalizations, Lethality, Co-Occurring Substance Abuse, and Case Management Needs.
- The PRO: Promoting Recovery in Organizations addresses the agency’s special needs. It is an annual survey broken into sections for each type of staff that interact with the consumers. For example, the front-desk staff, clinical, medical, case managers, and rehabilitation staff. The results tell the agency how well specific staff are doing at promoting consumers’ recovery.
2) How reliable are the instruments?
For reliability Reaching Recovery looks at how consistently we get the same score for individuals with the same level of indicators of recovery. We want high reliability, meaning high constancy in scoring. Mathematically, it is hard to get a high reliability with a small number of questions. However, by using Classical Test Theory and Item Response Theory, we are able to produce highly reliable instruments with a small number of questions.
The Recovery Marker Inventory is scoring:
IRT reliability: Person = .75, Item = 1.00
CTT reliability = .78
The Consumer Recovery Measure is scoring:
IRT reliability: Person = 0.83, Item = 0.96
CTT reliability = 0.86
The Recovery Needs Level is scoring:
IRT reliability: Person = .75
CTT reliability = .78
The PRO Survey (for each of the different types of staff, reliability testing is done)
| Sub-Scale | # of Items | CTT alpha: | IRT Reliability: Person | IRT Reliability: Item |
|---|---|---|---|---|
| Therapist | 15 | .91 | .70 | .79 |
| Case Manager | 13 | .88 | .75 | .92 |
| Medical Staff | 15 | .91 | .70 | .82 |
| Nursing Staff | 10 | .92 | .77 | .94 |
| Residential Staff | 10 to 12 | .92 or .96 | n.a. | n.a. |
| Front Desk | 11 | .91 | .73 | .87 |
| Rehab Staff | ? | n.a. | n.a. | n.a. |
3) How will these instruments work with our Electronic Medical Record?
The templates for the Recovery Marker Inventory, the Consumer Recovery Measure, and the Recovery Needs Level can be placed into an electronic medical record, making the collection of data a seamless process. The reports generated from the data are then viewed on a web-based system that can be linked to your medical record, allowing clinicians’ to access the reports with a simple “click of the mouse.”
The PRO survey has been done by hiring a team of consumer surveyors to conduct the survey personally with the consumers. This is done using a paper form and then manually entered into a database.
4) Do you need an electronic medical record?
All of these instruments can be done on hard copy and then entered into our web-based reporting system. However, it is best to use an electronic medical record to decrease human errors.
5) How easy is it for clinicians to enter the data and to use the data?
The Recovery Marker Inventory consists of only eight questions with clearly defined categories for each answer. Once a clinician familiarizes themselves with the definitions for each answer it simply takes a clinician a few minutes to complete
The same is true for the Consumer Recovery Measure. This is a 14 question survey and takes a few minutes to complete. Once a consumer becomes accustom to filling it out, it only takes moments.
Once the Recovery Marker Inventory and/or the Consumer Recovery Measure is entered into the database, the results are then accessible. The results are complied in easy to understand charts and graphs. The graphs and charts were designed to give clinicians and managers easily understandable data to track consumers’ progress in recovery over time and continue to improve their practice.
The Recovery Needs Level is done in conjunction with a consumer’s treatment/service plan. There are 17 areas addressed in this instrument. The more a clinician knows about the consumer, the faster it is for the clinician to complete. The results are automatically produced in the medical record for the clinician to view and discuss with his/her team immediately. Clinicians purposely do not know the details of the algorithm that generates the overall rating score. This has helped clinicians look at service need more objectively and has decreased reviewer bias.
The PRO survey is done annually. For each type of staff there are between 10 to 15 questions. MHCD hires a team of consumers to administer the survey to a 10% stratified sample of consumers.
6) How often is the data collected?
The Recovery Maker Inventory and the Consumer Recovery Measure are collected quarterly on high intensity treatment teams. The more traditional outpatient team, which may not see a consumer often, collects the data on every third visit.
The Recovery Needs Level is done every six months in conjunction with the consumer’s treatment/service plan.
The PRO Survey is an annual survey.
7) What is your definition of recovery?
Recovery is rooted in the principle that people with mental illness can live, work, learn, and lead productive and fulfilling lives given the proper treatment and services.
8) Do I have to use all of the instruments at once?
No. Mental health centers can choose the instruments that best fit their need. Mental health centers have found it best to phase in the use of the instruments throughout their centers. For example, one center started with the use of the RNL and then the next year implemented the Recovery Markers Inventory and the Consumer Recovery Measure.
©2010