Measurement-based care provides indicators of treatment progress to clinicians and patients that enable them to tailor the treatment approach to patient needs in real time. While standard practice in many medical settings, it is not yet commonplace in substance use disorder treatment. In this study, the researchers pilot tested a measurement-based care assessment tool in an outpatient treatment program.
l
Measurement-based care is standard practice in medicine and has been tested many times in non-substance use disorder mental health treatment settings, where it is generally associated with better treatment retention and outcomes and a better ability for clinicians to detect stagnation in treatment progress and modify treatment accordingly. However, the application of measurement-based care in specialty substance use disorder treatment settings has been limited, even though it is possible that the benefits of measurement-based care observed in other mental health treatment settings would likely extend to substance use disorder treatment settings.
In this study, researchers developed a measurement-based care system designed to be used in outpatient substance use disorder treatment settings. Following a design framework centered around the patient, the prototype was refined based on several rounds of patient and clinician feedback. The resulting system had two main components: 1) a patient-facing care questionnaire they called the weekly check-in, and 2) a web-based clinician-facing dashboard for reviewing care results the referred to as the clinician dashboard.
This pilot study evaluated the feasibility and acceptability of this care system in the context of a typical United States outpatient substance use disorder treatment clinic. The researchers reported outcomes related to patients and clinicians’ engagement with the care system and their assessments of its usability and clinical utility when using this system in the context of substance use disorder treatment as usual.
This was a non-randomized, proof-of-concept, pilot study with 30 patient participants who completed weekly questionnaires designed to track treatment progress or completed these weekly questionnaires + their results were shared with their primary clinic provider, to assess the feasibility and acceptability of this measurement-based care approach. All participants were asked to complete the weekly questionnaires for 6 months. Six clinician providers also participated in this study.
All participants were recruited from the same large, public, addiction and mental health treatment clinic in Washington State, USA. The clinic offered: 1) individual psychotherapy, 2) case management, 3) group therapy, 4) peer support, and 5) medication management for psychiatric conditions including substance use disorder.
To be eligible for the study, patients needed to, 1) be receiving treatment for substance use disorder from a clinician in the study, 2) own a smartphone, 3) be able to speak and read English, 4) be over 18 years old, 5) have past-year unhealthy alcohol use reflected by an Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) score of 3 (women) or 4 (men) suggesting alcohol use disorder, and/or 6) use of illicit or non-prescribed drugs in the past-year.
Study recruitment ran from October 2019–June 2021, with a pause between March and June 2020 because of COVID-19.
At study baseline, participants were assigned to one of two conditions based on when they enrolled in the study. Patients who enrolled in the first wave were assigned to the ‘weekly check-in only’ condition, in which they completed weekly check-in questionnaires, however their clinicians did not have access to their check-in results.
Patients who enrolled in the second wave were assigned to ‘weekly check-in + clinician dashboard’. In this condition, the results of weekly check-ins were accessible to the participant’s clinician via the clinician dashboard.
This was an unblinded study, meaning participants in both conditions were informed as to whether their weekly check-in results would be viewable to their clinician.
Patient weekly check-ins.
Study participants were sent invitations to complete weekly check-ins via text message or email depending on their preference.
At the initial study appointment, the researchers encouraged participants to complete weekly check-ins in a timely fashion. They then monitored check-in completion throughout the study period, and for the first 12 weeks of the study contacted participants to offer support completing check-ins. For participants in the weekly check in + clinician dashboard condition, the researchers encouraged participants to answer weekly check-in questionnaires knowing that their clinician would review their responses.
The weekly check-in included 22 questions derived from existing questionnaires. Questions assessed past-week drinking and other drug use (Figure 1A), past-week experiences with craving, coping skills, abstinence self-efficacy (one’s confidence that they can maintain abstinence), depression symptoms, positive life outlook, and therapeutic alliance (Figure 1B). Questions also assessed goals for the upcoming week related to reducing substance use and cravings, learning more effective coping strategies, increasing abstinence self-efficacy and improving mental health, and having a more positive outlook (Figure 1C). Two optional, open-ended questions also allowed participants to share any additional goals for the upcoming week and information that they wanted their clinician to have (Figure 1D).
Figure 1. Screenshots of the measurement-based care weekly questionnaires receive by patients.
The clinician dashboard.
Clinician participants were given access to a web-based dashboard on which they could review the summarized results from the weekly patient check-ins. The dashboard displayed change in measures across time using line graphs (Figure 2A), text-based summaries (Figure 2B), bar graphs showing the most recent responses to each question (Figure 2C), and a table showing participants past-week question responses (Figure 2D). Clinicians received email reminders to review the dashboard every 2 weeks.
Figure 2. Screenshots of the clinician dashboard showing summarized, weekly results from patient questionnaires.
Study participants were found to be similar to the full clinic population in terms of age, gender, race, and ethnicity. About two-thirds of patients were aged 35–54 (63%), about half were male (53%), and White (57%). Just over three-quarters (77%) had an associate’s degree, trade degree, or higher education degree. A large majority of patients were not currently employed (90%), and just over half (53%) were experiencing homelessness. Most participants had symptoms consistent with severe substance use disorder (77%). Participants’ primary substance included stimulants (60%), opioids (53%), alcohol (50%), cannabis (17%), sedatives (1%), and hallucinogens (<1%).
All 30 patient participants completed the baseline and 6-week research appointments, with all except one completing the 12- and 24-week follow-ups.
Twenty-nine patient participants chose to receive weekly check-in prompts via text and one elected to receive them by email.
Eight clinicians participated in the study. Six had at least 1 patient who was enrolled in the weekly check-in + clinician dashboard condition (median = 2 patients per clinician) allowing the clinician to review patient progress in the clinician dashboard. Clinicians were asked to provide feedback on the dashboard by completing a dashboard usability questionnaire.
The patient response rate to check-ins was high.
On average, participants completed around 21 weekly check-ins (~86%). All participants completed the first 2 weekly check-ins, with the proportion of patients completing the weekly check-ins decreasing over time but then stabilizing. By week 12, 80% of participants completed the weekly check-in with this percentage remaining fairly stable through study week 24.
Participants in the weekly check-in only condition and the weekly check-in + clinician dashboard condition were similar in terms of the number of weekly check-ins completed.
Clinicians may have under-utilized the dashboard.
Five out of six clinicians who had a patient in the weekly check-in + clinician dashboard condition logged into the clinician dashboard at least once. Among them, there was a mean of approximately 12 logins per clinician (min= 3 / max= 25). Login sessions were, on average, around 2 minutes long.
Patient and provider usability ratings were positive.
Usability was deemed to be good when assessed at the 6- and 24-week time points, with most patients (86–100%) agreeing or strongly agreeing that the weekly check-ins were helpful and that they would be willing to use this tool again in the future. Most also indicated that they would recommend the check-in tool to others. Most participants (86%) felt the length of the weekly check-ins was “just right”. Most also reported feeling confident in their ability to complete check-ins independently or during treatment sessions (83–100%) with a small portion of the sample feeling that they would experience stress completing weekly check-ins (7–14%).
Five out of 6 clinicians completed the dashboard usability questionnaire. Generally speaking, their ratings were favorable, with all responding clinicians feeling the dashboard was easy to navigate, and that the information provided was meaningful and useful for treatment planning. Most also felt that the information was helpful to their patients.
Results from this study provide preliminary support for the acceptability and feasibility of a digital ‘check-in’ system in substance use disorder outpatient treatment settings. Among patients and clinicians in this pilot study, rates of engagement with the care system were high for patients with an 86% check-in completion rate, but less robust for clinicians who logged into the dashboard a mean of around 12 times.
By their report, check-ins imposed very little burden on patients, who on average took less than 5 minutes to complete each weekly check-in. Usability ratings were also favorable, with most patients feeling the weekly check-ins aided self-reflection, and something they could see themselves continuing to use. Additionally, most clinicians felt that the information on the dashboard was helpful, noting that they discussed the provided information with their patients.
Notably, the vast majority of patient participants (97%) indicated a preference for receiving the weekly questionnaires via text on their smartphone, versus through email. Based on this finding, future digital care practices such as the one assessed in this study may do best to focus on smartphone/app-based assessments rather than email.
Taken together, findings from this research are important because measurement-based care is known to improve addiction recovery outcomes. At the same time, these findings may not accurately reflect how measurement-based care may be accepted and utilized in other outpatient substance use disorder treatment settings. It is possible that clinicians may be resistant to utilizing such care systems, perceiving them to be additional work on top of already busy schedules, or may be uncertain about how to integrate these new systems into their clinical practice. Healthcare systems may also struggle to adopt measurement-based care systems due to, 1) a lack of training and technical support, 2) insurance reimbursement models that do not cover the time spent using measurement-based care tools, and 3) limited integration of measurement-based care systems with other treatment technologies. Some treatment systems may also be hesitant to integrate measurement-based care into their clinics as it may expose them to outcomes-based reimbursement from insurers who are increasingly moving to ‘pay for performance’ models of reimbursement.
As noted by the authors:
In addition:
Results from this clinical pilot study suggest that the measurement-based care system developed and assessed by these researchers can be incorporated into a typical substance use disorder treatment setting, and that patients and providers will engage with it. Further, these preliminary findings suggest this care system is helpful for patients and providers. This complements previous work that has suggested that implementation of measurement-based care can improve substance use disorder treatment outcomes. At the same time, clinician engagement was difficult to determine by the results presented, and it is possible clinicians under-utilized the patient reports. Additionally, clinicians volunteering to participate in such studies may be more motivated to use such technology and may not reflect the average community substance use disorder provider. More research is needed to assess potential barriers to implementing measurement-based care in substance use disorder treatment, as well as to determine how these kinds of interventions influence patient outcomes.
Hallgren, K. A., Cohn, E. B., Ries, R. K., & Atkins, D. C. (2022). Delivering remote measurement-based care in community addiction treatment: engagement and usability over a 6-month clinical pilot. Frontiers in Psychiatry, 13, 840409. doi: 10.3389/fpsyt.2022.840409
l
Measurement-based care is standard practice in medicine and has been tested many times in non-substance use disorder mental health treatment settings, where it is generally associated with better treatment retention and outcomes and a better ability for clinicians to detect stagnation in treatment progress and modify treatment accordingly. However, the application of measurement-based care in specialty substance use disorder treatment settings has been limited, even though it is possible that the benefits of measurement-based care observed in other mental health treatment settings would likely extend to substance use disorder treatment settings.
In this study, researchers developed a measurement-based care system designed to be used in outpatient substance use disorder treatment settings. Following a design framework centered around the patient, the prototype was refined based on several rounds of patient and clinician feedback. The resulting system had two main components: 1) a patient-facing care questionnaire they called the weekly check-in, and 2) a web-based clinician-facing dashboard for reviewing care results the referred to as the clinician dashboard.
This pilot study evaluated the feasibility and acceptability of this care system in the context of a typical United States outpatient substance use disorder treatment clinic. The researchers reported outcomes related to patients and clinicians’ engagement with the care system and their assessments of its usability and clinical utility when using this system in the context of substance use disorder treatment as usual.
This was a non-randomized, proof-of-concept, pilot study with 30 patient participants who completed weekly questionnaires designed to track treatment progress or completed these weekly questionnaires + their results were shared with their primary clinic provider, to assess the feasibility and acceptability of this measurement-based care approach. All participants were asked to complete the weekly questionnaires for 6 months. Six clinician providers also participated in this study.
All participants were recruited from the same large, public, addiction and mental health treatment clinic in Washington State, USA. The clinic offered: 1) individual psychotherapy, 2) case management, 3) group therapy, 4) peer support, and 5) medication management for psychiatric conditions including substance use disorder.
To be eligible for the study, patients needed to, 1) be receiving treatment for substance use disorder from a clinician in the study, 2) own a smartphone, 3) be able to speak and read English, 4) be over 18 years old, 5) have past-year unhealthy alcohol use reflected by an Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) score of 3 (women) or 4 (men) suggesting alcohol use disorder, and/or 6) use of illicit or non-prescribed drugs in the past-year.
Study recruitment ran from October 2019–June 2021, with a pause between March and June 2020 because of COVID-19.
At study baseline, participants were assigned to one of two conditions based on when they enrolled in the study. Patients who enrolled in the first wave were assigned to the ‘weekly check-in only’ condition, in which they completed weekly check-in questionnaires, however their clinicians did not have access to their check-in results.
Patients who enrolled in the second wave were assigned to ‘weekly check-in + clinician dashboard’. In this condition, the results of weekly check-ins were accessible to the participant’s clinician via the clinician dashboard.
This was an unblinded study, meaning participants in both conditions were informed as to whether their weekly check-in results would be viewable to their clinician.
Patient weekly check-ins.
Study participants were sent invitations to complete weekly check-ins via text message or email depending on their preference.
At the initial study appointment, the researchers encouraged participants to complete weekly check-ins in a timely fashion. They then monitored check-in completion throughout the study period, and for the first 12 weeks of the study contacted participants to offer support completing check-ins. For participants in the weekly check in + clinician dashboard condition, the researchers encouraged participants to answer weekly check-in questionnaires knowing that their clinician would review their responses.
The weekly check-in included 22 questions derived from existing questionnaires. Questions assessed past-week drinking and other drug use (Figure 1A), past-week experiences with craving, coping skills, abstinence self-efficacy (one’s confidence that they can maintain abstinence), depression symptoms, positive life outlook, and therapeutic alliance (Figure 1B). Questions also assessed goals for the upcoming week related to reducing substance use and cravings, learning more effective coping strategies, increasing abstinence self-efficacy and improving mental health, and having a more positive outlook (Figure 1C). Two optional, open-ended questions also allowed participants to share any additional goals for the upcoming week and information that they wanted their clinician to have (Figure 1D).
Figure 1. Screenshots of the measurement-based care weekly questionnaires receive by patients.
The clinician dashboard.
Clinician participants were given access to a web-based dashboard on which they could review the summarized results from the weekly patient check-ins. The dashboard displayed change in measures across time using line graphs (Figure 2A), text-based summaries (Figure 2B), bar graphs showing the most recent responses to each question (Figure 2C), and a table showing participants past-week question responses (Figure 2D). Clinicians received email reminders to review the dashboard every 2 weeks.
Figure 2. Screenshots of the clinician dashboard showing summarized, weekly results from patient questionnaires.
Study participants were found to be similar to the full clinic population in terms of age, gender, race, and ethnicity. About two-thirds of patients were aged 35–54 (63%), about half were male (53%), and White (57%). Just over three-quarters (77%) had an associate’s degree, trade degree, or higher education degree. A large majority of patients were not currently employed (90%), and just over half (53%) were experiencing homelessness. Most participants had symptoms consistent with severe substance use disorder (77%). Participants’ primary substance included stimulants (60%), opioids (53%), alcohol (50%), cannabis (17%), sedatives (1%), and hallucinogens (<1%).
All 30 patient participants completed the baseline and 6-week research appointments, with all except one completing the 12- and 24-week follow-ups.
Twenty-nine patient participants chose to receive weekly check-in prompts via text and one elected to receive them by email.
Eight clinicians participated in the study. Six had at least 1 patient who was enrolled in the weekly check-in + clinician dashboard condition (median = 2 patients per clinician) allowing the clinician to review patient progress in the clinician dashboard. Clinicians were asked to provide feedback on the dashboard by completing a dashboard usability questionnaire.
The patient response rate to check-ins was high.
On average, participants completed around 21 weekly check-ins (~86%). All participants completed the first 2 weekly check-ins, with the proportion of patients completing the weekly check-ins decreasing over time but then stabilizing. By week 12, 80% of participants completed the weekly check-in with this percentage remaining fairly stable through study week 24.
Participants in the weekly check-in only condition and the weekly check-in + clinician dashboard condition were similar in terms of the number of weekly check-ins completed.
Clinicians may have under-utilized the dashboard.
Five out of six clinicians who had a patient in the weekly check-in + clinician dashboard condition logged into the clinician dashboard at least once. Among them, there was a mean of approximately 12 logins per clinician (min= 3 / max= 25). Login sessions were, on average, around 2 minutes long.
Patient and provider usability ratings were positive.
Usability was deemed to be good when assessed at the 6- and 24-week time points, with most patients (86–100%) agreeing or strongly agreeing that the weekly check-ins were helpful and that they would be willing to use this tool again in the future. Most also indicated that they would recommend the check-in tool to others. Most participants (86%) felt the length of the weekly check-ins was “just right”. Most also reported feeling confident in their ability to complete check-ins independently or during treatment sessions (83–100%) with a small portion of the sample feeling that they would experience stress completing weekly check-ins (7–14%).
Five out of 6 clinicians completed the dashboard usability questionnaire. Generally speaking, their ratings were favorable, with all responding clinicians feeling the dashboard was easy to navigate, and that the information provided was meaningful and useful for treatment planning. Most also felt that the information was helpful to their patients.
Results from this study provide preliminary support for the acceptability and feasibility of a digital ‘check-in’ system in substance use disorder outpatient treatment settings. Among patients and clinicians in this pilot study, rates of engagement with the care system were high for patients with an 86% check-in completion rate, but less robust for clinicians who logged into the dashboard a mean of around 12 times.
By their report, check-ins imposed very little burden on patients, who on average took less than 5 minutes to complete each weekly check-in. Usability ratings were also favorable, with most patients feeling the weekly check-ins aided self-reflection, and something they could see themselves continuing to use. Additionally, most clinicians felt that the information on the dashboard was helpful, noting that they discussed the provided information with their patients.
Notably, the vast majority of patient participants (97%) indicated a preference for receiving the weekly questionnaires via text on their smartphone, versus through email. Based on this finding, future digital care practices such as the one assessed in this study may do best to focus on smartphone/app-based assessments rather than email.
Taken together, findings from this research are important because measurement-based care is known to improve addiction recovery outcomes. At the same time, these findings may not accurately reflect how measurement-based care may be accepted and utilized in other outpatient substance use disorder treatment settings. It is possible that clinicians may be resistant to utilizing such care systems, perceiving them to be additional work on top of already busy schedules, or may be uncertain about how to integrate these new systems into their clinical practice. Healthcare systems may also struggle to adopt measurement-based care systems due to, 1) a lack of training and technical support, 2) insurance reimbursement models that do not cover the time spent using measurement-based care tools, and 3) limited integration of measurement-based care systems with other treatment technologies. Some treatment systems may also be hesitant to integrate measurement-based care into their clinics as it may expose them to outcomes-based reimbursement from insurers who are increasingly moving to ‘pay for performance’ models of reimbursement.
As noted by the authors:
In addition:
Results from this clinical pilot study suggest that the measurement-based care system developed and assessed by these researchers can be incorporated into a typical substance use disorder treatment setting, and that patients and providers will engage with it. Further, these preliminary findings suggest this care system is helpful for patients and providers. This complements previous work that has suggested that implementation of measurement-based care can improve substance use disorder treatment outcomes. At the same time, clinician engagement was difficult to determine by the results presented, and it is possible clinicians under-utilized the patient reports. Additionally, clinicians volunteering to participate in such studies may be more motivated to use such technology and may not reflect the average community substance use disorder provider. More research is needed to assess potential barriers to implementing measurement-based care in substance use disorder treatment, as well as to determine how these kinds of interventions influence patient outcomes.
Hallgren, K. A., Cohn, E. B., Ries, R. K., & Atkins, D. C. (2022). Delivering remote measurement-based care in community addiction treatment: engagement and usability over a 6-month clinical pilot. Frontiers in Psychiatry, 13, 840409. doi: 10.3389/fpsyt.2022.840409
l
Measurement-based care is standard practice in medicine and has been tested many times in non-substance use disorder mental health treatment settings, where it is generally associated with better treatment retention and outcomes and a better ability for clinicians to detect stagnation in treatment progress and modify treatment accordingly. However, the application of measurement-based care in specialty substance use disorder treatment settings has been limited, even though it is possible that the benefits of measurement-based care observed in other mental health treatment settings would likely extend to substance use disorder treatment settings.
In this study, researchers developed a measurement-based care system designed to be used in outpatient substance use disorder treatment settings. Following a design framework centered around the patient, the prototype was refined based on several rounds of patient and clinician feedback. The resulting system had two main components: 1) a patient-facing care questionnaire they called the weekly check-in, and 2) a web-based clinician-facing dashboard for reviewing care results the referred to as the clinician dashboard.
This pilot study evaluated the feasibility and acceptability of this care system in the context of a typical United States outpatient substance use disorder treatment clinic. The researchers reported outcomes related to patients and clinicians’ engagement with the care system and their assessments of its usability and clinical utility when using this system in the context of substance use disorder treatment as usual.
This was a non-randomized, proof-of-concept, pilot study with 30 patient participants who completed weekly questionnaires designed to track treatment progress or completed these weekly questionnaires + their results were shared with their primary clinic provider, to assess the feasibility and acceptability of this measurement-based care approach. All participants were asked to complete the weekly questionnaires for 6 months. Six clinician providers also participated in this study.
All participants were recruited from the same large, public, addiction and mental health treatment clinic in Washington State, USA. The clinic offered: 1) individual psychotherapy, 2) case management, 3) group therapy, 4) peer support, and 5) medication management for psychiatric conditions including substance use disorder.
To be eligible for the study, patients needed to, 1) be receiving treatment for substance use disorder from a clinician in the study, 2) own a smartphone, 3) be able to speak and read English, 4) be over 18 years old, 5) have past-year unhealthy alcohol use reflected by an Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) score of 3 (women) or 4 (men) suggesting alcohol use disorder, and/or 6) use of illicit or non-prescribed drugs in the past-year.
Study recruitment ran from October 2019–June 2021, with a pause between March and June 2020 because of COVID-19.
At study baseline, participants were assigned to one of two conditions based on when they enrolled in the study. Patients who enrolled in the first wave were assigned to the ‘weekly check-in only’ condition, in which they completed weekly check-in questionnaires, however their clinicians did not have access to their check-in results.
Patients who enrolled in the second wave were assigned to ‘weekly check-in + clinician dashboard’. In this condition, the results of weekly check-ins were accessible to the participant’s clinician via the clinician dashboard.
This was an unblinded study, meaning participants in both conditions were informed as to whether their weekly check-in results would be viewable to their clinician.
Patient weekly check-ins.
Study participants were sent invitations to complete weekly check-ins via text message or email depending on their preference.
At the initial study appointment, the researchers encouraged participants to complete weekly check-ins in a timely fashion. They then monitored check-in completion throughout the study period, and for the first 12 weeks of the study contacted participants to offer support completing check-ins. For participants in the weekly check in + clinician dashboard condition, the researchers encouraged participants to answer weekly check-in questionnaires knowing that their clinician would review their responses.
The weekly check-in included 22 questions derived from existing questionnaires. Questions assessed past-week drinking and other drug use (Figure 1A), past-week experiences with craving, coping skills, abstinence self-efficacy (one’s confidence that they can maintain abstinence), depression symptoms, positive life outlook, and therapeutic alliance (Figure 1B). Questions also assessed goals for the upcoming week related to reducing substance use and cravings, learning more effective coping strategies, increasing abstinence self-efficacy and improving mental health, and having a more positive outlook (Figure 1C). Two optional, open-ended questions also allowed participants to share any additional goals for the upcoming week and information that they wanted their clinician to have (Figure 1D).
Figure 1. Screenshots of the measurement-based care weekly questionnaires receive by patients.
The clinician dashboard.
Clinician participants were given access to a web-based dashboard on which they could review the summarized results from the weekly patient check-ins. The dashboard displayed change in measures across time using line graphs (Figure 2A), text-based summaries (Figure 2B), bar graphs showing the most recent responses to each question (Figure 2C), and a table showing participants past-week question responses (Figure 2D). Clinicians received email reminders to review the dashboard every 2 weeks.
Figure 2. Screenshots of the clinician dashboard showing summarized, weekly results from patient questionnaires.
Study participants were found to be similar to the full clinic population in terms of age, gender, race, and ethnicity. About two-thirds of patients were aged 35–54 (63%), about half were male (53%), and White (57%). Just over three-quarters (77%) had an associate’s degree, trade degree, or higher education degree. A large majority of patients were not currently employed (90%), and just over half (53%) were experiencing homelessness. Most participants had symptoms consistent with severe substance use disorder (77%). Participants’ primary substance included stimulants (60%), opioids (53%), alcohol (50%), cannabis (17%), sedatives (1%), and hallucinogens (<1%).
All 30 patient participants completed the baseline and 6-week research appointments, with all except one completing the 12- and 24-week follow-ups.
Twenty-nine patient participants chose to receive weekly check-in prompts via text and one elected to receive them by email.
Eight clinicians participated in the study. Six had at least 1 patient who was enrolled in the weekly check-in + clinician dashboard condition (median = 2 patients per clinician) allowing the clinician to review patient progress in the clinician dashboard. Clinicians were asked to provide feedback on the dashboard by completing a dashboard usability questionnaire.
The patient response rate to check-ins was high.
On average, participants completed around 21 weekly check-ins (~86%). All participants completed the first 2 weekly check-ins, with the proportion of patients completing the weekly check-ins decreasing over time but then stabilizing. By week 12, 80% of participants completed the weekly check-in with this percentage remaining fairly stable through study week 24.
Participants in the weekly check-in only condition and the weekly check-in + clinician dashboard condition were similar in terms of the number of weekly check-ins completed.
Clinicians may have under-utilized the dashboard.
Five out of six clinicians who had a patient in the weekly check-in + clinician dashboard condition logged into the clinician dashboard at least once. Among them, there was a mean of approximately 12 logins per clinician (min= 3 / max= 25). Login sessions were, on average, around 2 minutes long.
Patient and provider usability ratings were positive.
Usability was deemed to be good when assessed at the 6- and 24-week time points, with most patients (86–100%) agreeing or strongly agreeing that the weekly check-ins were helpful and that they would be willing to use this tool again in the future. Most also indicated that they would recommend the check-in tool to others. Most participants (86%) felt the length of the weekly check-ins was “just right”. Most also reported feeling confident in their ability to complete check-ins independently or during treatment sessions (83–100%) with a small portion of the sample feeling that they would experience stress completing weekly check-ins (7–14%).
Five out of 6 clinicians completed the dashboard usability questionnaire. Generally speaking, their ratings were favorable, with all responding clinicians feeling the dashboard was easy to navigate, and that the information provided was meaningful and useful for treatment planning. Most also felt that the information was helpful to their patients.
Results from this study provide preliminary support for the acceptability and feasibility of a digital ‘check-in’ system in substance use disorder outpatient treatment settings. Among patients and clinicians in this pilot study, rates of engagement with the care system were high for patients with an 86% check-in completion rate, but less robust for clinicians who logged into the dashboard a mean of around 12 times.
By their report, check-ins imposed very little burden on patients, who on average took less than 5 minutes to complete each weekly check-in. Usability ratings were also favorable, with most patients feeling the weekly check-ins aided self-reflection, and something they could see themselves continuing to use. Additionally, most clinicians felt that the information on the dashboard was helpful, noting that they discussed the provided information with their patients.
Notably, the vast majority of patient participants (97%) indicated a preference for receiving the weekly questionnaires via text on their smartphone, versus through email. Based on this finding, future digital care practices such as the one assessed in this study may do best to focus on smartphone/app-based assessments rather than email.
Taken together, findings from this research are important because measurement-based care is known to improve addiction recovery outcomes. At the same time, these findings may not accurately reflect how measurement-based care may be accepted and utilized in other outpatient substance use disorder treatment settings. It is possible that clinicians may be resistant to utilizing such care systems, perceiving them to be additional work on top of already busy schedules, or may be uncertain about how to integrate these new systems into their clinical practice. Healthcare systems may also struggle to adopt measurement-based care systems due to, 1) a lack of training and technical support, 2) insurance reimbursement models that do not cover the time spent using measurement-based care tools, and 3) limited integration of measurement-based care systems with other treatment technologies. Some treatment systems may also be hesitant to integrate measurement-based care into their clinics as it may expose them to outcomes-based reimbursement from insurers who are increasingly moving to ‘pay for performance’ models of reimbursement.
As noted by the authors:
In addition:
Results from this clinical pilot study suggest that the measurement-based care system developed and assessed by these researchers can be incorporated into a typical substance use disorder treatment setting, and that patients and providers will engage with it. Further, these preliminary findings suggest this care system is helpful for patients and providers. This complements previous work that has suggested that implementation of measurement-based care can improve substance use disorder treatment outcomes. At the same time, clinician engagement was difficult to determine by the results presented, and it is possible clinicians under-utilized the patient reports. Additionally, clinicians volunteering to participate in such studies may be more motivated to use such technology and may not reflect the average community substance use disorder provider. More research is needed to assess potential barriers to implementing measurement-based care in substance use disorder treatment, as well as to determine how these kinds of interventions influence patient outcomes.
Hallgren, K. A., Cohn, E. B., Ries, R. K., & Atkins, D. C. (2022). Delivering remote measurement-based care in community addiction treatment: engagement and usability over a 6-month clinical pilot. Frontiers in Psychiatry, 13, 840409. doi: 10.3389/fpsyt.2022.840409
SIGN UP FOR THE BULLETIN
