Research assessments for alcohol use disorder patients are therapeutic in their own right
Accurately measuring outcomes in studies of alcohol use disorder treatment is vital to test and to improve clinical interventions and treatment. Yet, the actual act of evaluating progress is itself not neutral – it may also encourage positive change. In fact, prior research has found that more research assessment among alcohol patients improves alcohol-related outcomes. This study extended this line of research by investigating if the frequency and comprehensiveness of assessments for alcohol use disorder treatment also were associated with other drug use and related consequences.
Alcohol contributes to more than 200 diseases and injury-related health conditions, including liver disease, cancer, and suicide. Much work is being done to develop effective treatments. For example, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) has a 2023 budget of $595.3 million, with millions devoted specifically to alcohol treatment and recovery research. Yet, designing and implementing research to evaluate treatment and recovery is costly, time consuming, and fallible. Clinical trials of drug treatments, when done under very controlled conditions and with participants checked regularly after treatment, tend to show bigger impacts on drug use than what happens when the same treatments are used in real-world settings.
Furthermore, participating in research can confer its own benefits above and beyond the treatment being studied. The observed therapeutic benefits of participating in research highlights the issue of so called, ‘measurement reactivity’ – changes in behavior due to being studied. This effect needs to be accounted for to isolate an intervention’s true and independent impact, to help estimate the expected benefit in real-world clinical settings without ongoing, regular research assessments.
Measurement reactivity specific to alcohol treatment research has been studied for over 50 years. However, findings have been mixed and seem to depend on the population, outcomes, and mode of research being studied. Two potential determinants of measurement reactivity are the comprehensiveness and frequency of the assessment.
When research participants complete longer assessments, more frequent assessment or longer assessments more frequently, they may exhibit measurement reactivity differently than someone participating in shorter, less frequent, or less frequent brief assessments.
In a prior study of measurement reactivity, research participants that received frequent comprehensive assessments reported less alcohol use and fewer alcohol-related consequences than those that received infrequent-brief assessments. This study extends that work to examine drug use apart from alcohol. The purpose of this study was to examine to the extent to which frequency and comprehensiveness of an alcohol use disorder treatment research protocol contributes to reductions in drug use and related consequences.
HOW WAS THIS STUDY CONDUCTED?
Study participants were adults recruited from two hospital-based substance use disorder treatment outpatient clinics. The outpatient programs offered similar substance use services, including individual and group counseling, education, and support. Patients typically received a combination of individual and group treatment sessions. All treatment providers had relevant master’s degrees and were certified Alcohol and Substance Abuse Counselors (CASAC). Participants were recruited when they presented for outpatient alcohol treatment. They were then randomized to one of four conditions.
There were two assessment frequency and two assessment comprehensiveness designs that were combined in a 2 by 2 factorial design. The infrequent assessment conditions consisted of two in-person follow-up interviews after study enrollment (6- and 12-months post-baseline). The frequent follow-up conditions included four in-person follow-up interviews (3-, 6-, 9-, and 12-months post-baseline) and telephone interviews during the intervening months (1-, 2-, 4-, 5-, 7-, 8-, 10-, and 11 post-baseline). The telephone interviews only focused on alcohol and other substance use and were, on average, only 5 to 10 minutes long. The comprehensive conditions include a broad range of component parts, including 15 different scales and measures. The comprehensive assessment took, on average, 3.5 hours at baseline and 1.5 hours at each in-person follow-up to complete. The brief assessment conditions were much more limited and took, on average, 1.5 hours to administer at baseline and 20–30 minutes at follow-ups to complete.
The study then looked at the association between frequency (i.e., frequent vs. infrequent), comprehensiveness (i.e., comprehensive vs. brief), and frequency by comprehensiveness (frequent-brief, frequent-comprehensive, infrequent-brief, infrequent-comprehensive) and three specific outcomes. The first was the proportion of days that the participant reported any nonmedical (i.e., non-prescribed) drug use. The second was complete abstinence from alcohol and other drugs. The third was drug use-related negative consequences and was assessed by examining the frequency of 45 drug-related negative consequences that occurred in the last 3 months. The negative consequences were also broken down into five domains: physical, interpersonal, intrapersonal, impulse control, and social responsibility. For the analysis, the outcomes were assessed by categorizing the data into 3 periods – baseline, follow-up months 1–6, and follow-up months 7–12. The time periods aligned with the in-person interviews for all participants and allowed comparison across the four groups.
A total of 235 adults participated in the study. They were, on average, 40 years old and most were White (75%), male (63%), and single (77%). About half (53%) had a high school education or less, and 58% had an income of $19,000 or less. More than two-thirds (69%) had received prior substance use treatment. All the participants in this study presented for treatment with alcohol use disorder as their primary diagnosis.
WHAT DID THIS STUDY FIND?
Frequent assessments were linked with fewer drug use days.
Participants in the two frequent assessment conditions reported a similar improvement in drug use days compared to those in the infrequent conditions between baseline and 6-month follow-up. By the 12-month follow-up, however, those in the infrequent conditions reported an increase in the drug use days while those in the frequent conditions continued to improve. Of note, frequency of assessments only accounted for about 2% of the participants’ drug use days, which is a relatively small effect. Comprehensiveness of assessment was not found to be related to drug use days. The effect of frequency also did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on drug use days). In other words, the effect of assessment frequency on reducing drug use remained at about the same magnitude regardless of how many measures the patients were asked to complete at any given time.
Frequent assessments were associated with greater likelihood of complete abstinence and fewer negative consequences.
A greater proportion of participants in the frequent conditions reported abstinence from alcohol and other drugs during both the 1–6 (29% vs 16%) and 7–12-month (43% vs. 34%) follow-up periods compared to those in the infrequent conditions. Those in the frequent assessment conditions also had fewer total negative consequences and fewer consequences on each of the five subscales: physical-, interpersonal-, intrapersonal-, impulse-, and social responsibility-related negative consequences compared to those in the infrequent conditions.
Comprehensiveness was not found to be related to abstinence or negative consequences. Similar to the link between frequency and drug use days, the effect of frequency on abstinence and negative consequences did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on odds of abstinence or negative consequences).
WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?
The researchers in this study found that the frequency but not the comprehensiveness of assessments was tied to drug use days, odds of abstinence, and drug use-related consequences.
Individuals that were in the frequent assessment conditions (frequent-brief and frequent-comprehensive) reported fewer days of drug use, a higher likelihood of complete abstinence, and fewer negative consequences compared to those in the infrequent conditions (infrequent-brief and infrequent-comprehensive). Thus, the researchers found that more frequent assessment itself can likely improve drug use outcomes beyond effects from treatment for adults with alcohol use disorder that received outpatient treatment.
This effect was especially apparent during the 6-to-12-month period after baseline, which suggests that the frequent assessment during this second follow-up period is critical to sustaining or enhancing treatment benefits. In other words, ongoing clinical monitoring and continuing care may improve a range of treatment outcomes compared to continuing care as usual, a result supported by meta-analytic research (a study of studies). One study even found that continuing care provided solely through telephone monitoring was associated with more abstinent days and cost less than treatment as usual. Other continuity of care treatment studies also have found that the “extensity” (i.e., continuity of care over time) is more important to improving clinical outcomes than the intensity of care (how much is delivered during a briefer period).
However, the mechanisms through which continuing care and measurement reactivity occurs remains underexplored. It could be the expectations of assessment increases motivation to align with socially desirable outcomes (e.g., reduced drug use). Alternatively, interacting with research staff may facilitate a therapeutic treatment booster through which individuals are able to reflect more deeply on the past impact of their drug use, potentially spurring motivation for change. Future research is needed to understand what causes measurement reactivity, how to account for it in treatment studies, and how to leverage those mechanisms for enhanced interventions.
The participants in this study were primarily White, male, and single, which limits generalizability of the findings to populations not exhibiting these kinds of characteristics.
Only individuals that expressed interest in a research study about follow-ups at treatment presentations were included. Those that declined interest may be different (e.g., higher addiction severity level) from those that participated.
Although all individuals were recruited from two outpatient treatment clinics, no measures of treatment participation or completion were included. It is unclear whether the frequency of assessment follow-up may have facilitated more treatment engagement, which could have resulted in improved outcomes.
The study did not have a true control group (i.e., a group that received little or no treatment). Thus, it is unknown if some aspect of treatment interacted with the frequency and comprehensiveness of assessments and impacted outcomes.
BOTTOM LINE
Frequent assessment of treatment outcomes, compared to infrequent assessment, following alcohol use disorder outpatient treatment was associated with decreased drug use days, increased likelihood of abstinence, and fewer drug use-related negative consequences. Although improved outcomes are generally desirable, the reactivity to measurement that was observed suggests that the frequency of treatment assessment clouds researchers understanding of standalone treatment effects. Nevertheless, these results highlight the benefits of continuing care and ongoing clinical monitoring. On-going check-ins, especially 6-to-12 months following treatment entry, in real-world clinical settings may be critical to maximize the benefits conferred from treatment. Future research is needed to understand what aspects related to the frequency of assessment may impact treatment and how to account for measurement reactivity in evaluation of treatment.
For individuals and families seeking recovery: This study found that frequent assessments within a treatment research project were linked with improved outcomes compared to infrequent assessments. Individuals or families seeking treatment services should look for programs that offer empirically supported continuing care to leverage the benefits of frequent, on-going monitoring, interpersonal contact, and outcomes measurement.
For treatment professionals and treatment systems: Understanding the relationship between treatment components (e.g., treatment modality such as motivational interviewing) and treatment outcomes (e.g., drug use and related consequences) is vital to develop and test treatment interventions. However, this study found significant measurement reactivity–frequent assessment of outcomes was linked with changes in outcomes compared to infrequent assessment. To isolate the effect of treatment, researchers and evaluators might consider reducing the frequency of follow-ups. However, when developing a continuing care plan in practice, it may be beneficial to incorporate frequent follow-ups. Treatment programs and interventions should develop continuing care components to leverage the benefits of frequent, on-going measurement.
For scientists: This study randomized participants in a 2 by 2 factorial design crossing frequent and infrequent and comprehensive and brief assessments. They found that frequent compared to infrequent, but not comprehensiveness or the frequency-comprehensiveness interaction, was related to treatment outcomes. The findings of this study demonstrate the need to account for and control assessment reactivity when designing studies and interpreting their results. Frequent assessment may obscure detection of treatment effects. Future studies can seek to isolate the dose-response of frequency through larger trials, especially those incorporating a true control condition with no or minimal assessment exposure. Further work is also needed to understand the mechanisms that underly benefits conferred from research participation independently from the treatment being studied.
For policy makers: This study found that frequent assessment was linked with improved drug use outcomes compared to infrequent assessment. Policies and interventions that allow for increased methodological rigor (i.e., increased funding) are likely to facilitate better understanding of measurement reactivity and independent treatment effects. An improved understanding of standalone treatment effects is likely to improve research efficiency and validity that can translate to improved lives and reduced societal costs. Furthermore, additional funding is needed to support continuing care after initial treatment episodes to maximize impact and get real-world outcomes as close as possible to rigorous clinical research students that often include long-term, regular assessments.
Alcohol contributes to more than 200 diseases and injury-related health conditions, including liver disease, cancer, and suicide. Much work is being done to develop effective treatments. For example, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) has a 2023 budget of $595.3 million, with millions devoted specifically to alcohol treatment and recovery research. Yet, designing and implementing research to evaluate treatment and recovery is costly, time consuming, and fallible. Clinical trials of drug treatments, when done under very controlled conditions and with participants checked regularly after treatment, tend to show bigger impacts on drug use than what happens when the same treatments are used in real-world settings.
Furthermore, participating in research can confer its own benefits above and beyond the treatment being studied. The observed therapeutic benefits of participating in research highlights the issue of so called, ‘measurement reactivity’ – changes in behavior due to being studied. This effect needs to be accounted for to isolate an intervention’s true and independent impact, to help estimate the expected benefit in real-world clinical settings without ongoing, regular research assessments.
Measurement reactivity specific to alcohol treatment research has been studied for over 50 years. However, findings have been mixed and seem to depend on the population, outcomes, and mode of research being studied. Two potential determinants of measurement reactivity are the comprehensiveness and frequency of the assessment.
When research participants complete longer assessments, more frequent assessment or longer assessments more frequently, they may exhibit measurement reactivity differently than someone participating in shorter, less frequent, or less frequent brief assessments.
In a prior study of measurement reactivity, research participants that received frequent comprehensive assessments reported less alcohol use and fewer alcohol-related consequences than those that received infrequent-brief assessments. This study extends that work to examine drug use apart from alcohol. The purpose of this study was to examine to the extent to which frequency and comprehensiveness of an alcohol use disorder treatment research protocol contributes to reductions in drug use and related consequences.
HOW WAS THIS STUDY CONDUCTED?
Study participants were adults recruited from two hospital-based substance use disorder treatment outpatient clinics. The outpatient programs offered similar substance use services, including individual and group counseling, education, and support. Patients typically received a combination of individual and group treatment sessions. All treatment providers had relevant master’s degrees and were certified Alcohol and Substance Abuse Counselors (CASAC). Participants were recruited when they presented for outpatient alcohol treatment. They were then randomized to one of four conditions.
There were two assessment frequency and two assessment comprehensiveness designs that were combined in a 2 by 2 factorial design. The infrequent assessment conditions consisted of two in-person follow-up interviews after study enrollment (6- and 12-months post-baseline). The frequent follow-up conditions included four in-person follow-up interviews (3-, 6-, 9-, and 12-months post-baseline) and telephone interviews during the intervening months (1-, 2-, 4-, 5-, 7-, 8-, 10-, and 11 post-baseline). The telephone interviews only focused on alcohol and other substance use and were, on average, only 5 to 10 minutes long. The comprehensive conditions include a broad range of component parts, including 15 different scales and measures. The comprehensive assessment took, on average, 3.5 hours at baseline and 1.5 hours at each in-person follow-up to complete. The brief assessment conditions were much more limited and took, on average, 1.5 hours to administer at baseline and 20–30 minutes at follow-ups to complete.
The study then looked at the association between frequency (i.e., frequent vs. infrequent), comprehensiveness (i.e., comprehensive vs. brief), and frequency by comprehensiveness (frequent-brief, frequent-comprehensive, infrequent-brief, infrequent-comprehensive) and three specific outcomes. The first was the proportion of days that the participant reported any nonmedical (i.e., non-prescribed) drug use. The second was complete abstinence from alcohol and other drugs. The third was drug use-related negative consequences and was assessed by examining the frequency of 45 drug-related negative consequences that occurred in the last 3 months. The negative consequences were also broken down into five domains: physical, interpersonal, intrapersonal, impulse control, and social responsibility. For the analysis, the outcomes were assessed by categorizing the data into 3 periods – baseline, follow-up months 1–6, and follow-up months 7–12. The time periods aligned with the in-person interviews for all participants and allowed comparison across the four groups.
A total of 235 adults participated in the study. They were, on average, 40 years old and most were White (75%), male (63%), and single (77%). About half (53%) had a high school education or less, and 58% had an income of $19,000 or less. More than two-thirds (69%) had received prior substance use treatment. All the participants in this study presented for treatment with alcohol use disorder as their primary diagnosis.
WHAT DID THIS STUDY FIND?
Frequent assessments were linked with fewer drug use days.
Participants in the two frequent assessment conditions reported a similar improvement in drug use days compared to those in the infrequent conditions between baseline and 6-month follow-up. By the 12-month follow-up, however, those in the infrequent conditions reported an increase in the drug use days while those in the frequent conditions continued to improve. Of note, frequency of assessments only accounted for about 2% of the participants’ drug use days, which is a relatively small effect. Comprehensiveness of assessment was not found to be related to drug use days. The effect of frequency also did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on drug use days). In other words, the effect of assessment frequency on reducing drug use remained at about the same magnitude regardless of how many measures the patients were asked to complete at any given time.
Frequent assessments were associated with greater likelihood of complete abstinence and fewer negative consequences.
A greater proportion of participants in the frequent conditions reported abstinence from alcohol and other drugs during both the 1–6 (29% vs 16%) and 7–12-month (43% vs. 34%) follow-up periods compared to those in the infrequent conditions. Those in the frequent assessment conditions also had fewer total negative consequences and fewer consequences on each of the five subscales: physical-, interpersonal-, intrapersonal-, impulse-, and social responsibility-related negative consequences compared to those in the infrequent conditions.
Comprehensiveness was not found to be related to abstinence or negative consequences. Similar to the link between frequency and drug use days, the effect of frequency on abstinence and negative consequences did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on odds of abstinence or negative consequences).
WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?
The researchers in this study found that the frequency but not the comprehensiveness of assessments was tied to drug use days, odds of abstinence, and drug use-related consequences.
Individuals that were in the frequent assessment conditions (frequent-brief and frequent-comprehensive) reported fewer days of drug use, a higher likelihood of complete abstinence, and fewer negative consequences compared to those in the infrequent conditions (infrequent-brief and infrequent-comprehensive). Thus, the researchers found that more frequent assessment itself can likely improve drug use outcomes beyond effects from treatment for adults with alcohol use disorder that received outpatient treatment.
This effect was especially apparent during the 6-to-12-month period after baseline, which suggests that the frequent assessment during this second follow-up period is critical to sustaining or enhancing treatment benefits. In other words, ongoing clinical monitoring and continuing care may improve a range of treatment outcomes compared to continuing care as usual, a result supported by meta-analytic research (a study of studies). One study even found that continuing care provided solely through telephone monitoring was associated with more abstinent days and cost less than treatment as usual. Other continuity of care treatment studies also have found that the “extensity” (i.e., continuity of care over time) is more important to improving clinical outcomes than the intensity of care (how much is delivered during a briefer period).
However, the mechanisms through which continuing care and measurement reactivity occurs remains underexplored. It could be the expectations of assessment increases motivation to align with socially desirable outcomes (e.g., reduced drug use). Alternatively, interacting with research staff may facilitate a therapeutic treatment booster through which individuals are able to reflect more deeply on the past impact of their drug use, potentially spurring motivation for change. Future research is needed to understand what causes measurement reactivity, how to account for it in treatment studies, and how to leverage those mechanisms for enhanced interventions.
The participants in this study were primarily White, male, and single, which limits generalizability of the findings to populations not exhibiting these kinds of characteristics.
Only individuals that expressed interest in a research study about follow-ups at treatment presentations were included. Those that declined interest may be different (e.g., higher addiction severity level) from those that participated.
Although all individuals were recruited from two outpatient treatment clinics, no measures of treatment participation or completion were included. It is unclear whether the frequency of assessment follow-up may have facilitated more treatment engagement, which could have resulted in improved outcomes.
The study did not have a true control group (i.e., a group that received little or no treatment). Thus, it is unknown if some aspect of treatment interacted with the frequency and comprehensiveness of assessments and impacted outcomes.
BOTTOM LINE
Frequent assessment of treatment outcomes, compared to infrequent assessment, following alcohol use disorder outpatient treatment was associated with decreased drug use days, increased likelihood of abstinence, and fewer drug use-related negative consequences. Although improved outcomes are generally desirable, the reactivity to measurement that was observed suggests that the frequency of treatment assessment clouds researchers understanding of standalone treatment effects. Nevertheless, these results highlight the benefits of continuing care and ongoing clinical monitoring. On-going check-ins, especially 6-to-12 months following treatment entry, in real-world clinical settings may be critical to maximize the benefits conferred from treatment. Future research is needed to understand what aspects related to the frequency of assessment may impact treatment and how to account for measurement reactivity in evaluation of treatment.
For individuals and families seeking recovery: This study found that frequent assessments within a treatment research project were linked with improved outcomes compared to infrequent assessments. Individuals or families seeking treatment services should look for programs that offer empirically supported continuing care to leverage the benefits of frequent, on-going monitoring, interpersonal contact, and outcomes measurement.
For treatment professionals and treatment systems: Understanding the relationship between treatment components (e.g., treatment modality such as motivational interviewing) and treatment outcomes (e.g., drug use and related consequences) is vital to develop and test treatment interventions. However, this study found significant measurement reactivity–frequent assessment of outcomes was linked with changes in outcomes compared to infrequent assessment. To isolate the effect of treatment, researchers and evaluators might consider reducing the frequency of follow-ups. However, when developing a continuing care plan in practice, it may be beneficial to incorporate frequent follow-ups. Treatment programs and interventions should develop continuing care components to leverage the benefits of frequent, on-going measurement.
For scientists: This study randomized participants in a 2 by 2 factorial design crossing frequent and infrequent and comprehensive and brief assessments. They found that frequent compared to infrequent, but not comprehensiveness or the frequency-comprehensiveness interaction, was related to treatment outcomes. The findings of this study demonstrate the need to account for and control assessment reactivity when designing studies and interpreting their results. Frequent assessment may obscure detection of treatment effects. Future studies can seek to isolate the dose-response of frequency through larger trials, especially those incorporating a true control condition with no or minimal assessment exposure. Further work is also needed to understand the mechanisms that underly benefits conferred from research participation independently from the treatment being studied.
For policy makers: This study found that frequent assessment was linked with improved drug use outcomes compared to infrequent assessment. Policies and interventions that allow for increased methodological rigor (i.e., increased funding) are likely to facilitate better understanding of measurement reactivity and independent treatment effects. An improved understanding of standalone treatment effects is likely to improve research efficiency and validity that can translate to improved lives and reduced societal costs. Furthermore, additional funding is needed to support continuing care after initial treatment episodes to maximize impact and get real-world outcomes as close as possible to rigorous clinical research students that often include long-term, regular assessments.
Alcohol contributes to more than 200 diseases and injury-related health conditions, including liver disease, cancer, and suicide. Much work is being done to develop effective treatments. For example, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) has a 2023 budget of $595.3 million, with millions devoted specifically to alcohol treatment and recovery research. Yet, designing and implementing research to evaluate treatment and recovery is costly, time consuming, and fallible. Clinical trials of drug treatments, when done under very controlled conditions and with participants checked regularly after treatment, tend to show bigger impacts on drug use than what happens when the same treatments are used in real-world settings.
Furthermore, participating in research can confer its own benefits above and beyond the treatment being studied. The observed therapeutic benefits of participating in research highlights the issue of so called, ‘measurement reactivity’ – changes in behavior due to being studied. This effect needs to be accounted for to isolate an intervention’s true and independent impact, to help estimate the expected benefit in real-world clinical settings without ongoing, regular research assessments.
Measurement reactivity specific to alcohol treatment research has been studied for over 50 years. However, findings have been mixed and seem to depend on the population, outcomes, and mode of research being studied. Two potential determinants of measurement reactivity are the comprehensiveness and frequency of the assessment.
When research participants complete longer assessments, more frequent assessment or longer assessments more frequently, they may exhibit measurement reactivity differently than someone participating in shorter, less frequent, or less frequent brief assessments.
In a prior study of measurement reactivity, research participants that received frequent comprehensive assessments reported less alcohol use and fewer alcohol-related consequences than those that received infrequent-brief assessments. This study extends that work to examine drug use apart from alcohol. The purpose of this study was to examine to the extent to which frequency and comprehensiveness of an alcohol use disorder treatment research protocol contributes to reductions in drug use and related consequences.
HOW WAS THIS STUDY CONDUCTED?
Study participants were adults recruited from two hospital-based substance use disorder treatment outpatient clinics. The outpatient programs offered similar substance use services, including individual and group counseling, education, and support. Patients typically received a combination of individual and group treatment sessions. All treatment providers had relevant master’s degrees and were certified Alcohol and Substance Abuse Counselors (CASAC). Participants were recruited when they presented for outpatient alcohol treatment. They were then randomized to one of four conditions.
There were two assessment frequency and two assessment comprehensiveness designs that were combined in a 2 by 2 factorial design. The infrequent assessment conditions consisted of two in-person follow-up interviews after study enrollment (6- and 12-months post-baseline). The frequent follow-up conditions included four in-person follow-up interviews (3-, 6-, 9-, and 12-months post-baseline) and telephone interviews during the intervening months (1-, 2-, 4-, 5-, 7-, 8-, 10-, and 11 post-baseline). The telephone interviews only focused on alcohol and other substance use and were, on average, only 5 to 10 minutes long. The comprehensive conditions include a broad range of component parts, including 15 different scales and measures. The comprehensive assessment took, on average, 3.5 hours at baseline and 1.5 hours at each in-person follow-up to complete. The brief assessment conditions were much more limited and took, on average, 1.5 hours to administer at baseline and 20–30 minutes at follow-ups to complete.
The study then looked at the association between frequency (i.e., frequent vs. infrequent), comprehensiveness (i.e., comprehensive vs. brief), and frequency by comprehensiveness (frequent-brief, frequent-comprehensive, infrequent-brief, infrequent-comprehensive) and three specific outcomes. The first was the proportion of days that the participant reported any nonmedical (i.e., non-prescribed) drug use. The second was complete abstinence from alcohol and other drugs. The third was drug use-related negative consequences and was assessed by examining the frequency of 45 drug-related negative consequences that occurred in the last 3 months. The negative consequences were also broken down into five domains: physical, interpersonal, intrapersonal, impulse control, and social responsibility. For the analysis, the outcomes were assessed by categorizing the data into 3 periods – baseline, follow-up months 1–6, and follow-up months 7–12. The time periods aligned with the in-person interviews for all participants and allowed comparison across the four groups.
A total of 235 adults participated in the study. They were, on average, 40 years old and most were White (75%), male (63%), and single (77%). About half (53%) had a high school education or less, and 58% had an income of $19,000 or less. More than two-thirds (69%) had received prior substance use treatment. All the participants in this study presented for treatment with alcohol use disorder as their primary diagnosis.
WHAT DID THIS STUDY FIND?
Frequent assessments were linked with fewer drug use days.
Participants in the two frequent assessment conditions reported a similar improvement in drug use days compared to those in the infrequent conditions between baseline and 6-month follow-up. By the 12-month follow-up, however, those in the infrequent conditions reported an increase in the drug use days while those in the frequent conditions continued to improve. Of note, frequency of assessments only accounted for about 2% of the participants’ drug use days, which is a relatively small effect. Comprehensiveness of assessment was not found to be related to drug use days. The effect of frequency also did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on drug use days). In other words, the effect of assessment frequency on reducing drug use remained at about the same magnitude regardless of how many measures the patients were asked to complete at any given time.
Frequent assessments were associated with greater likelihood of complete abstinence and fewer negative consequences.
A greater proportion of participants in the frequent conditions reported abstinence from alcohol and other drugs during both the 1–6 (29% vs 16%) and 7–12-month (43% vs. 34%) follow-up periods compared to those in the infrequent conditions. Those in the frequent assessment conditions also had fewer total negative consequences and fewer consequences on each of the five subscales: physical-, interpersonal-, intrapersonal-, impulse-, and social responsibility-related negative consequences compared to those in the infrequent conditions.
Comprehensiveness was not found to be related to abstinence or negative consequences. Similar to the link between frequency and drug use days, the effect of frequency on abstinence and negative consequences did not depend on the comprehensiveness of the assessment (i.e., there was no interaction between frequency and comprehensiveness on odds of abstinence or negative consequences).
WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?
The researchers in this study found that the frequency but not the comprehensiveness of assessments was tied to drug use days, odds of abstinence, and drug use-related consequences.
Individuals that were in the frequent assessment conditions (frequent-brief and frequent-comprehensive) reported fewer days of drug use, a higher likelihood of complete abstinence, and fewer negative consequences compared to those in the infrequent conditions (infrequent-brief and infrequent-comprehensive). Thus, the researchers found that more frequent assessment itself can likely improve drug use outcomes beyond effects from treatment for adults with alcohol use disorder that received outpatient treatment.
This effect was especially apparent during the 6-to-12-month period after baseline, which suggests that the frequent assessment during this second follow-up period is critical to sustaining or enhancing treatment benefits. In other words, ongoing clinical monitoring and continuing care may improve a range of treatment outcomes compared to continuing care as usual, a result supported by meta-analytic research (a study of studies). One study even found that continuing care provided solely through telephone monitoring was associated with more abstinent days and cost less than treatment as usual. Other continuity of care treatment studies also have found that the “extensity” (i.e., continuity of care over time) is more important to improving clinical outcomes than the intensity of care (how much is delivered during a briefer period).
However, the mechanisms through which continuing care and measurement reactivity occurs remains underexplored. It could be the expectations of assessment increases motivation to align with socially desirable outcomes (e.g., reduced drug use). Alternatively, interacting with research staff may facilitate a therapeutic treatment booster through which individuals are able to reflect more deeply on the past impact of their drug use, potentially spurring motivation for change. Future research is needed to understand what causes measurement reactivity, how to account for it in treatment studies, and how to leverage those mechanisms for enhanced interventions.
The participants in this study were primarily White, male, and single, which limits generalizability of the findings to populations not exhibiting these kinds of characteristics.
Only individuals that expressed interest in a research study about follow-ups at treatment presentations were included. Those that declined interest may be different (e.g., higher addiction severity level) from those that participated.
Although all individuals were recruited from two outpatient treatment clinics, no measures of treatment participation or completion were included. It is unclear whether the frequency of assessment follow-up may have facilitated more treatment engagement, which could have resulted in improved outcomes.
The study did not have a true control group (i.e., a group that received little or no treatment). Thus, it is unknown if some aspect of treatment interacted with the frequency and comprehensiveness of assessments and impacted outcomes.
BOTTOM LINE
Frequent assessment of treatment outcomes, compared to infrequent assessment, following alcohol use disorder outpatient treatment was associated with decreased drug use days, increased likelihood of abstinence, and fewer drug use-related negative consequences. Although improved outcomes are generally desirable, the reactivity to measurement that was observed suggests that the frequency of treatment assessment clouds researchers understanding of standalone treatment effects. Nevertheless, these results highlight the benefits of continuing care and ongoing clinical monitoring. On-going check-ins, especially 6-to-12 months following treatment entry, in real-world clinical settings may be critical to maximize the benefits conferred from treatment. Future research is needed to understand what aspects related to the frequency of assessment may impact treatment and how to account for measurement reactivity in evaluation of treatment.
For individuals and families seeking recovery: This study found that frequent assessments within a treatment research project were linked with improved outcomes compared to infrequent assessments. Individuals or families seeking treatment services should look for programs that offer empirically supported continuing care to leverage the benefits of frequent, on-going monitoring, interpersonal contact, and outcomes measurement.
For treatment professionals and treatment systems: Understanding the relationship between treatment components (e.g., treatment modality such as motivational interviewing) and treatment outcomes (e.g., drug use and related consequences) is vital to develop and test treatment interventions. However, this study found significant measurement reactivity–frequent assessment of outcomes was linked with changes in outcomes compared to infrequent assessment. To isolate the effect of treatment, researchers and evaluators might consider reducing the frequency of follow-ups. However, when developing a continuing care plan in practice, it may be beneficial to incorporate frequent follow-ups. Treatment programs and interventions should develop continuing care components to leverage the benefits of frequent, on-going measurement.
For scientists: This study randomized participants in a 2 by 2 factorial design crossing frequent and infrequent and comprehensive and brief assessments. They found that frequent compared to infrequent, but not comprehensiveness or the frequency-comprehensiveness interaction, was related to treatment outcomes. The findings of this study demonstrate the need to account for and control assessment reactivity when designing studies and interpreting their results. Frequent assessment may obscure detection of treatment effects. Future studies can seek to isolate the dose-response of frequency through larger trials, especially those incorporating a true control condition with no or minimal assessment exposure. Further work is also needed to understand the mechanisms that underly benefits conferred from research participation independently from the treatment being studied.
For policy makers: This study found that frequent assessment was linked with improved drug use outcomes compared to infrequent assessment. Policies and interventions that allow for increased methodological rigor (i.e., increased funding) are likely to facilitate better understanding of measurement reactivity and independent treatment effects. An improved understanding of standalone treatment effects is likely to improve research efficiency and validity that can translate to improved lives and reduced societal costs. Furthermore, additional funding is needed to support continuing care after initial treatment episodes to maximize impact and get real-world outcomes as close as possible to rigorous clinical research students that often include long-term, regular assessments.
