How Well Do Web-Based Interventions Work & For Whom?

Home
Research
How Well Do Web-Based Interventions Work & For Whom?

An examination of the web-app: “Overcoming Addictions”

Stay on the frontiers of

recovery science

with the free, monthly

Recovery Bulletin

WHAT PROBLEM DOES THIS STUDY ADDRESS?

Web-based interventions to assist individuals to overcome problematic alcohol and substance use are increasingly being developed. In theory, they are optimally suited to deliver evidence-based behavioral health interventions on a large scale, as they can deliver accurate and valid assessment protocols, produce reliable computations, use complex algorithms to tailor recommendations based on specific characteristics, and thus can provide impartial results, specific to individuals.

In practice, empirical tests of such interventions have so far yielded small- to medium-sized effects, suggesting that the effectiveness of these intervention tools can still be improved. (That said, even a small positive effect can have a tremendous public health impact, if the intervention has the ability to reach a lot of people, as can be true for web-based interventions.) Numerous questions remain open as the efforts to improve these interventions continue.

In this article, Campbell et al. examined the web-based intervention “Overcoming Addictions (OA),” which can be used as either a stand-alone intervention tool, or as an additional resource when participating in the meetings and other activities organized by SMART Recovery, a secular, evidence-informed mutual-help group. In addition to addressing the question: “Does it work?” this article also provides insight into “Who does this intervention work for?”

HOW WAS THIS STUDY CONDUCTED?

Participants were adults interested in seeking support to abstain from drinking, as recruited via the SMART Recovery home page. Eligibility requirements stipulated that participants had, at a minimum, consumed 5+/4+ drinks (male/female) on a single occasion at least once within the prior 3 months, and scored 8 or higher on the Alcohol Use Disorders Identification Test (AUDIT), suggesting at least a moderate level alcohol problem.

Enrolled participants reported significant depressive, anxious and/or somatic distress at screening, and scored at the more severe end of the use disorder spectrum (average AUDIT score of 25). Participants were screened by phone, and if eligible, were then randomized, stratified by race.

Originally, the study was designed to randomize study participants into three groups:

only using the web-based intervention OA vs.
using OA and attending SMART Recovery meetings (online and face-to-face), vs.
only attending SMART Recovery meetings.

It turned out, however, that potential participants were unwilling to be randomized to a condition that would not allow them to participate in SMART Recovery meetings. Thus, in this paper, two groups were compared: those who used OA in addition to engaging in SMART Recovery activities (n=102) vs. those who only engaged in SMART Recovery activities (n=86). Participants were followed up 3 and 6 months after they completed their baseline assessment, with an 80% and 70% retention rate, respectively.

WHAT DID THIS STUDY FIND?

Results showed that both groups improved over the 6 months following their study enrollment, where participants on average increased their percent of days staying abstinent, reduced their number of drinks per drinking day, and reported experiencing fewer negative consequences of their alcohol and/or drug use.

These positive effects were very large for participants who had stopped drinking for at least one day prior to study enrollment, and small to large for participants who had not done so. In comparing the randomized groups, results did not favor the web-app group: outcomes were similar across the two compared groups.

In looking at the original 3 groups, there was a significant time by group interaction effect that showed that improvements were made in percent days abstinent across the three groups.

But after making comparable gains in percent days abstinent initially, participants using the “Overcoming Addictions” web-app without the other SMART recovery activities (19 participants) had lower percent days abstinent between the 3- and 6-month follow-up (i.e. participants in the “Overcoming Addiction’s” web-app-only group consumed alcohol on more days).

WHY IS THIS STUDY IMPORTANT

This study is important because it provides empirical evidence about the effectiveness of a web-based intervention to overcome problematic alcohol & substance use.

While a plethora of smartphone apps exist to help individuals managing their drinking, and numerous web-based programs have been developed, the literature on such programs remains limited.

The intervention in question, “Overcoming Addictions”, is readily available, thereby making an empirical evaluation of its effectiveness highly relevant for anyone looking for a specific intervention to try out.

The investigation of factors that may influence the effectiveness of web-based interventions is another very important contribution of this paper, as many questions currently still exist about who is most likely to benefit from such interventions, and how this type of interventions can be best designed to reach and support the largest segment of the population they seek to serve.

As with any study, some limitations existed in this study, which should be kept in mind. The most important design issue to keep in mind is that there was no non-intervention control group: all participants in this study engaged with intervention materials and activities. Thus, effectively, this study quantifies to what degree engaging in these activities helped people who specifically sought them out, but without a no-intervention control group of people being randomized to wanting this type of support but not receiving it, it is not clear how much of this positive effect is simply due to participants already being on the path to positive outcomes versus participants actually and effectively being supported on that path by the intervention.
Additionally, participants were not representative of the population struggling with alcohol and substance use, but rather over-represented women, Whites, and more highly educated individuals. These characteristics are in line with the characteristics of people who seek out support through SMART Recovery, whose home page was used to recruit participants for this study, and thus, would be representative of the people who would seek out this web-based intervention. Nevertheless, in a sample that is fairly highly educated, the finding that internet fluency does not affect the effectiveness of the intervention needs to be taken with a grain of salt, as it is quite possible that internet fluency may make more of a difference in less educated populations.
Finally, the findings may be influenced by recall biases. That is, participants were asked about their baseline drinking during the same phone call as they were asked about their 3-month drinking (though in distinctly different interactions). As recall becomes more inaccurate as the length of time in between now and then increases, that description of baseline drinking may be more prone to measurement error than the 3-month description. Similarly, participants were asked about their experience navigating the web-based intervention three months after they first engaged with it, thereby again introducing potential recall errors, but would equally apply across groups, and consequently would not impact conclusions about the performance of the groups relative to each other.

NEXT STEPS

The next step for this team, as described in this article, is to continue to use user input to improve the web-based intervention “Overcoming Addictions” – indeed, some further developments have already been implemented. More generally, the next steps will be to explore innovative and novels ways to increase individuals’ engagement in this web-based intervention and others like it.

BOTTOM LINE

For individuals & families seeking recovery: “Overcoming Addictions” is a science-based web-based interventions that addiction scientists continue to update and improve to facilitate greater reach and effectiveness. For those who are intrigued by it, and willing to try it out, this paper suggests that participation in “Overcoming Addictions” could be helpful in reducing drinking and negative consequences of alcohol and/or substance use.
For scientists: This paper describes an important within-person test into the effectiveness of a web-based intervention, but a true randomized test with a non-intervention control group is still needed to determine the effectiveness of engaging in this intervention as opposed to not participating in an intervention activity. Such a test may be impossible to conduct, however, as individuals ready to change their behavior may not be willing to be randomized to no intervention control conditions; there are also ethical issues to consider in delaying treatment to those ready to engage in it.
For policy makers: Internet-based and technology-facilitated interventions continue to be introduced to the public, but research on their effectiveness continues to be sparse. Innovative mechanisms for increasing the reach of these programs and to support the necessary scientific process to develop them are sorely needed.
For treatment professionals and treatment systems: “Overcoming Addictions” is a well-tested, continuously updated web-based support option that has promising (though not causality conclusive) effects, and thus may be an intriguing option to present to patients interested in this type of support.

CITATIONS

Campbell W, Hester RK, Lenberg KL, Delaney HD. Overcoming Addictions, a Web-Based Application, and SMART Recovery, an Online and In-Person Mutual Help Group for Problem Drinkers, Part 2: Six-Month Outcomes of a Randomized Controlled Trial and Qualitative Feedback From Participants. Journal of medical Internet research. 2016;18(10).

Stay on the Frontiers of

recovery science

with the free, monthly

Recovery Bulletin

WHAT PROBLEM DOES THIS STUDY ADDRESS?

Web-based interventions to assist individuals to overcome problematic alcohol and substance use are increasingly being developed. In theory, they are optimally suited to deliver evidence-based behavioral health interventions on a large scale, as they can deliver accurate and valid assessment protocols, produce reliable computations, use complex algorithms to tailor recommendations based on specific characteristics, and thus can provide impartial results, specific to individuals.

HOW WAS THIS STUDY CONDUCTED?

Originally, the study was designed to randomize study participants into three groups:

only using the web-based intervention OA vs.
using OA and attending SMART Recovery meetings (online and face-to-face), vs.
only attending SMART Recovery meetings.

WHAT DID THIS STUDY FIND?

Results showed that both groups improved over the 6 months following their study enrollment, where participants on average increased their percent of days staying abstinent, reduced their number of drinks per drinking day, and reported experiencing fewer negative consequences of their alcohol and/or drug use.

In looking at the original 3 groups, there was a significant time by group interaction effect that showed that improvements were made in percent days abstinent across the three groups.

WHY IS THIS STUDY IMPORTANT

This study is important because it provides empirical evidence about the effectiveness of a web-based intervention to overcome problematic alcohol & substance use.

While a plethora of smartphone apps exist to help individuals managing their drinking, and numerous web-based programs have been developed, the literature on such programs remains limited.

The investigation of factors that may influence the effectiveness of web-based interventions is another very important contribution of this paper, as many questions currently still exist about who is most likely to benefit from such interventions, and how this type of interventions can be best designed to reach and support the largest segment of the population they seek to serve.

As with any study, some limitations existed in this study, which should be kept in mind. The most important design issue to keep in mind is that there was no non-intervention control group: all participants in this study engaged with intervention materials and activities. Thus, effectively, this study quantifies to what degree engaging in these activities helped people who specifically sought them out, but without a no-intervention control group of people being randomized to wanting this type of support but not receiving it, it is not clear how much of this positive effect is simply due to participants already being on the path to positive outcomes versus participants actually and effectively being supported on that path by the intervention.
Additionally, participants were not representative of the population struggling with alcohol and substance use, but rather over-represented women, Whites, and more highly educated individuals. These characteristics are in line with the characteristics of people who seek out support through SMART Recovery, whose home page was used to recruit participants for this study, and thus, would be representative of the people who would seek out this web-based intervention. Nevertheless, in a sample that is fairly highly educated, the finding that internet fluency does not affect the effectiveness of the intervention needs to be taken with a grain of salt, as it is quite possible that internet fluency may make more of a difference in less educated populations.
Finally, the findings may be influenced by recall biases. That is, participants were asked about their baseline drinking during the same phone call as they were asked about their 3-month drinking (though in distinctly different interactions). As recall becomes more inaccurate as the length of time in between now and then increases, that description of baseline drinking may be more prone to measurement error than the 3-month description. Similarly, participants were asked about their experience navigating the web-based intervention three months after they first engaged with it, thereby again introducing potential recall errors, but would equally apply across groups, and consequently would not impact conclusions about the performance of the groups relative to each other.

NEXT STEPS

BOTTOM LINE

For individuals & families seeking recovery: “Overcoming Addictions” is a science-based web-based interventions that addiction scientists continue to update and improve to facilitate greater reach and effectiveness. For those who are intrigued by it, and willing to try it out, this paper suggests that participation in “Overcoming Addictions” could be helpful in reducing drinking and negative consequences of alcohol and/or substance use.
For scientists: This paper describes an important within-person test into the effectiveness of a web-based intervention, but a true randomized test with a non-intervention control group is still needed to determine the effectiveness of engaging in this intervention as opposed to not participating in an intervention activity. Such a test may be impossible to conduct, however, as individuals ready to change their behavior may not be willing to be randomized to no intervention control conditions; there are also ethical issues to consider in delaying treatment to those ready to engage in it.
For policy makers: Internet-based and technology-facilitated interventions continue to be introduced to the public, but research on their effectiveness continues to be sparse. Innovative mechanisms for increasing the reach of these programs and to support the necessary scientific process to develop them are sorely needed.
For treatment professionals and treatment systems: “Overcoming Addictions” is a well-tested, continuously updated web-based support option that has promising (though not causality conclusive) effects, and thus may be an intriguing option to present to patients interested in this type of support.

CITATIONS

WHAT PROBLEM DOES THIS STUDY ADDRESS?

Web-based interventions to assist individuals to overcome problematic alcohol and substance use are increasingly being developed. In theory, they are optimally suited to deliver evidence-based behavioral health interventions on a large scale, as they can deliver accurate and valid assessment protocols, produce reliable computations, use complex algorithms to tailor recommendations based on specific characteristics, and thus can provide impartial results, specific to individuals.

HOW WAS THIS STUDY CONDUCTED?

Originally, the study was designed to randomize study participants into three groups:

only using the web-based intervention OA vs.
using OA and attending SMART Recovery meetings (online and face-to-face), vs.
only attending SMART Recovery meetings.

WHAT DID THIS STUDY FIND?

Results showed that both groups improved over the 6 months following their study enrollment, where participants on average increased their percent of days staying abstinent, reduced their number of drinks per drinking day, and reported experiencing fewer negative consequences of their alcohol and/or drug use.

In looking at the original 3 groups, there was a significant time by group interaction effect that showed that improvements were made in percent days abstinent across the three groups.

WHY IS THIS STUDY IMPORTANT

This study is important because it provides empirical evidence about the effectiveness of a web-based intervention to overcome problematic alcohol & substance use.

While a plethora of smartphone apps exist to help individuals managing their drinking, and numerous web-based programs have been developed, the literature on such programs remains limited.

The investigation of factors that may influence the effectiveness of web-based interventions is another very important contribution of this paper, as many questions currently still exist about who is most likely to benefit from such interventions, and how this type of interventions can be best designed to reach and support the largest segment of the population they seek to serve.

As with any study, some limitations existed in this study, which should be kept in mind. The most important design issue to keep in mind is that there was no non-intervention control group: all participants in this study engaged with intervention materials and activities. Thus, effectively, this study quantifies to what degree engaging in these activities helped people who specifically sought them out, but without a no-intervention control group of people being randomized to wanting this type of support but not receiving it, it is not clear how much of this positive effect is simply due to participants already being on the path to positive outcomes versus participants actually and effectively being supported on that path by the intervention.
Additionally, participants were not representative of the population struggling with alcohol and substance use, but rather over-represented women, Whites, and more highly educated individuals. These characteristics are in line with the characteristics of people who seek out support through SMART Recovery, whose home page was used to recruit participants for this study, and thus, would be representative of the people who would seek out this web-based intervention. Nevertheless, in a sample that is fairly highly educated, the finding that internet fluency does not affect the effectiveness of the intervention needs to be taken with a grain of salt, as it is quite possible that internet fluency may make more of a difference in less educated populations.
Finally, the findings may be influenced by recall biases. That is, participants were asked about their baseline drinking during the same phone call as they were asked about their 3-month drinking (though in distinctly different interactions). As recall becomes more inaccurate as the length of time in between now and then increases, that description of baseline drinking may be more prone to measurement error than the 3-month description. Similarly, participants were asked about their experience navigating the web-based intervention three months after they first engaged with it, thereby again introducing potential recall errors, but would equally apply across groups, and consequently would not impact conclusions about the performance of the groups relative to each other.

NEXT STEPS

BOTTOM LINE

For individuals & families seeking recovery: “Overcoming Addictions” is a science-based web-based interventions that addiction scientists continue to update and improve to facilitate greater reach and effectiveness. For those who are intrigued by it, and willing to try it out, this paper suggests that participation in “Overcoming Addictions” could be helpful in reducing drinking and negative consequences of alcohol and/or substance use.
For scientists: This paper describes an important within-person test into the effectiveness of a web-based intervention, but a true randomized test with a non-intervention control group is still needed to determine the effectiveness of engaging in this intervention as opposed to not participating in an intervention activity. Such a test may be impossible to conduct, however, as individuals ready to change their behavior may not be willing to be randomized to no intervention control conditions; there are also ethical issues to consider in delaying treatment to those ready to engage in it.
For policy makers: Internet-based and technology-facilitated interventions continue to be introduced to the public, but research on their effectiveness continues to be sparse. Innovative mechanisms for increasing the reach of these programs and to support the necessary scientific process to develop them are sorely needed.
For treatment professionals and treatment systems: “Overcoming Addictions” is a well-tested, continuously updated web-based support option that has promising (though not causality conclusive) effects, and thus may be an intriguing option to present to patients interested in this type of support.