About This Code

This code is used to investigate whether (and, if so, how) a baseline or time-varying covariate can be a useful tailoring variables in an adaptive intervention.

How can a behavioral scientist use this code?

Behavioral intervention scientists can use this code to generate hypotheses about more deeply-tailored adaptive interventions using data from a SMART.

What method does this code implement?

This code implements an easy-to-use version of Q-learning (a common reinforcement learning algorithm) and a method for estimating confidence intervals. The method can be viewed as an extension of moderated regression analysis for data arising from a SMART. In a two-stage SMART, Q-learning implements two regressions, starting at the second stage: The first regression selects the best second-stage treatment. The second regression selects the best first-stage treatment given that the best second stage treatment has been assigned to each participant. The backward ordering of the regressions avoids selecting treatment options that appear optimal in the short term, but may lead to a less desirable outcome in the long run.