We consider the problem of counterfactual inference in sequentially designed experiments wherein a collection of N units each undergoes a sequence of interventions for T time periods, based on policies that sequentially adapt over time. We introduce a suitable latent factor model where the potential outcomes are determined by exogenous unit and time level latent factors. Under suitable conditions, we show that it is possible to estimate the missing (potential) outcomes using a simple variant of nearest neighbors.First, assuming a bilinear latent factor model and allowing for an arbitrary adaptive sampling policy, we establish a distribution-free non-asymptotic guarantee for estimating the missing outcome of any unit at any time; under suitable regularity conditions, this guarantee implies that our estimator is consistent. Second, for a generic non-parametric latent factor model, we establish that the estimate for the missing outcome of any unit at time T satisfies a central limit theorem as T goes to infinity, under suitable regularity conditions. Finally, en route to establishing this central limit theorem, we prove a non-asymptotic mean-squared-error bound for the estimate of the missing outcome of any unit at time T. Our work extends the recently growing literature on inference with adaptively collected data by allowing for policies that pool across units and also compliment the matrix completion literature when the entries are revealed sequentially in an arbitrarily dependent manner based on prior observed data.