The current article explores some common methodological issues facing educational technology research. This paper also highlight new data collection approaches using examples from the literature and the authors' own experience Next, the authors highlight other challenges and opportunities inherent in the study of educational technology. In addition, the authors discuss the critical importance of aligning outcome measures with the technological innovation. In conclusion, the authors hope that the issues this article raises and the specific examples it includes spur critical reflection on some of the details important to data collection and educational technology research.