In this paper, the authors describe a new, free resource for continuous video assessment named YouDemo. This tool enables real time rating of uploaded YouTube videos for use in science, technology, engineering, and mathematics (STEM) education and beyond. The authors discuss the discrepancies between preservice science teachers’ assessments of self- and peer-created videos using the tool. The findings reveal that preservice teachers, who used the YouDemo, engaged in reflection and discussion on a deeper level than traditional means of pedagogical skill building in the classroom. Furthermore, preservice teachers perceived continuous video rating beneficial in enabling video assessment, promoting critical thinking, and increasing engagement the authors found that based on the discrepancies we found in peer and self-evaluations.