ISTE Standards for Coaches 6a, “Assist educators and leaders in securely collecting and analyzing student data,” establishes the goal of keeping data collection in mind because instruction cannot be based on data-driven decisions if data is not collected in an effective manner.
Source: EDTC 6106 Program Evaluation Project, Seattle Pacific University
Evidence: “[Nonprofit education organization] created and implemented online professional learning experiences for K-12 teachers across the United States in 2020. These were done due to the global pandemic and resulting lock-down conditions. Participants were surveyed.”
Evidence: “Analysis of this data provided valuable insights for the professional learning design team. The Digital Experience Math Community of Practice was one of the most popular among the roughly 30,000 educators participating in these trainings and garnered 450+ open-ended question responses for evaluation.”
Evidence: “One of [the Nonprofit education organization’s] most popular first-time online offerings, the [Nonprofit education organization] Digital Experience Math Community of Practice received survey responses from over 450 educators. This provided tremendous insight for the design team with respect to redesigning this course for the year two experience offered in summer of 2021. This made the math training experience a great place to zoom in with a close analysis of the survey feedback.”
Explanation: Analyzing the 450+ open-ended responses required a significant amount of time and effort. The first step after obtaining the data in a basic spreadsheet form was labeling the columns of data. This allowed for a general identification of themes for each of the 450+ entries. The general themes were then categorized into the six main types of response topics: Breakout Rooms, Content & Pacing, Face-to-Face versus Virtual, Not Applicable, Positive, and Technology Related. Given that every response had a corresponding net promoter score, these scores could be disaggregated by topic and then analyzed in response to the overall average net promoter score. This entire process required a minimum of 40-50 hours of work. Through this project, I learned a tremendous amount about working with qualitative data, labeling data, and analyzing data.
Given how involved data analysis can be and the required learning curve, supporting educators with data collection is essential because it’s not something that most educators are familiar with by trade. They are instructors. While grading is a form of data collection, it’s often not conducted in a secure, systematic, and research-based approach. This support becomes all the more important with regard to large-scale levels of instruction such as district or state-based initiatives and national professional learning implementation. Effective support at the outset before any trainings or instruction have been conducted, means that data can be successfully gathered. Waiting until after-the-fact risks either being unable to gather data or collecting data that may not be sound from a research-based perspective.