(Responding on behalf of Jere) Thanks for your questions, Mark. To start, yes, Math-Mapper is a piece of software built by our small team and our work involves both building the tool and conducting research with it at the same time! We are currently piloting it in a few partner schools, and plan to scale up its use in the coming school year. Folks interested in using Math-Mapper in their schools can email us at sudds_group@ncsu.edu to get more information. So, yes, it is available for use. The sudds.co website provides access to all the learning trajectories (LTs) and links them to the CCSS-M standards. Only one assessment on ratio is publicly available from sudds.co as a demo. You do have to sign up to have access to the full set of assessments and reporting features.  

A second question you asked was about the nature of the formative feedback and we are glad you asked. We expect that our diagnostic assessments are used for formative purposes. We want to affect the learning process as it is occurring and help students and teachers recognize what has been learned and what remains to be learned. Our students find the items challenging and conceptual and some teachers call them “stretch items”. This is because we want to offer both teachers and learners a rigorous elaboration of a mathematical concept. Our aim is also to support learner-centered instruction and the exchange and discussion of students’ ways of thinking. We have found in our work, that this does not happen by simply providing teachers with the data on LT-based assessments (heatmaps). It takes teachers time to see how to use a review of items aligned to an LT to understand the LT, and learn to focus on student thinking simultaneously. Our students have an opportunity to rework and resubmit responses to assessments and they love that! In addition, we offer retests and practice. We want to communicate to students that learning is a process and they don’t have to get everything the first time. They can learn it!

We are intrigued by the idea of another research team expanding on the concept of Math-Mapper, particularly its foundation of learning trajectories, for use in science classrooms. We would love to hear from science ed researchers and teachers to find out their thoughts on this. We have the mechanisms for making the maps, linking to standards, creating scope and sequences, assigning appropriate assessments, and creating assessments for LTs, so we would be interested in such a partnership.

(Responding on behalf of Jere) Great questions, Kevin. We’ve answered each of your questions below.

Measurable impacts: We have studied this in two ways, but we want to remind readers that the first year of our project was used to build and test items and the second was to provide reports and study their use. This work takes time to see and evaluate the impact. In the first two years, we conducted several design studies at our lower performing school using a shared curriculum.  We collected data on pre- and post-tests and documented 15 percentage point learning gains in both of our ratio and percents design studies. From the first year to the second, our sixth graders improved in performance by 16 percentage points. We see significant differences among teachers. We see no significant differences in learning gains among subgroups of students (White, African-Americans, Hispanic/Latinos).  

We work in both higher- and lower- performing schools. Our higher performing schools report strong correlations with their students’ performance on the PARCC exam, and teachers want to continue using it. At our lower-performing school, it has taken two years to see substantial gains in the sixth graders on end-of-year tests with two of three teachers formally recognized within the district for moving up their students in the top 25% of growth for the state. Progress in seventh and eighth grades has been slower, perhaps due to the time lag between teaching students without Math Mapper in prior grades. We will know more after this year.

Professional development: With our research partner schools, we have offered initial introductory sessions and in-person professional development around using the tool. Our user guide, is accessible to anyone with an account, and occasional blog posts that synthesize our research are meant to give other users a basis for making use of the tool. One effective form of PD is to use PLCs to have teachers report and share data. We found that at first some teachers were reluctant to share data, but now they simply use it as an opportunity to a) refine their own approaches during the unit, and b) to plan collaboratively for changes for the next year.  

Interventions for students based on data: This is a hard and important question. We are going to be candid; at this point, we are concentrating on getting the measures to produce valid data on student progress across the class. Because this is feedback into a learning process and not a judgment of performance accomplishment, the intervention heavily emphasizes the idea that students keep learning but at the same time have clear ideas about what they need to be able to do. This is a surprisingly strong form of intervention. We also provide access to practice at the level of a single construct and students can choose what level to work at. In some classes, they work in pairs and in practice, the feedback is given at the item level. If by next step, you mean the assignment of instructional material tailored to their performance, we don’t have that yet and rely on the teacher. Our learning map has a curated set of open educational resources and more can be added, but we have not tried to automate that connection.

Hi Megan,

Thanks for your thoughtful questions.  Our students collaborate in a number of ways.  Some teachers encourage students to work together once they get their reports back. The students will open up items and discuss them.  They can also revise and resubmit, a feature they love.  Other teachers will have students work together on practice, which gives the same bank of assessments at the construct rather than cluster level.  But in this case, the students get feedback immediately after doing an item and then they can choose the next level to work at.  In both of these cases, there is a lot of peer to peer teaching going on, but as you suggested, other forms of social networking would be a good thing to add.  
As to the development of the tool, teachers were involved in much of the work.  I built the map first as a means to hierarchically organize the content and chunk it to help with conceptualizing the whole of it.  Then came the LTs which were done by me and my team, all of whom are experienced teachers and thoroughly familiar with different parts of the literature.  We worked with local teachers to write initial items and to put interesting content links on the map and then using some of those items, we learned to write conceptual items and did so from elaboration documents describing each level and the cases.  We implemented early in our partner schools and they gave us feedback as we built the reports.  They were the ones that wanted an item analysis and a means to display the percent of students with misconceptions.  As the team worked on validating the assessments, we studied how teachers discussed the data with students and then built our professional development materials based on what we saw.  We believe that involving teachers, pyschometricians, learning scientists, and engineers as an interdisciplinary team is essential to get a product that is useful in classrooms.   Thanks for asking.

Hi Beth.

Yes, there is professional development available but it focuses more on the use of the tool.  We have been studying the different ways teachers discuss the data with their classes and developing a framework to capture the major categories of approaches.   We have also been observing teachers working in PLC to share their strategies with others.  We are at the beginning of this work and would be glad to share more of it over time.  Most importantly, there is a need to be explicit about how to think about the relationship between student-centered instruction and the data on the LTs.   Thanks for asking.