1. Jere Confrey
  2. Joseph D. Moore University Professor of Mathematics Education
  3. Building a Next Generation Diagnostic Assessment and Reporting System within a Learning Trajectory-Based Mathematics Learning Map for Grades 6-8
  4. https://sudds.ced.ncsu.edu/
  5. North Carolina State University
  1. Michael Belcher
  2. Graduate Student
  3. Building a Next Generation Diagnostic Assessment and Reporting System within a Learning Trajectory-Based Mathematics Learning Map for Grades 6-8
  4. https://sudds.ced.ncsu.edu/
  5. North Carolina State University
  1. Meetal Shah
  2. Graduate Student
  3. Building a Next Generation Diagnostic Assessment and Reporting System within a Learning Trajectory-Based Mathematics Learning Map for Grades 6-8
  4. https://sudds.ced.ncsu.edu/
  5. North Carolina State University
Public Discussion
  • Icon for: Michael Belcher

    Michael Belcher

    Co-Presenter
    May 13, 2018 | 04:55 p.m.

    Thanks for watching our video! We invite you to ask questions and give feedback below. In particular, we are interested in your thoughts on the following question: How can digitally administered and scored measures of students' progress along learning trajectories improve classroom discourse? 

  • Icon for: Mark Windschitl

    Mark Windschitl

    Facilitator
    May 14, 2018 | 12:19 p.m.

    Michael, Jere, I liked the questions you posed at the outset of the video, some of these get to the heart of "what are we doing together in this project and why"? I was also curious about the formative feedback you mentioned that helps both teachers and students understand what the current state of understanding is, and what might happen next in instruction. I really want to learn more about Math Mapper. Is that your software and if so is it aviaible? I was curious about whether it's use as a kind of ongoing curriculum, learning, and assessment mapper could apply to science as well. 

  • Icon for: Meetal Shah

    Meetal Shah

    Co-Presenter
    May 15, 2018 | 12:43 a.m.

    (Responding on behalf of Jere) Thanks for your questions, Mark. To start, yes, Math-Mapper is a piece of software built by our small team and our work involves both building the tool and conducting research with it at the same time! We are currently piloting it in a few partner schools, and plan to scale up its use in the coming school year. Folks interested in using Math-Mapper in their schools can email us at sudds_group@ncsu.edu to get more information. So, yes, it is available for use. The sudds.co website provides access to all the learning trajectories (LTs) and links them to the CCSS-M standards. Only one assessment on ratio is publicly available from sudds.co as a demo. You do have to sign up to have access to the full set of assessments and reporting features.  

    A second question you asked was about the nature of the formative feedback and we are glad you asked. We expect that our diagnostic assessments are used for formative purposes. We want to affect the learning process as it is occurring and help students and teachers recognize what has been learned and what remains to be learned. Our students find the items challenging and conceptual and some teachers call them “stretch items”. This is because we want to offer both teachers and learners a rigorous elaboration of a mathematical concept. Our aim is also to support learner-centered instruction and the exchange and discussion of students’ ways of thinking. We have found in our work, that this does not happen by simply providing teachers with the data on LT-based assessments (heatmaps). It takes teachers time to see how to use a review of items aligned to an LT to understand the LT, and learn to focus on student thinking simultaneously. Our students have an opportunity to rework and resubmit responses to assessments and they love that! In addition, we offer retests and practice. We want to communicate to students that learning is a process and they don’t have to get everything the first time. They can learn it!

    We are intrigued by the idea of another research team expanding on the concept of Math-Mapper, particularly its foundation of learning trajectories, for use in science classrooms. We would love to hear from science ed researchers and teachers to find out their thoughts on this. We have the mechanisms for making the maps, linking to standards, creating scope and sequences, assigning appropriate assessments, and creating assessments for LTs, so we would be interested in such a partnership.

  • Icon for: Kevin Brown

    Kevin Brown

    Facilitator
    May 14, 2018 | 02:25 p.m.

    This looks like a really powerful, comprehensive tool with a strong grounding in the theory of how students learn mathematics! I would expect to see considerable payoff in terms of student learning so am wondering about the kinds of impacts you’ve been able to measure and if you’ve been able to compare those learning gains to a business-as-usual control group? I’d also like to hear more about the kinds of professional supports that might be required to help teachers adopt Math-Mapper into their everyday classroom practices? Finally, I’m wondering if you can say more about the kinds of interventions that are available to help students who might be struggling at a given stage of the learning progression and how ongoing student assessment data is used to select the appropriate intervention?

  • Icon for: Meetal Shah

    Meetal Shah

    Co-Presenter
    May 15, 2018 | 01:01 a.m.

    (Responding on behalf of Jere) Great questions, Kevin. We’ve answered each of your questions below.

    Measurable impacts: We have studied this in two ways, but we want to remind readers that the first year of our project was used to build and test items and the second was to provide reports and study their use. This work takes time to see and evaluate the impact. In the first two years, we conducted several design studies at our lower performing school using a shared curriculum.  We collected data on pre- and post-tests and documented 15 percentage point learning gains in both of our ratio and percents design studies. From the first year to the second, our sixth graders improved in performance by 16 percentage points. We see significant differences among teachers. We see no significant differences in learning gains among subgroups of students (White, African-Americans, Hispanic/Latinos).  

    We work in both higher- and lower- performing schools. Our higher performing schools report strong correlations with their students’ performance on the PARCC exam, and teachers want to continue using it. At our lower-performing school, it has taken two years to see substantial gains in the sixth graders on end-of-year tests with two of three teachers formally recognized within the district for moving up their students in the top 25% of growth for the state. Progress in seventh and eighth grades has been slower, perhaps due to the time lag between teaching students without Math Mapper in prior grades. We will know more after this year.

    Professional development: With our research partner schools, we have offered initial introductory sessions and in-person professional development around using the tool. Our user guide, is accessible to anyone with an account, and occasional blog posts that synthesize our research are meant to give other users a basis for making use of the tool. One effective form of PD is to use PLCs to have teachers report and share data. We found that at first some teachers were reluctant to share data, but now they simply use it as an opportunity to a) refine their own approaches during the unit, and b) to plan collaboratively for changes for the next year.  

    Interventions for students based on data: This is a hard and important question. We are going to be candid; at this point, we are concentrating on getting the measures to produce valid data on student progress across the class. Because this is feedback into a learning process and not a judgment of performance accomplishment, the intervention heavily emphasizes the idea that students keep learning but at the same time have clear ideas about what they need to be able to do. This is a surprisingly strong form of intervention. We also provide access to practice at the level of a single construct and students can choose what level to work at. In some classes, they work in pairs and in practice, the feedback is given at the item level. If by next step, you mean the assignment of instructional material tailored to their performance, we don’t have that yet and rely on the teacher. Our learning map has a curated set of open educational resources and more can be added, but we have not tried to automate that connection.

  • Icon for: Megan McKinley

    Megan McKinley

    Facilitator
    May 14, 2018 | 05:05 p.m.

    Hello, Michael and all. This program sounds really interesting. Could you elaborate on the collaborative capabilities of Math-Mapper for both students and teachers?

     

    I like the question you posted around how Math-Mapper can support classroom discourse. What is your thinking around this?  Here are some of my thoughts - Is there a blog post function? I wonder if students could post questions that could guide discussion in class. For example, students could post questions around the “big” ideas and how these ideas fit into the real world. I also immediately thought of using Math-Mapper as a way to get students to teach their peers about content that they already mastered, so that they may explain their thinking and the challenges they ran into when solving a particular problem.

     

    I’m also interested in hearing more about the development of this program and if (and how) various stakeholders are involved in this.

  • Icon for: Jere Confrey

    Jere Confrey

    Lead Presenter
    May 15, 2018 | 08:23 a.m.

    Hi Megan,

    Thanks for your thoughtful questions.  Our students collaborate in a number of ways.  Some teachers encourage students to work together once they get their reports back. The students will open up items and discuss them.  They can also revise and resubmit, a feature they love.  Other teachers will have students work together on practice, which gives the same bank of assessments at the construct rather than cluster level.  But in this case, the students get feedback immediately after doing an item and then they can choose the next level to work at.  In both of these cases, there is a lot of peer to peer teaching going on, but as you suggested, other forms of social networking would be a good thing to add.  
    As to the development of the tool, teachers were involved in much of the work.  I built the map first as a means to hierarchically organize the content and chunk it to help with conceptualizing the whole of it.  Then came the LTs which were done by me and my team, all of whom are experienced teachers and thoroughly familiar with different parts of the literature.  We worked with local teachers to write initial items and to put interesting content links on the map and then using some of those items, we learned to write conceptual items and did so from elaboration documents describing each level and the cases.  We implemented early in our partner schools and they gave us feedback as we built the reports.  They were the ones that wanted an item analysis and a means to display the percent of students with misconceptions.  As the team worked on validating the assessments, we studied how teachers discussed the data with students and then built our professional development materials based on what we saw.  We believe that involving teachers, pyschometricians, learning scientists, and engineers as an interdisciplinary team is essential to get a product that is useful in classrooms.   Thanks for asking.

  • Icon for: Beth Hulbert

    Beth Hulbert

    Co-Director
    May 14, 2018 | 10:03 p.m.

    This sounds really interesting and relevant to teachers. I am wondering if there is training for the teachers to help them develop tools and strategies to move students along the learning trajectories?

     

  • Icon for: Jere Confrey

    Jere Confrey

    Lead Presenter
    May 15, 2018 | 08:27 a.m.

    Hi Beth.

    Yes, there is professional development available but it focuses more on the use of the tool.  We have been studying the different ways teachers discuss the data with their classes and developing a framework to capture the major categories of approaches.   We have also been observing teachers working in PLC to share their strategies with others.  We are at the beginning of this work and would be glad to share more of it over time.  Most importantly, there is a need to be explicit about how to think about the relationship between student-centered instruction and the data on the LTs.   Thanks for asking.

  • May 16, 2018 | 04:36 p.m.

    Dear Jere,

    This work seems very integrative of prior work you have led, it seems, in terms of emphasizing underlying structure and mathematics and relying on coherence in large concepts rather than aggregation of constituent concepts.  Do you find significant variation in learning trajectories for the big ideas.  My concern about LTs has always been around the hazy but crucial role that intuition plays, and the difficulty in fully sorting out that role in understanding a youngster's trajectory....

     

  • Small default profile

    Jere Confrey

    Researcher
    May 16, 2018 | 08:42 p.m.

    Hi Eric,

    Thanks so much for looking at our video.  Yes, we did really decide to focus the structure of the map on big ideas with the notion that Standards can change but big ideas, not so much.  This way the map is aligned to CCSS-M but not dependent on them and can be adapted to different standards across states and around the world.    I think intuition is really important and it plays out significantly especially captured at the lower level of the trajectories.  But in other ways, this design does not really get at problem solving in that much of that rests in the connections among the clusters and spaces between them.  My view is that I would like to curriculum rich in projects and problem solving and I just want to come behind and check to see that in the rich buzzing complexity , the fundamental ideas are being learned deeply.  Does that make sense?  Thanks again for your thoughtful comment.

  • Further posting is closed as the showcase has ended.