R305A150456

The Strategic Education Research Partnership (SERP) partnered with Temple University Professor Julie Booth and teachers from several school districts to develop and test “MathByExample”—a set of math assignments for 4th and 5th grade students aimed at preventing the development of common misconceptions. This IES-funded research and development project is rooted in work begun a decade ago when a SERP partnership team was formed to respond to a request by district leaders in the Minority Student Achievement Network for support in narrowing the achievement gap in Algebra 1. They asked for a solution that could be incorporated into the regular classroom without singling out minority students and that did not require a whole new curriculum. AlgebraByExample was collaboratively developed to address the problem within the districts' constraint. Drawing on the research literature, assignments were designed with correct and incorrect “worked examples”—problems that have solutions worked out and marked as right or wrong—interleaved with problems to solve. Students respond to prompts for explanation of the correct or incorrect strategy. Despite the minimal change required by teachers, the intervention had a surprisingly powerful impact. AlgebraByExample students made substantially greater gains than peers in the control condition and, unlike many other improvement strategies, gains were greatest for students at the bottom of the performance distribution. Analysis of student responses revealed that many misconceptions have their roots in earlier grades. Thus, MathByExample was developed for younger students mimicking the AlgebraByExample approach. Preliminary results from a randomized controlled trial in five districts show similarly positive results. By strengthening students’ foundational mathematics understanding in 4th and 5th grades, teachers in higher-level mathematics courses will be freed to focus on the grade-level content, increasing the opportunity for more students to succeed in higher level mathematics.

## Janet Yowell

This is an outstanding video! Very well done. What a fantastic project with such positive impact.

## Julie Booth

Co-PresenterThank you so much! We have been thrilled with the results thus far.

## Nadine Bonda

This is an interesting idea to teach mathematical thinking. Teachers often shy away from showing students an incorrect example for fear that they will remember the incorrect process and incorrect thinking rather than a correct process and correct thinking. What prompted you to develop this technique? Do you have plans for a much larger testing of the materials than only the 5 initial classes involved in your study? Can you provide details of how you conducted the study and measured the results? Thank you.

## Julie Booth

Co-PresenterYou're absolutely right--teachers generally think that showing incorrect examples will be harmful, especially for students who struggle with learning math. However, we have published evidence that the incorrect examples may be even more helpful than the correct ones, and the low-achieving students benefit the most from the incorrect examples!

We draw on decades of research in cognitive science that have identified a variety of principles of processes that positively impact learning. In this case, we are combining the techniques of self-explanation, worked examples, and learning from errors to collectively benefit student learning in math. We actually did this work first in algebra classrooms and conducted a larger-scale RCT that showed positive results for students' conceptual understanding, procedural skills, and performance on released items from standardized tests. This work is published in the Journal of Education for Special Populations at Risk. I will try to figure out how to upload papers here for you!

For the MathByExample project, we conducted an RCT with 5th grade last year and are finishing up the 4th grade RCT currently. Results so far are promising! Again, I will try to upload a brief writeup for you to view!

## Allie Huyghe

Lead PresenterUnfortunately we are unable to upload additional attachments, but here are links to the two items Julie mentioned:

AlgebraByExample Research published in JESPAR: https://serpinstitute.box.com/v/booth-jespar-2015

MathByExample brief write-up of preliminary analyses presented at NCTM in April 2018: https://serpinstitute.box.com/v/mbe-nctmproposa...

## Nancy McGowan

Your video was very informative! :)The process outlined in the video will be very helpful for students grappling with mathematical problems and trying to make sense of what they see. Through your research, what specific areas of math do yo see as problematic for 4th and 5th graders? Where do these areas, if left unchecked, tend to catch up to students and create problems as they move on in higher math?

## Julie Booth

Co-PresenterWe're so glad you liked it! One of the reasons we embarked on the MathByExample project was because many of the persistent misconceptions we were seeing in the Algebra students like issues with order of operations, properties of numbers, and fraction and decimal concepts and operations were really elementary school math content. These types of misconceptions--perhaps especially those dealing with order of operations and fractions--are definitely problematic for students when they try to learn Algebra. We've created the 4th and 5th grade materials to target these and other potential misconceptions earlier so that, hopefully, the students will not have these lingering problems in middle school and can focus on refining their knowledge of other concepts and procedures.

## Karen Economopoulos

Math by Example seems like an important learning opportunity for students and their teachers as well! Are their professional learning materials aimed at helping teachers understanding students' misconceptions and errors as well as suggestions and support for teachers about how to untangle these misconceptions?

## Allie Huyghe

Lead PresenterThanks for your comment! You're absolutely right - teachers have reported learning from these materials as well, both in terms of research-based practices and gaining insights to how their students are thinking through the mathematics. In the original partnership work, one of the constraints district leaders put on the design of the approach was to not require professional development and training in order to adopt the materials, so we had to strategically design materials that were easy to adopt, but fostered "learning-by-doing." The questions paired with the correct and incorrect examples are specifically targeted to untangle common misconceptions that students have. That being said, we will be developing a comprehensive MathByExample website for teachers to learn about the approach and how it can be beneficial for learning, similar to the AlgebraByExample website (http://math.serpmedia.org/algebra_by_example/). This website includes resources, such as a set of slides (http://math.serpmedia.org/algebra_by_example/sl...), which includes information about myths related to incorrect examples and student learning.

## Beth Hulbert

This is an interesting project and the video is well done. I am wondering if you have used these with teachers as a way to discuss what the student misconceptions are and the potential basis for the misconceptions?

Dawnavyn James

## Allie Huyghe

Lead PresenterThanks for your comment! Our project has been focused on the development and testing of the assignments themselves, looking primarily at student outcomes. But with both the AlgebraByExample work and MathByExample work, several instructional leaders who work with the teachers using the assignments have reported that the assignments serve as a natural conversation starter for math team meetings. It is definitely something we will consider as we fully develop the website and provide resources and suggestions for use!

## Dawnavyn James

I teach project based math in a multi-age classroom. I can see Math by Example being beneficial for K-5. The idea of analyzing both an incorrect and correct worked example to identify procedure, promote critical thinking and help correct common mistakes is an amazing concept. I cannot wait to integrate this into our math curriculum! Thank you for sharing!

## Allie Huyghe

Lead PresenterThat's great to hear! If you'd like to be notified as soon as the materials become available, you may join SERP's mailing list: http://serpinstitute.org/mailing-list.html We'd love to hear your feedback when you integrate it into your curriculum!

## Rebecca Maryott

As I watched your video, I found myself nodding over and over again, agreeing with everything you said! I teach some low level and some higher level math courses in the high school, and continuously, the mistakes made are not about the new concepts, but involve misconceptions from previous math courses. I've often wanted to go into an elementary school in our district and see what is going on in the 4th-6th grade classes, and where the disconnect is with what I consider basic math knowledge. I'm excited to see that you are developing materials for the 4th-5th grade levels to help with these misconceptions!

With the focus of your set of problems being on the process instead of the answer, students are focusing on what matters in most math problems-analyzing the process of solving a problem. Mistakes are perfectly fine in a classroom, as long as students and teachers are willing to work towards turning those mistakes into learning opportunities.

I've already bookmarked the AlgebraByExample website and sent a link to my co-workers. I plan to utilize many of the resources next school year in my Algebra 1 classes! Thank you for providing such a powerful resource that requires very little training on a teacher's end.

## Allie Huyghe

Lead PresenterThank you for your comment and for already sharing with your colleagues! We're thrilled to hear how relevant this work is to you and your students. We encourage you to reach out and let us know your experience with the materials next year! (info@serpinstitute.org)

## Katey Walton

I teach high school level students. I often still see basic mathematical misconceptions from my students on a regular basis. I was encouraged to see that this project is attempting to dislodge those common mathematical misconceptions. I think that a lot of teachers are apprehensive to show students incorrect ways to solve a question, because they are afraid that the incorrect way may be the method the student remembers. Have you found this to be true in any of your research? Last of all, is your AlgebraByExample book meant to be supplemental material for an algebra classroom or do you believe that this workbook would provide enough practice for a student to master Algebra?

## Allie Huyghe

Lead PresenterThank you for your questions!

As for teachers being apprehensive about showing students incorrect ways to solve a problem, that has certainly come up. Prior research has shown that the combination of incorrect and correct examples is more beneficial to students' learning than correct examples alone (Durkin & Rittle-Johnson, April 2009; Grosse & Renkl, 2004, 2007; Rittle-Johnson, 2006; Siegler, 2002; Siegler & Chen, 2008). But it's really important to explicitly mark/notate the example as correct or incorrect, and not ask or expect students to figure it out. In addition to the clear indication of incorrect work with the X, the presence of targeted questions that are directly related to the error eases hesitations. In fact, many of the teachers who have partnered with us as part of the research study have voluntarily expressed how they think the incorrect examples are actually more helpful than the correct examples!

As for the AlgebraByExample workbook (and the soon-to-be released MathByExample workbooks), we definitely consider the assignments to be a supplemental resource. It can be used in addition to or instead of other practice worksheets as a do-now, exit ticket, classwork, etc. But we do not consider it to be comprehensive. Please let us know if you have other questions!

## Julie Booth

Co-PresenterHi Katey--we've definitely heard teachers' apprehensions about showing incorrect work, and they are often particularly concerned about the lower-achieving students in their classes being harmed by working with incorrect examples. Because we wanted to be able to address these concerns directly, we have tested the impact of correct vs. incorrect examples (e.g., do they need the incorrect examples to learn, or are the correct ones enough) and investigated interactions between the effectiveness of the different types of examples and individual differences in student prior knowledge. Our research has shown that the incorrect examples are particularly beneficial, and that the benefit of incorrect examples is highest for students who struggle with learning math. If you're interested, you can find the article here! https://www.researchgate.net/publication/301759...

## Mari Strand Cary

Hi Julie and team! This is a great video and such an important topic / time period to be getting kids on track. Love the animated characters. The properly- and improperly-worked examples approach is so useful across domains. As I'm moving into the computer science space, I'm thinking about this more and more in terms of debugging.

Applying incorrect example reviews by students is an important and novel approach in it's own right, but is there anything this project revealed about what makes a particularly good or poor worked example that surprised you (e.g., that wasn't in the literature yet)?

In my work developing a iPad math program/intervention for kindergarten, we pursued the idea of incorrect examples for awhile, but had to table it due to lack of resources; this discussion is inspiring me to pursue it again! Thanks!

## Julie Booth

Co-PresenterHi Mari! The worked examples themselves need to be clearly marked as correct or incorrect, and should each target a single concept or misconception. But the bigger surprises were in what matters for the accompanying self-explanation prompts. Many people's instincts are to just ask what was right or what was wrong, and have them fix the wrong answers. Instead, we focus the self-explanation prompts on the particular features of the problem that are relevant to the target concept or misconception. The key is to get learners to notice the components of the problem that make the shown procedure right or wrong, to think about the meaning of that feature in the problem, and to force them to confront common misconceptions about those features. Even if a student doesn't hold that particular misconception themselves, they can still learn quite a bit from refining their thinking about that feature.

Would love to hear more about your efforts with little kids! I was on a dissertation committee recently where the student tied this approach to debugging in college-level intro computer science. My sense is that with coding you do need to let them do the correction as well, but that it would be good to have them think critically about the conceptual features first.

Further posting is closed as the showcase has ended.