1. Catherine McCulloch
  2. https://www.edc.org/catherine-mcculloch
  3. Senior Project Director; PI, CADRE
  4. Community for Advancing Discovery Research in Education (CADRE)
  5. cadrek12.org
  6. Education Development Center
  1. Hilda Borko
  2. Professor of Education
  3. Community for Advancing Discovery Research in Education (CADRE)
  4. cadrek12.org
  5. Stanford University
  1. Amy Busey
  2. https://www.linkedin.com/pub/amy-busey/7/516/484
  3. Senior Research Associate
  4. Community for Advancing Discovery Research in Education (CADRE)
  5. cadrek12.org
  6. Education Development Center
  1. Christine Cunningham
  2. http://eie.org/sites/default/files/cmc_short_bio.pdf
  3. Founder & Director, Engineering is Elementary
  4. Community for Advancing Discovery Research in Education (CADRE)
  5. cadrek12.org
  6. Boston Museum of Science
Public Discussion
  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 13, 2018 | 09:09 p.m.

    Thank you for visiting the CADRE video. With this video we are hoping to bring a new kind of discussion to the video showcase.  The video is not about one project’s work. It is about all of our work to improve education through research. The video is meant to foster discussion about different approaches to having a broad impact. What are these approaches? How are we making impacts? With whom? And how do we know we are making an impact? What new approaches are needed? We invite you to respond to these questions.

  • Icon for: Mark Windschitl

    Mark Windschitl

    Facilitator
    May 14, 2018 | 12:59 p.m.

    Catherine, you certainly have a big impact in terms of numbers. I have used some of these resources myself and found them to be "one of a kind" that help me support learning. What have you learned about scaling up in terms of how you market these resources, make them available, innovate on them based on user input? 

  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 14, 2018 | 02:54 p.m.

    Mark, as you note, disseminating resources is one of the ways that we (the DRK-12 resource network) try to increase our impact and that of research funded through NSF’s DRK-12 program. We continue to learn how important it is to involve the user group in development from the outset. For all of our efforts, we start by assessing need and interest in a process and/or product. In response, we set off on an iterative process of development, testing, and refining. We have used this process for continuous improvement of long standing efforts such as our early career development program and the STEM Smart initiative, as well as for the development of products that come out of efforts such as those. We mine information about how, when, and through what means resources and information are accessed to refine our knowledge about user needs and interests, but we are still looking for better ways to understand how those resources and information are being used by those outside of our immediate network.

     

    Working in partnership with folks across the education system is key too. Others (including DRK-12 projects) are using approaches such as ongoing research and practice partnerships. We would love to hear more from these projects as well as those using other approaches, and how folks are collecting evidence of impact.

     
    1
    Discussion is closed. Upvoting is no longer available

    Megan McKinley
  • Icon for: Kevin Brown

    Kevin Brown

    Facilitator
    May 14, 2018 | 02:07 p.m.

    Yes, very inspirational stories of the successful dissemination of NSF funded research to achieve broader impacts! I know that there is often a tension between achieving wide dissemination and maintaining sufficient fidelity of implementation to ensure that the intervention continues to function as originally designed. I’m wondering how the projects highlighted have grappled with this dilemma as they scale up their efforts and if they’ve been able to study the conditions under which interventions can successfully scale, such as the comment in the video about the advantages of interventions delivered online? Also welcome would be any advice or lessons learned on how these projects have been able to reach millions of students!

  • Icon for: Christine Cunningham

    Christine Cunningham

    Co-Presenter
    May 14, 2018 | 05:39 p.m.

    I lead one of the projects featured in the video, Engineering is Elementary. We've had many team discussions about fidelity/integrity of implementation. One of which is: (how) does it matter for outcomes? Our current DRK12 project, an efficacy project with hundreds of teachers, is taking a glimpse at this facet--we are asking teachers to report what they have done that aligns with our lessons but also what they have adapted to better suit their needs. We also are audio and videotaping a subset of the lessons. We're looking at how these issues affect student outcomes related to engineering and science knowledge, as well as attitudes. We are crunching the data now!

    Our initial field testing of our curricular materials (in about 60 very diverse classrooms) was designed to help us create materials that could scale in a range of conditions. So far, such robust testing in difficult (under-resourced) environments, seems to have helped us distill a set of materials that can be used by teachers in their classrooms as they exist today. In our curricular development we have tried to walk the line between helping teachers to innovate and introduce new practices in their classrooms while keeping the realities of classroom life in mind (large classes, no time, extremely diverse classrooms etc etc). It'd be interested in hearing whether and how others straddle these two dimensions.

  • Icon for: Barbara Berns

    Barbara Berns

    Education Planner
    May 14, 2018 | 03:02 p.m.

    Having seen some of these projects at their very beginnings, it is incredible to hear of their very significant impacts. For materials development and dissemination, quantitative results are impressive. BUT for the DRK-12 projects focused on professional development for strategies, can I assume it is still more difficult to identify impacts? What are some successful measures?

  • Icon for: Amy Busey

    Amy Busey

    Co-Presenter
    May 14, 2018 | 04:04 p.m.

    Barbara, the challenge of identifying impacts of professional development (PD) strategies and programs is persistent and has been the focus of several recent CADRE activities. Over the last year, CADRE led efforts to characterize, advance, and promote research and development around online and blended PD strategies in the DRK-12 portfolio. For example, in a 2017 survey of active projects focused on online or blended PD, fifteen awardees provided information about the outcomes measured as part of their research. All 15 were measuring teacher outcomes, with changes in (1) instructional practice (N=12), (2) attitudes/beliefs (N=9), (3) implementation of project-developed materials (N=9), and (4) pedagogical content knowledge being the most common. Eleven of the 15 were measuring student outcomes, with changes in (1) student achievement/performance (N=9) and attitudes/beliefs (N=3) being the most common. For more information on the instruments that DRK-12 projects have used, see CADRE's Compendia of Research Instruments for teacher (Part I) and student (Part II) outcomes.

     

    In addition, twelve projects recently shared shared emerging findings related to their online or blended PD strategies at the 2018 AERA PI Meeting as part of a structured poster session. The posters, which characterize their research and initial findings (in some cases including impacts), are accessible in a virtual poster hall on cadrek12.org. 

     

    We invite others who are examining the impacts of professional development strategies in their project work to offer their perspectives and insights around this challenge!

  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 18, 2018 | 03:32 p.m.

    Visit the DRK-12 video page, Impact of a Teacher-Led Early Algebra Intervention (PI: Maria Blanton), for a discussion of evidence of the impact of a professional development effort.

  • Icon for: Hilda Borko

    Hilda Borko

    Co-Presenter
    May 15, 2018 | 01:00 a.m.

    Barbara, you raise a great question about the impact of projects focused on professional development. In a previous DRK-12 collaborative project between Jonathan Osborne and myself at Stanford, and Craig Strang and Emily Weiss at Lawrence Hall of Science, the Stanford team studied the impact of PD facilitated by the LHS team on science discourse practices and students’ science achievement.  Using a measure of science discourse that we developed, we did find an impact on teachers’ and students’ discourse practices. We also found some evidence for an impact on student achievement; however, this finding is based on very limited data due to difficulties in attrition and collecting data. One article we published describes the instrument and presents initial findings:

     

    Fishman, E.J., Borko, H., Osborne, J., Gomez, F., Rafanelli, S., Reigh, E., Tseng, A., Million, S., & Berson, E. (2017). A practice-based professional development program to support scientific argumentation from evidence in the elementary classroom. Journal of Science Teacher Education. http://dx.doi.org/10.1080/1046560X.2017.1302727

     

    We are currently revising an article that includes analyses of the entire data set.

     

    I’d also like to add another layer of complexity: projects focused on scaling up by preparing PD leaders. In another DRK-12 project, Karen Koellner, Jennifer Jacobs and I worked with a local school district to prepare site-based teacher leaders to facilitate the Problem-Solving Cycle model of mathematics professional development at their schools. We were able to identify changes in teachers’ knowledge and some features of instructional practice, although our analyses were descriptive due in large part to attrition in the samples of teacher leaders and teachers.  The nesting of students within classrooms (teachers), and teachers within schools (teacher leaders) also added to the challenge of attributing changes to the preparation of the PD leaders. The project is described in our book, which also lists additional publications:

     

    Borko, H., Jacobs, J., Koellner, K., & Swackhamer, L. (2015). Mathematics professional development: Improving teaching using the Problem-Solving Cycle and Leadership Preparation models.  New York: Teachers College Press.

     

    My current project adds yet another layer of complexity in that it is a Research-Practice Partnership to adapt and enact the Problem-Solving Cycle and Leadership Preparation models. We don’t have findings to share yet. You can learn about the project in this video:  http://stemforall2018.videohall.com/p/1299

     

    I agree that it’s more difficult to identify the impacts of projects focused on professional development. As Amy pointed out, we are making progress in developing measurement tools and identifying outcomes, in large part because of DRK-12 funding and CADRE efforts. I also look forward to reading about other projects’ experiences and insights!

  • Icon for: Jake Foster

    Jake Foster

    Informal Educator
    May 15, 2018 | 09:26 a.m.

    As a prior state education department member, I am particularly interested in how we facilitate the dissemination of effective strategies and programs to educators and leaders across the K-12 system. The work CADRE is doing to coordinate across federally funded projects and support dissemination is an excellent step in making such work from the research community more systemic and readily available to the practitioner community. As a project advisor for CADRE I am grateful for the work Catherine, Amy, and all the staff are undertaking to disseminate to a wide audience.

    With that said, I am interested in hearing from educators, leaders, and policy makers about what how findings and products from the research community would be most useful for your work. Is it the actual products that are developed that have the potential to be taken up in your school, district, or state? Is it the broader strategies or findings that you could apply to your own program development? Or might it be a process that you can employ as you review or build STEM programs?

  • Icon for: Megan McKinley

    Megan McKinley

    Facilitator
    May 15, 2018 | 10:54 a.m.

    Hello, Katherine and colleagues. It’s incredible that you’ve been able to reach so many students and teachers. What do you envision as next steps for your work? Have you investigated the longitudinal impact of Engineering is Elementary on students’ interest in engineering specifically and STEM fields in general?

  • Icon for: Christine Cunningham

    Christine Cunningham

    Co-Presenter
    May 15, 2018 | 11:28 a.m.

    EiE is working on a number of exciting ways that we can reach more students. We have just finished development of Wee Engineer, a curriculum for preschool engineering, and EiE-K, a curriculum for kindergarten. These are in production but will be available in early fall. We are starting to discuss research projects to study facets of these with external researchers.

    In terms of our impact, we have looked at children's interest in engineering and in science and find those increase after engaging with one of our units (these data are from only one year, we have not been able to track students longitudinally). We have also measured students' engineering and science understanding and those both increase as a results of engaging in engineering design challenges. 

  • Icon for: Barbara Berns

    Barbara Berns

    Education Planner
    May 15, 2018 | 09:39 p.m.

    Jake's comments brought back an idea I've been thinking about for a while. It is not a long term or systemic solution, but it could lead to some short and mid-term improvements in STEM education.  So much of all of our dissemination efforts focus on awareness building - building familiarity and knowledge of practitioners, policy makers, education leaders, researchers and others. While there have been resources for this type of  dissemination centers/networks/etc., in the recent past, there have not been resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts. Is this something worth pursuing with foundations, corporations, or government agencies? 

  • Small default profile

    Judith Sandler

    May 16, 2018 | 11:22 a.m.

    Given that there are funds under ESSA that could be focused on STEM, it would be great to share both the research data from CADRE and the contacts for researchers with STEM leaders in SEAs and LEAs. At the local level, teachers and administrators are going to need guidance and resources to help them use funds wisely. We all need to keep the pressure on the urgency for STEM curricula and instructional materials, professional development, teacher training and support, especially when funds are so competitive at the local level.  Also critical will be to educate policymakers and the administration at the federal level on the importance of foundational STEM in K-12 as policies are being written for career and technical education. While we know the importance of having a well skilled workforce, we also know how easily resources and time for science, especially, and math can be marginalized if we don't participate in and steer the conversation.

  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 17, 2018 | 12:14 p.m.

    Judi, CADRE recently hosted a webinar that included tips for those meeting with policymakers. It may be helpful for anyone who is interested in making informative visits about STEM education and research. 

    http://edc.adobeconnect.com/pgoroc9h4ohz/ 

     

  • Small default profile

    Judith Sandler

    May 17, 2018 | 12:59 p.m.

    Thanks for the update Catherine. I am on the Board of the STEM Education Coalition and would recommend it as another source for your site. 

    There are excellent resources at www.stemedcoalition.org to help folks working with policymakers... current updates on the federal STEM legislative and policy agenda as well as a growing list of resources for state policy work. Information on membership is also available on the site.

    Happy to answer any questions: jsandler@edc.org

  • Small default profile

    Terrell Morton

    Researcher
    May 16, 2018 | 02:17 p.m.

    Hi, 

    I appreciate both the video and the questions posed regarding the impact of education research and the question of "who" is actually benefiting from it. I think these are critical questions that all researchers should consider when designing and implementing their studies, particularly when it comes to developing and testing new technologies, practices, and programs. From personal experience, I've seen some great ideas that bring about a promise for a substantial impact that meets a lot of structural and systematic barriers that often reduce the level of impact or the potential of the project. I wonder how people are accounting for those structural and systematic barriers that occur. One prime example for such would be implementing new technologies or practices without formal professional development for the teachers or considering the actual infrastructure of the school to support and sustain those technologies and programs. 

    More often than not, when we run into these types of barriers, we find great ideas that are either implemented or scaled to districts and sites that have the resources to support them leaving those underresourced without the opportunity. Or, those same ideas only last for the intermittent period of the research grant, which again leaves those underresourced without the opportunity. In our efforts of promoting "STEM for All," I cannot help but wonder if we are truly fulfilling this mantra, or if we are promoting a "STEM for All, but with specific conditions."

    How did you (are you) accounting for institutional and systematic barriers? Were these a part of the conversation during the planning phase or did they come about after initiating the project? In facing those barriers, who (which demographic of students, teachers, etc.) were most detrimentally affected and how are you accounting for it? 

     

  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 18, 2018 | 03:06 p.m.

    Terrell,

    You have brought up very important points. Many DRK-12 projects are addressing systemic issues and approaches. For example, you might read the discussion on Ellen Meier's video page about work in the NYC public schools.

  • Icon for: Barbara Berns

    Barbara Berns

    Education Planner
    May 17, 2018 | 06:19 p.m.

    A few delayed responses. To Amy and Hilda, sounds like there has been a lot of progress and I look forward to reading the articles and reports you cited. To Judi and Jake, it seems like there may be some opportunities on the horizon. 

  • Icon for: Lisa Hogan

    Lisa Hogan

    K-12 Teacher
    May 18, 2018 | 01:29 p.m.

    Hello Catherine, Amy, Barbara and all,


    As a former science teacher and now an instructional coach I enjoyed both the video and the questions posed about the impact of education research, who is actually benefiting from it, and which professional development strategies are most impactful.


    I would like to comment on  Barbara’s question, “So much of all of our dissemination efforts focus on awareness building - building familiarity and knowledge of practitioners, policymakers, education leaders, researchers and others. While there have been resources for this type of dissemination centers/networks/etc., in the recent past, there have not been resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts. Is this something worth pursuing with foundations, corporations, or government agencies?”


    As an instructional coach, I work with teachers sharing education research, modeling practice, and debriefing instructional strategies. There is an awareness among teachers about STEM teaching, as well as some familiarity and knowledge about products that have the potential to impact student learning. What continues to be a challenge is helping teachers with the day-to-day “how to implement” STEM learning.  Teachers are trying to use education research to make changes in their practice but they are doing so while they are “flying the plane.”


    Resources set aside for states, districts or schools, who would like to move beyond awareness to initiate planning or early implementation efforts might be a way to provide the support needed for schools/teachers in a manner that would free teachers from “building the plane” while they are flying it.  I wonder if this would help teachers develop a deeper, thoughtful understanding of how to implement effective STEM learning.

  • Icon for: Catherine McCulloch

    Catherine McCulloch

    Lead Presenter
    May 18, 2018 | 03:24 p.m.

    Lisa, 

    Great to hear from you, and to have an educator's point of view. Thank you for pointing out the importance of finding ways to make research come alive for teachers, and to translate the abstract into the concrete, A wise educator once said to me, "...while teachers are aware of STEM teaching and research, they can't implement it because many have never seen it in action.  They can't do what they haven't observed. <They need more> modeling in classes.   As a result, the teachers might know the education research, might even know materials or programs that they could use, but trying to change their practice alone in classes is tough."

    There are DRK-12 projects applying different approaches to providing a more concrete vision of research-based instructional practice. I invite folks to view these videos as a showcase sampling:

    Understanding and Improving Learning from Online Mathematics Classroom Videos (PI: Meg Bates)

    Doing the Math with Paraeducators (PI: Judi Storeygard)

     
  • Icon for: Judy Storeygard

    Judy Storeygard

    Researcher
    May 19, 2018 | 11:52 p.m.

    In our Doing the Math with Pareducators project, we found that our group of paras  did not have experience thinking about math as sense-making. A goal of our PD was to provide them with opportunities to think, reason, and make sense instead of relying on procedures. At the same time, our PD featured activities that they need to teach to their students. They were able to make connections between the way they solved math problems in our PD to the strategies their students were using.

     

                   
  • Icon for: Hilda Borko

    Hilda Borko

    Co-Presenter
    May 18, 2018 | 09:58 p.m.

    Terrell,

    A delayed response to your comment: I agree with Catherine that you are raising very important points. Research-Practice Partnerships are one approach that I and others have taken to address structural and systemic conditions. Typically, the partners – often university researchers and district personnel – work collaboratively to identify a problem of practice and plan possible solutions, and they continue to collaborate as they work iteratively to design, enact, and study the solutions. I mentioned our research team’s partnership with a local school district focused on mathematics professional development in an earlier post. You can learn about the project on this video:  http://stemforall2018.videohall.com/p/1299

     

  • Icon for: Ellen Meier

    Ellen Meier

    Higher Ed Faculty
    May 19, 2018 | 07:27 p.m.

    Thank you, Catherine, for noting our project’s interest in systemic approaches for introducing STEM to schools!

    And thank you, Terrell, for your thoughtful query:  How did you (are you) accounting for institutional and systematic barriers? Were these a part of the conversation during the planning phase or did they come about after initiating the project? In facing those barriers, who (which demographic of students, teachers, etc.) were most detrimentally affected and how are you accounting for it? 

    In our Center’s experience, "institutional and systematic barriers" are present in every school. Our NSF project, "Systemic Transformation of Inquiry Learning Environments 2.0," (http://videohall.com/p/1182), identifies key elements for developing the capacity of teachers to design STEM projects in their individual school settings. In this way we are able to co-create and tailor the project to the needs of the teachers and administrators in each school. "Situating" the process (Putnam & Borko, 2000) in this fashion is fundamental to our approach.  Because we are focused on implementing key elements rather than a lockstep program, we have some flexibility in addressing barriers, even as they emerge.

    However, what is considered a "barrier" varies according to one's perspective. Teachers may see the lack of collaborative planning time as a barrier, even though the administrators may not yet identify this as a problem. The partnership between the university researchers, teachers, and administrators is critical, as Hilda Borko notes (above).  Strong partnerships create "shared meaning" (Fullan, 2015) rooted in the needs and interests of the school.  Shared meaning, in turn, makes it possible for those involved to identify creative ways to remove barriers inhibiting the effective implementation of the project.

  • Further posting is closed as the showcase has ended.