1. Heather Howell
  2. Associate Research Scientist
  3. Developing Preservice Elementary Teachers’ Ability to Facilitate Goal-Oriented Discussions in Science & Mathematics via the Use of Simulated Classroom Interactions
  4. Educational Testing Service, Mursion
  1. Jamie Mikeska
  2. Research Scientist
  3. Developing Preservice Elementary Teachers’ Ability to Facilitate Goal-Oriented Discussions in Science & Mathematics via the Use of Simulated Classroom Interactions
  4. Educational Testing Service
  1. Carrie Straub
  2. https://www.linkedin.com/in/carriestraub/
  3. Executive Director
  4. Developing Preservice Elementary Teachers’ Ability to Facilitate Goal-Oriented Discussions in Science & Mathematics via the Use of Simulated Classroom Interactions
  5. Mursion
Public Discussion
  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 13, 2018 | 04:29 p.m.


    Welcome and thank you for taking a look at our video!


    Our project is now in its second year, and we are now wrapping up our initial task design and deep into small-scale tryouts of the new performance tasks in both mathematics and science. And we are just getting started with our faculty collaborators who will be using the simulated teaching tools in their elementary math and science methods courses.


    In the video, you'll see what it looks like to be in the simulator teaching the "students". Much of our work this year has focused on the first iteration of tryouts, in which preservice teachers enter the simulator, try out a task, and give us feedback that we can then use to improve the task.


    One of the really exciting parts for me has been watching groups of performances on a single task and seeing the variety of different ways that the preservice teachers approach the lesson. There is something to see in watching each teacher take on the same teaching challenge that is really different than what I used to see when I observed teachers in their own classrooms, each responding to a different set of challenges. I can't wait to see how this plays out in our collaborators' methods courses in coming years.


    Please jump in and share your thoughts and questions - we are so glad you stopped by and we are looking forward to the conversation!


  • Icon for: Miriam Sherin

    Miriam Sherin

    Professor, Associate Dean of Teacher Education
    May 13, 2018 | 08:34 p.m.

    Such an interesting project! Really fascinating to give pre-service teachers a chance to develop their ability to lead a discussion through the technology rich environment of the simulator context.

  • Icon for: Carrie Straub

    Carrie Straub

    Co-Presenter
    May 14, 2018 | 11:18 a.m.

    Thanks, Miriam.  We hope to provide opportunities for practice that might not be readily available.  The ability to facilitate classroom discussion is something that even veteran teachers may need support with, so we find that this technology gives a safe place to practice and make mistakes.

  • Small default profile

    Courtney Bell

    Researcher
    May 14, 2018 | 04:34 a.m.

    The work looks great!  I would love to learn how you are thinking about working with methods instructors or other teacher educators to design what the avatars say and/or what the formative feedback is to the preservice teachers.  

  • Icon for: Carrie Straub

    Carrie Straub

    Co-Presenter
    May 14, 2018 | 11:16 a.m.

    Hi, Courtney! :) We are eager to see how the teacher educators provide formative feedback as well.  When we began talking about the project, I had it in my mind that we would be structuring the feedback, but actually that was not the case.  One of the interesting facets of this project is that we are not prescriptive in terms of feedback,  but looking to see how teacher educators use this technology to inform their teaching, and what method of feedback they find to be most helpful.  As we start in on the formative tasks in the coming year and teacher educators are integrating the simulations into their coursework, we will surely find out a lot. 

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 14, 2018 | 09:02 p.m.

    We are working out the feedback question now. Our plan is that there are (at least) two layers to this. First, our raters will provide some feedback as they score performances. We won't be giving the preservice teachers scores (and they've made in quite clear in focus groups that they don't want them!) but we will provide feedback, which will be aligned to the scoring rubric but written in narrative form and citing specific examples. Second, as Carrie mentioned, the teacher educators may choose to give additional feedback. Beyond this, its possible that the teacher educators will choose to structure course assignments in ways that provide other types of feedback - for example asking the preservice teachers to reflect on their own or others' performances.

  • Icon for: Brian Drayton

    Brian Drayton

    Researcher
    May 14, 2018 | 10:00 a.m.

    Hi, Heather, Many questions come to mind!  Here's two:

    1. What is the bridge for the pre-service teachers between this simulation and leading discussions with actual students?  Have you studied that transition?  (or maybe you will down the road) —What characteristics of discussion-leadership would you predict would change for the pre-teachers, or be hard to change?

    2. Are all the simulated conversations one-to-many?  That is, the teacher leading a discussion with the group — as opposed to a discussion among the students (say, over a problem solution), with the teacher listening, and intervening strategically? 

    -- brian

  • Icon for: Carrie Straub

    Carrie Straub

    Co-Presenter
    May 14, 2018 | 11:10 a.m.

    Hi, Brian!  I can help to respond to question 2, and I am sure Heather has great knowledge in this area too.  The technology allows for both type of interactions, whether the teacher is mediating the conversation, or has built the skills with the group so that they can discuss with minimal teacher intervention. One of the challenges of the work is knowing when to shift the simulation in response to the teacher's behaviors.  For example, should students "unlock" and begin discussing after 1, 2, 3, ... prompts, and if some prompts to discuss are more effective than others, how should students respond differently?  These questions often arise when creating simulations and student profiles to support class discussion. 

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 14, 2018 | 09:07 p.m.

    Great questions! So far we are only in tryouts and early data collection, so haven't had the opportunity to observe change. But some of the things that stand out in the performances so far are (not surprisingly) a tendency for teacher-talk to dominate, and for the participants to struggle with the idea of what argumentation is. I think we already know from prior studies that these things are hard to change, but our hope is that doing it "with training wheels on" might make it a little easier in a number of ways. First, having that common experience with peers and the chance to receive feedback should help with noticing. Second, having a chance to engage multiple times should create a space of relative freedom in which to try out some more ambitious and perhaps intimidating teaching moves.

    We don't plan in this study to study the bridge between this approximation and working with real students, but its a fantastic follow up idea.

  • Icon for: Carrie Straub

    Carrie Straub

    Co-Presenter
    May 14, 2018 | 11:05 a.m.

    Thank you for taking the time to visit this site and learn about our project.  We are in our second year of work, and learning so much about how to create practice opportunities that strike the right balance between interactivity and consistency. At Mursion, we are very focused on how to support our simulation specialists who drive the simulations, so that they have a deep understanding of the content and can provide rich opportunities for candidates to facilitate discussions in science and mathematics. We are happy to respond to your questions, because they help us think more deeply about the work!

  • Icon for: Courtney Arthur

    Courtney Arthur

    Facilitator
    May 14, 2018 | 07:46 p.m.

    This is a really great project. I love the way you are allowing preservice teachers to really practice and build upon their skills in leading classroom discussions.

  • Icon for: Jamie Mikeska

    Jamie Mikeska

    Co-Presenter
    May 14, 2018 | 11:19 p.m.

    Thanks Courtney for taking a look at our video. We agree that the power in this approach is in providing a safe space for the preservice teachers to practice facilitating discussions. We also think that it will be important for them to receive formative feedback based on their performance and then have an opportunity to try again using the feedback to guide their next time in the simulator. We will begin implementing the formative feedback piece next year and can't wait to see how the preservice teachers respond and use the simulations to build their skills over time in the elementary math and science method courses.

  • Icon for: Louis Gross

    Louis Gross

    Facilitator
    May 15, 2018 | 11:28 a.m.

    Heather et al., Thanks for creating an enticing video about new technology methods to both enhance teachers practice in a non-threatening manner and to advance research on how novice educators approach dealing with groups. I can certainly see how this has many possible future directions including the evaluation and transition questions you addressed in comments already. I assume that the pre-service teachers involved have considerable life experience with game/avatar systems, so I wonder if indeed any of them have tried to "game the system" itself by looking for ways to trip up the avatar coding! I certainly see some of my tech-savvy students trying to muck around with any tech tools I present to see if they can "break them". Indeed, it might be good to get some testers involved if they are not already to assess how the avatar system breaks down (if it does). 

    Cheers,

         Lou

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 15, 2018 | 10:25 p.m.

    So far we haven't seen that behavior. Our tryout participants have predominantly been novice teachers, ranging from very early in coursework before fieldwork to a few who have been in the classroom for less than a year. For the most part they take it seriously, and if anything find it a bit nerve racking because they are teaching in the lab in front of research staff. And to the extent that they occasionally don’t give it a full effort I’d say it falls much more in the category of not preparing sufficiently and not deliberately trying to undermine the effort. There also isn't technically avatar "coding" in terms of their responses because, although the images are of course coded and the interaction mediated by software, they are controlled on the back end by a trained human being, a "simulation specialist". That said, I think your point still holds. Whether computer coded or dictated by the training and experience of the simulation specialist, participants can still push the envelope by attempting things that are truly unexpected, perhaps because the want to break the system as you suggest, but also perhaps because we just haven’t anticipated some of the things they could do or say adequately to prepare the simulation specialist. The training is extensive, but it doesn’t mean something really unexpected could not happen. I’d say we have strong training to cover most of the likely approaches participants will take, some general sets of parameters to help in the moment if something that makes sense but is less likely occurs, but in truth if a participant were to go 100% off the intended path (say, they decide not to teach math today but instead to discuss world literature), well, in those cases we really rely on the simulation specialist’s background training in improvisation to figure out what to do in the moment that makes sense as a response. Again, not something we’ve seen, but you never know:)

  • Icon for: Louis Gross

    Louis Gross

    Facilitator
    May 16, 2018 | 01:25 p.m.

    Heather, now I understand the situation better - thanks. I thought that this was a sim in the usual sense of an independently coded set of responses that might be adaptive, but were hard-wired. I wasn't aware that there was a real-time behind the scenes individual monitoring and changing the avatar responses. That is a rational way to do this, though of course less scalable. Have you considered crowd-sourcing responses of avatars though? Though difficult, that would be cool in that it could bring to bear the experiences of many teachers.

  • Icon for: Meixia Ding

    Meixia Ding

    Higher Ed Faculty
    May 15, 2018 | 03:31 p.m.

    This is a very cool project! Have you found (or do you anticipate) any difficulties in transitioning the learned knowledge from virtual to actual classrooms? 

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 15, 2018 | 10:38 p.m.

    In the present study this is not a question we have yet taken on, although its clearly one of the more pressing ones for future research to address. One of the things I find exciting about the technology is that I think it pushes us to think systematically about what we really think the theory of action is behind how novice teachers learn to teach by practicing teaching. Pushes us because to set up a simulation you have to make choices, and making those choices adopts a theory of action, whether explicit or implicit. We’ve taken some stands in this work. For example, we hypothesize that practice leading a discussion in a smaller group of students helps prepare the novice to lead discussions in larger groups; likewise, we hypothesize that taking on ambitious teaching first absent the challenges of classroom behavior will make it more likely a novice can handle ambitious teaching in the face of those challenges, and we choose to use student avatars who behave. These are big claims about how novice teachers learn. That said, they aren’t bigger than the ones we routinely make in teacher preparation; ones like the assumption that peer teaching or microteaching in the context of a methods class where the “students” are adults and peers will help you learn to teach children who are not your peers, or even the assumption that student teaching under the watchful eye of a cooperating teacher prepares you do teach without that support. Note that my point is not that I don’t believe these claims (I do), but that we don’t often articulate them or support them with rigor.

     
    1
    Discussion is closed. Upvoting is no longer available

    Carrie Straub
  • Icon for: Dave Barnes

    Dave Barnes

    Facilitator
    May 15, 2018 | 07:51 p.m.

    Heather, Jamie, and team,

    Very interesting and exciting. I have to say that establishing productive and engaging classroom discourse in mathematics is, I believe, one of the keys to engaging all students in productive and learning and sense making.

    I am very interested the models you are using for mathematics with respect to the tasks which the teachers choose to do with the students and what strategies the teachers are working to implement to generate productive discourse.  Are the teachers able to ultimately facilitate in ways were the students are mostly driving the discussions? 

     

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 15, 2018 | 10:49 p.m.

    So far, the data we’ve collected has been from small scale tryouts largely aimed at refining the tasks. Our participants in tryouts have engaged in the simulation just a single time, and without feedback or support. And so far, while we do see some variation in approach, the majority demonstrate more an IRE interaction pattern than true discussion. That said, the simulation can certainly handle it, and in our training we have worked towards this and put it in the test by going in and acting the part of the teacher ourselves, sometimes over exaggerating the degree to which we step back just to make sure the simulation specialists are prepared to deal with that if and when it happens. So I guess the conclusion from this is that, with notable similarity to what happens with novice teachers in real life outside the simulator, the students are available and able to drive the discussion but for the most part the teachers don’t know to or don’t know how to make that happen.

    Starting next year all of this changes from our point of view as we collect data in methods courses where each preservice teacher uses the simulator multiple times and has the benefit of being able to reflect on their video record, receive feedback, and have the support of the course instructor. And we certainly hope that ultimately they are able to facilitate the discussion in a way that lets students drive. But we will have to keep you posted on that because we just don’t have the data yet!

  • Icon for: Eli Meir

    Eli Meir

    Researcher
    May 15, 2018 | 09:43 p.m.

    This looks really interesting, thanks for sharing. Can you say a bit about the simulation itself? Does the simulation environment hear what the teacher in training is saying, and respond appropriately (i.e. the avatar students respond to the teachers specific questions)? That seems really challenging - can you say a little about how well it works? How often do the avatars do something that is well-matched to what real students would do in response to the the same words from the teacher? How often do they do something else? Do they respond to non-verbal cues, like a teacher that waits for a few seconds in silence?

  • Icon for: Carrie Straub

    Carrie Straub

    Co-Presenter
    May 16, 2018 | 11:02 a.m.

    Great question, Eli!  Heather addressed the main issue in her response below, so that should help to clarify how the simulation can respond in real time to the learners.  We use the virtual environment to create what I like to think of as a digital veil, that allows the simulation specialist to inhabit multiple characters (avatars) at the same time - up to 5 avatars at once!  These avatars can be operated by the simulation specialist or run by the computer through a series of algorithms.  The "human in the loop" adds the spontaneity and moves the simulation past being a basic logic-branching type of simulation, and into something much more interactive and realistic. 

  • Icon for: Heather Howell

    Heather Howell

    Lead Presenter
    May 15, 2018 | 11:36 p.m.

    If I understand the question, I think the most important thing to know about the simulation is that its not AI. Behind the curtain is a puppeteer, who is viewing the participant via webcam. And because behind the scenes is a human being, yes, they can see the teacher in training, respond appropriately (or if desired, inappropriately, but we don’t focus on classroom management!), respond in the moment in reasonable ways, and respond to non-verbal cues. Carrie can say more about the exact mechanics of how this works and how the simulation specialist is able to manage running five students at a time; its kind of breathtaking to be honest how well they pull it off.

    There are ways, of course, in which human beings also represent limitations. We can’t give instant feedback, and we have to design and rely on rigorous training processes to ensure comparable experiences for each participant. There are likely practical limits to the amount of information that a specialist can learn and manage (and therefore to the complexity of the task), and it remains more costly to run this kind of simulation than something fully automated would be. But its also kind of the magic behind how the system works. It feels real because in a way it is real; you’re talking to a person and that’s what it feels like, and that’s what allows you, in the moment, to suspend your skepticism and just, well, teach them. I think each of our team members experienced this the first time we tried it out. To quote one of our participants, “at first, I… I guess I had bias because I thought they were cartoons. You know, cartoons aren’t going to come up with the… like I said you quickly forget that they're cartoons because the personality shines through”.

  • Further posting is closed as the showcase has ended.