1. Kiley McElroy-Brown
  2. https://concord.org/about/staff/kiley-mcelroy-brown/
  3. Project Coordinator & Research Associate
  4. GeniGUIDE, GeniConnect
  5. https://concord.org/our-work/research-projects/geniguide/
  6. Concord Consortium
  1. Trudi Lord
  2. https://concord.org/about/staff/trudi-lord/
  3. Project Manager
  4. GeniGUIDE, GeniConnect
  5. https://concord.org/our-work/research-projects/geniguide/
  6. Concord Consortium
  1. Frieda Reichsman
  2. https://concord.org/about/staff/frieda-reichsman
  3. Senior Research Scientist
  4. GeniGUIDE, GeniConnect
  5. https://concord.org/our-work/research-projects/geniguide/
  6. Concord Consortium
Public Discussion
  • Icon for: Kiley McElroy-Brown

    Kiley McElroy-Brown

    Lead Presenter
    May 14, 2018 | 10:53 a.m.

    We hope you enjoyed our video highlighting Geniventure, our latest dragon-genetics tool for exploring genes and DNA. We are currently researching and further developing Geniventure's Intelligent Tutoring System (ITS) by using Bayesian Knowledge Tracing (BKT) to model each student’s understanding of the concepts in the game. We are also collaborating with experienced teachers to ensure that our Teacher Dashboard provides the support they need to help students succeed in the classroom.

    We invite you to make comments and ask questions about Geniventure and any other aspects of our project!

  • May 15, 2018 | 01:20 a.m.

    I am curious about the relationship between Bayesian Knowledge Tracing and the Teacher Dashboard. Can you say more about what the teacher sees or might see on the Dashboard?

    (I had seen BKT in a different context altogether of an online physics game that examined student growth, then generated specific feedback to that student based his/her patterns of play.) 

  • Icon for: Trudi Lord

    Trudi Lord

    Co-Presenter
    May 15, 2018 | 10:22 a.m.

    Hi Sherry!

    The dashboard provides two views for the teacher. The first view shows student progress and performance (P&P) and the second shows conceptual understanding (CI).

    The P&P view lets teachers know where each student is in the game, how many challenges they have completed, and their performance on each challenge, which is indicated by the color crystal earned. Teachers also see an indicator of how many times each student attempted each challenge. Teachers have found this view to be a game-changer. (No pun intended!)

    The CI view displays all the concepts tracked by the ITS and provides teachers with a window into student understanding. Teachers quickly see the specific concepts that students are struggling with (red flag) or have already mastered (green check). This information is provided for each individual student as well as in summary form for the entire class. Teachers can dive deeper into the report to learn more about each concept and make connections to related traits and specific game levels and challenges. For example, teachers may see that a majority of their students are struggling with incomplete dominance, which is addressed in Mission 2 with the Armor trait.

    Teachers have found the P&P and CO views helpful both in and outside of class time. Teachers use the dashboard to track students during the class period and often go back to the dashboard later, as they prepare for the next day's lesson. 

    Our BKT-infused ITS was built using past play data and is now being used to track student understanding of the concepts in real-time, delivering hints and remedial challenges as appropriate. Hints, which are text-based with visual highlights, are provided to the student after they meet a certain threshold of struggle. It's a fine balance--we let them have a little room to play but we don't want them to suffer too much! Hints are contextualized to each challenge type and address concepts and known misunderstandings with specific traits and patterns of inheritance. BKT is also used to determine if a student needs more than just a hint. If necessary, a simplified, remedial challenge is offered. Remedial challenges are highly scaffolded and guide students as they work through challenges that address one specific conceptual misunderstanding at a time. The dashboard alerts teachers when remediation is used. 

    Thanks!
    Trudi ~:)

     
    1
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
  • Icon for: Trudi Lord

    Trudi Lord

    Co-Presenter
    May 14, 2018 | 11:06 a.m.

    Thanks for watching our video!

    We would love to hear from others who have attempted to integrate an ITS into a game or provide teachers with real-time student performance data on complex tasks.

  • Icon for: Scot Osterweil

    Scot Osterweil

    Facilitator
    May 14, 2018 | 04:22 p.m.

    Nicely done. You've done a good job of explaining the product, and certainly captured enthusiastic responses from some users. I'd love to hear more about how the product is implemented in schools: who's using it, for how long, in what context. Also interested in what effects you hypothesize you will see, or already are seeing.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
  • Icon for: Kiley McElroy-Brown

    Kiley McElroy-Brown

    Lead Presenter
    May 14, 2018 | 06:02 p.m.

    Thanks Scot! 

    Currently, Geniventure is being tested in two levels of biology classrooms: middle school and high school. We've found that high school students can typically get through all of Geniventure in approximately 5five 45-minute class periods. However, that is roughly 45-minutes of strict gameplay. We hope to design teacher materials that would scaffold effective discussion points and opportunities for additional exploration outside of Geniventure into the classroom agenda.

    Through the GeniConnect project, Geniventure is also being used in more informal settings such as afterschool programs and summer camps. We are coupling the online game with hands-on activities that will not only foster middle school students' understanding of genetics concepts but also an awareness of STEM careers.

     
    1
    Discussion is closed. Upvoting is no longer available

    Trudi Lord
  • Icon for: Rachel Yim

    Rachel Yim

    Researcher
    May 15, 2018 | 11:52 a.m.

    Great video! I'm curious to hear whether you've compared students' learning gains using Geniventure against other modes of teaching and learning genetics.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
  • Icon for: Kiley McElroy-Brown

    Kiley McElroy-Brown

    Lead Presenter
    May 15, 2018 | 07:55 p.m.

    Thanks Rachel! We are still researching students' learning gains using Geniventure. The concepts that we've identified and are tracking were carefully chosen using Evidence Centered Design. However, we have not yet done a comparison against some other modes of teaching genetics. I'd be happy to share when we do!

  • Icon for: Jessica Hammer

    Jessica Hammer

    Facilitator
    May 15, 2018 | 02:23 p.m.

    Great video! You mentioned that you hope to design teacher materials to support discussion and exploration. In the video, it looked like some students were holding handouts. Are those already part of your experience, and if so, what are they? I also noticed that some students were playing in pairs. Is that a part of your study design? If so, can you tell us a bit about that choice?

     
    1
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
  • Icon for: Kiley McElroy-Brown

    Kiley McElroy-Brown

    Lead Presenter
    May 15, 2018 | 08:08 p.m.

    Hi Jessica - thank you and great questions! We do hope to design teacher materials as a part of our effort in developing a cohesive piece of technology for use in the classroom. The handouts you see in the video were developed by our teachers who are piloting Geniventure for us. We hope to work closely with these teachers as well as other experienced biology teachers to incorporate handouts and other guides for a productive classroom experience.

    Students playing in pairs is actually not part of our study design but turns out that it naturally happens in the classroom and we definitely encourage it when we set students up to play the game in class. We've observed that when students are given the opportunity to engage with each other during gameplay, they often learn from each other. Students are able to explain to each other, in their own words, how the mechanisms of genetics work, which hopefully will deepen their understanding of the concepts.

  • Icon for: Robert Zisk

    Robert Zisk

    Facilitator
    May 15, 2018 | 10:38 p.m.

    This program looks very interesting. The students in the video seem to be very engaged, and I am interested in seeing some of the data on learning outcomes. 

    You mention that you would like to see the students gain an awareness and interest in STEM careers. Do you have any data on student interest after playing the game? I wonder if the students are connecting the careers of characters in the game to possible real life careers.

     
    1
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
  • Icon for: Kiley McElroy-Brown

    Kiley McElroy-Brown

    Lead Presenter
    May 16, 2018 | 12:30 a.m.

    Hi Robert,

    We do have some preliminary data on student interest in science and STEM careers after our summer pilot of the GeniConnect program last year in 2017. We are collecting data using a retrospective survey designed by The PEAR Institute, specifically identifying interest in genetics (more broadly) and STEM careers. Our hope is that students will notice the characters in the game and their role in the “underground lab” in Geniventure. However, the connection to STEM careers in the GeniConnect program cannot solely be attributed to the characters in the game alone. The GeniConnect project couples Geniventure gameplay with engaging activities and experiences such as field trips to biotech labs, visits from volunteer scientists, and hands-on laboratory experiments.

    We have another implementation of the GeniConnect program scheduled to take place this summer with a local group of middle school students. We are eager to find out if our suite of activities yields the type of interest in STEM careers that we are anticipating!

  • Icon for: Eli Meir

    Eli Meir

    Researcher
    May 16, 2018 | 10:15 p.m.

    This looks like a really great tool. Could you say a little more about how you are implementing Baseyian knowledge tracing within the game context to deduce where students are having trouble? How much human thought goes into the rules for deducing particular confusions, vs. how much is scoring different student traces and then using machine learning? Did you design Geniventure in some ways to make it easier to deduce student confusions, and if so, can you say a little about that design?

    Trying figure out where students are confused is something we are addressing in our project as well, and I'm very curious to hear more about how you manage to do it in your complicated environment. Thanks.

  • Small default profile

    Robert Taylor

    Researcher
    May 18, 2018 | 10:41 a.m.

    Hi Eli,

    I'm a computer scientist on Geniventure's Intelligent Tutoring System (ITS) project. We are using a human-authored rule system to evaluate the student's move/answer in each step of the problem. The binary result (correct/incorrect) is then evaluated using a BKT-based tutor to calculate the probability that the student knows the skill/concept. The ITS then uses this probability (compared against a threshold) to decide whether to present a hint or remediate for the skill associated with the step.

    We don't specifically attempt to deduce student confusions. We simply evaluate whether the student's answer was correct or incorrect and use that to update the probability that the student knows the skill. The BKT algorithm has parameters to account for 'slips' (making a mistake even though they understand the concept) or 'guesses' (the student got it right even though they don't understand the concept).

    Regards,

    -Robert

     
    2
    Discussion is closed. Upvoting is no longer available

    Kiley McElroy-Brown
    Trudi Lord
  • Icon for: Trudi Lord

    Trudi Lord

    Co-Presenter
    May 18, 2018 | 11:09 a.m.

    Hi Eli,

    To answer your question about how much human thought went into the design of this system, the answer is... LOTS! In designing Geniventure, we specifically created the challenges such that student performance could be machine evaluated. Even though not all of our learning goals can be easily evaluated by the ITS, there is a definite subset of student moves that can help determine conceptual understanding at both low and high levels.

    Also note, Geniventure is the latest incarnation of The Concord Consortium's dragon genetics software. We have observed many students, over many years, use prior versions of dragon genetics (Genscope, Biologica, Geniverse, GeniVille, GeniGames). We have a pretty good idea where misconceptions arise and teachers do, too! (Sadly, we cannot be in every classroom and teachers cannot stand behind every student in a room of 25.) Our expert knowledge, along with our EDC-based concepts, helped drive the preliminary development of the ITS rules. Now, we have a larger set of student log data specific to Geniventure and are using BKT and other tools to reveal patterns in student use that will help inform improvements to the system.

    I hope this answer helps you! Please reach out if you would like more information on our design process.

    Thanks for your question!
    Trudi ~:)

  • Icon for: Eli Meir

    Eli Meir

    Researcher
    May 18, 2018 | 04:16 p.m.

    Thanks, that's interesting info. It makes sense to use a human-authored rule system (that's what we have generally done as well), I was just wondering (hoping?) that perhaps you had some algorithmic magic you sprinkled onto your project...

    Feeding that into the BKT is a nice idea that we should think about more too.

    Trudi, do you have an example of a design decision you made specifically to help you evaluate student performance? Or a paper on that design process? (I'm specifically interested in this for a conceptual paper I am mulling in my head about constraint as a tool that can be used both to enable categorization, but also to facilitate the proper amount of scaffolding for each student - depending how you think about your designs for assessment, might be interesting to talk).

  • Icon for: Trudi Lord

    Trudi Lord

    Co-Presenter
    May 21, 2018 | 09:22 a.m.

     Hi Eli,

    I'd love to chat about this! Send me email and we'll set something up! (tlord@concord.org)

    THANKS!!!
    Trudi ~:)

  • Small default profile

    Amy Berger

    K-12 Teacher
    May 21, 2018 | 06:25 p.m.

     

    I am amazed by this program and how it takes on a cool twist to genetics.  I think that the use of a diverse selection of scientists is so important for our students to see.  It is all too common, especially in our science textbook, for the white males to be highlighted for their inputs to science leaving teachers to need to supplement the text to highlight so many more scientist that are not white or male to connect to our very diverse culture of students.  I think this game like approach to genetics is engaging and helpful to students that need to manipulate concepts to best learn.  I think it is so cool to that there is real time assessments that allows the teacher to constantly monitor each students’ progress through the game.  I also really like that the program provides tutoring and hints to help students to be able to work through the problems.  While watching this video I couldn’t wait to check out the program and see how I could implement this activity into my curriculum for next year but was just wondering has it been aligned to Next Generation Science Standards (NGSS) or any state standards?  

  • Further posting is closed as the showcase has ended.