1. David DeLiema
  2. Postdoctoral Researcher
  3. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  4. University of California Berkeley
  1. Dor Abrahamson
  2. http://edrl.berkeley.edu/content/dor-abrahamson
  3. Associate Professor
  4. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  5. University of California Berkeley
  1. Melissa Chen
  2. Lead LA Program Director
  3. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  4. 9 Dots
  1. Maggie Dahn
  2. PhD Candidate
  3. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  4. UCLA
  1. Noel Enyedy
  2. http://sttep.org
  3. Professor
  4. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  5. Vanderbilt University
  1. Virginia Flood
  2. https://edrl.berkeley.edu/content/virginia-j-flood
  3. PhD Candidate
  4. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  5. University of California Berkeley
  1. Leiny Garcia
  2. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  3. 9 Dots
  1. Francis Steen
  2. http://commstudies.ucla.edu/content/francis-steen-phd
  3. Associate Professor
  4. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  5. UCLA
  1. Josh Taylor
  2. Debugging Failure: Fostering Youth Academic Resilience in Computer Science
  3. 9 Dots
Facilitators’
Choice
Public Discussion
  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 13, 2018 | 08:25 p.m.

    Thanks for checking out the Debugging Failure project! We would love to hear your thoughts.

     

    Some topics we might discuss: What ideas about failure cross your mind after watching this video? What parts of our approach to teaching debugging could you imagine using in your classroom, including outside of computer science? What questions do you have about our classroom designs and our research methods? Where you would like to see the project go?

  • Icon for: Katie Rich

    Katie Rich

    Facilitator
    May 14, 2018 | 09:54 a.m.

    Hi David and all,

    I love the focus of this project on building a culture around learning from failure! Can you say more about the aspects of storytelling in your project? What kinds of stories do kids tell about their failures, and how have you seen those stories change as they participate in the program?

     
    1
    Discussion is closed. Upvoting is no longer available

    David DeLiema
  • Icon for: Maggie Dahn

    Maggie Dahn

    Co-Presenter
    May 14, 2018 | 10:58 a.m.

    Hi Katie! 

    Thank you for your interest in our project!

    I can speak to the art making component of our work. In addition to coding classes, students participated in a visual arts class as part of the project. During the class we focused on how to tell stories of struggle and success through making art. Students made various pieces–--abstract watercolor paintings, narrative story panels, data visualization postcards, code poems (I can speak more about these individually if you are interested)–––and as they made these different kinds of art, they focused on different elements of stories. For example, the abstract watercolor paintings supported students in working through and attending through specific emotional experiences as they related to their coding experiences. Although I can't embed the physical artwork here, one student's explanation of her abstract work is particularly stirring: "This stem, I guess the blue one is fear, and that's why it's in a corner, it's in its own corner, even though it's slowly branching off, it still remains in this corner because it's like scared to integrate with this, and so the gold is creativity and wanting to do better ideas and the black is just attacking it so they're fighting for dominance. While this [gold] one blossoms beautiful ideas and concepts this [black one] is kind of like negative aspect, the limitations and everything. And so these two [gold and black] are kind of connected, but they are different in their own way. Fear is like not being able, fear is like the fear of, like so this [black] one is like the fear of failing and like no, this [blue one] is fear of failing and this [black one] is kind of just into action so it's when this actually branches into this and then that's when it attacks."

    As a second example, in the code poem students printed out in-progress code they had been working with and wrote their own poems (often using hashtags) to attend to a very specific experience in debugging. 

    As we look to redesign for our work this summer we are interested in creating more authentic occasions for storytelling throughout the camp so students have more opportunities to share stories in small and larger groups throughout rather than just in researcher interviews at the end of camp. 

    Let me know if you have any questions!  

     
    2
    Discussion is closed. Upvoting is no longer available

    Katie Rich
    David DeLiema
  • Icon for: Maggie Dahn

    Maggie Dahn

    Co-Presenter
    May 14, 2018 | 11:01 a.m.

    ...and it is worth noting that the narrative story panel project encouraged students to use a more traditional story structure (beginning, middle, end) to retell their failure experiences.

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 14, 2018 | 12:50 p.m.

    Hi Katie,

     

    As Maggie noted, the art projects have inspired rich, varied, and personal stories about failure that beg for more in-classroom, collaborative reflection. We are just scratching the surface of the potential of art to support creative reflection on failure and success. 

     

    Outside of the art space, students also generate micro stories about success and failure at an insatiable rate during coding. These are often brief, co-narrated accounts that nonetheless follow some story structure: a sequence of events that captures something about the internal lifeworld of the student during some problem. In terms of when these stories occur, we are noticing that students announce, dramatize, and propose causes of their bugs when they have the capacity to fix them and when the bug creates an occasion for a joke. The content of these stories varies widely. In their coding journals, students called out a sprawling number of debugging strategies: relying on themselves, asking peers, working with instructors, managing emotion, trying new things, working hard, experimenting, playing, focusing, observing code, comparing to working versions of code, stepping through code, using memory, experimenting with code, implementing creativity, thinking, finding the bug, preparing in advance, making mistakes, and having faith. During coding, students explain undesired outcomes and syntax errors by characterizing their own actions (“I’m doing something so dumb”) and epistemic states (“I don't even know what's wrong”) alongside attributions in multiple external spaces: the computer code (“So you only need like one function name”), the robot they are coding (“Toucan has failed me”), the computer (“The computer’s glitching”), and instructors (“[Your idea] doesn’t work, Bethany!”).

     

    We still have some way to go with this analysis. What IS clear is that storytelling during coding is a somewhat constrained space because students are motivated if not a bit urgent to get the bug fixed -- the stories students tell about how they feel or what they think is causing the bug must fit into that context. The art space, on the other hand, provides a slower route, and one in which "wrong" art is even encouraged. Letting students surface stories in both spaces, listening to what they say, and deciding with students how to support them to approach failure in new ways is what we are after.

     
    1
    Discussion is closed. Upvoting is no longer available

    Katie Rich
  • Icon for: Katie Rich

    Katie Rich

    Facilitator
    May 17, 2018 | 02:44 p.m.

    Thanks, Maggie and David, for sharing such interesting examples! I love the way kids are talking about their failures and working through their feelings about it. I think your point about the need for both storytelling during debugging and in slower spaces is important. I also find the discussion of their internal and external attributions very interesting. Inspiring work.

  • Icon for: Irene Lee

    Irene Lee

    Facilitator
    May 14, 2018 | 10:00 a.m.

    Thanks for sharing your work.  I am struck by the focus on fixing bugs rather than on designing to avoiding certain bugs. Are they one and the same?  To what extent do you integrate the two?

    Also, I'd love to hear more about objects for modeling code.  What can you tell me about that?

    Finally, are you able to follow students from clubs into the classroom to see the impact of their experiences in debugging as it relates to other content / process learning?

     
    1
    Discussion is closed. Upvoting is no longer available

    David DeLiema
  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 14, 2018 | 11:55 a.m.

    Irene, thanks for your interest in the project!

     

    We've thought a lot about your first question. Our current perspective is that students need a good balance between the two. We would consider our coding workshops successful if bugs get fixed; students over time avoid specific (recurring) bugs; students encounter new but tractable (for them) bugs; students, peers, and instructors use debugging strategies strategically; and students drive the debugging process. We see these as characteristics of a generative approach to debugging. I love your question because it points out implicit assumptions we have about the productivity of failure. Seeing a student encounter the same bug again and again over several days without capacity to fix it suggests that the student needs to learn new debugging strategies. On the flip side, seeing a student code smoothly over a few days without running into any bugs whatsoever suggests that the student might be ready for a new coding challenge. It is important for learning communities to empower students to avoid recurring problems in learning while also having capacity to handle new problems. We presented a poster on this topic at AERA 2018 that gives a bit more detail if you're interested. 

     

    For your second question, I would like to hand you off to Virginia Flood. Virginia is working on a detailed analysis of an instructor using a cube to help a student debug. I'll ping her offline and see if she can join the thread. 

     

    Your third question is something we are not currently pursuing in this grant, but we have seen some early indicators in interviews with students that some kind of transfer to other settings might be happening. This is all anecdotal so far, but we would absolutely like to pursue this topic down the road. 

  • Icon for: Virginia Flood

    Virginia Flood

    Co-Presenter
    May 17, 2018 | 10:46 p.m.

    Hi, Irene. Thanks for your question about how objects can be used to model code. Inspired by Charles Goodwin, one of our strands of analysis involves looking at the fine details of how instructors structure opportunities for students to interact with their code and its output in consequential ways in order to engage students in authentic debugging search strategies and tactics. One way we’ve found instructors do this is by actually taking up physical objects and/or using their bodies to act out the program output at each line and asking the student to evaluate each action as they go. By doing this, students and instructors are co-constructing a debugging strategy sometimes known as ‘hand simulation’ where a programmer reviews code line by line, imagining the expected and/or desired outcome for each statement and then comparing it with the actual outcome. While there’s been a lot of work on the benefits of having students act out their own code line by line with objects and/or their bodies, we know less about how instructors and students can co-construct this process together. We’re trying to better understand what it consists of when they do it together and what affordances it may have for students’ learning debugging. We'll be presenting some of this work this summer at the International Conference of the Learning Sciences in London. 

  • May 14, 2018 | 02:11 p.m.

    Wonderful topic. I appreciate in particular how you are looking at creating a culture around debugging. 

    How are you looking at the emotional and cognitive and social aspects of debugging? That sounds like quite a challenge.

     
    1
    Discussion is closed. Upvoting is no longer available

    Eric Hamilton
  • May 14, 2018 | 02:44 p.m.

    Agree!  On both the point of appreciation and the question about emotional, cognitive, and social aspects of debugging.

  • Icon for: Alex Lishinski

    Alex Lishinski

    Facilitator
    May 15, 2018 | 09:31 a.m.

    I agree too, I think there is definitely a significant component of this that has to do with the emotional quality of students' experiences. I have done some research about students' emotional reactions to programming assignments and found that experiences like frustration can have a long range effect on students' outcomes in a programming class, and I think that a large part of the frustration that can surround doing programming projects comes into play when debugging. So that all being said I'm very interested to hear how you might think about the emotional character of student experiences debugging, or how you might be looking at it as a part of the process of developing debugging skills, particularly for a younger group of students.

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 15, 2018 | 12:53 p.m.

    Debbie, Eric, and Alex --

     

    Thanks for your support and for your thoughtful comments. Alex, I'm wondering if you could share the reference to the paper you mentioned above. It sounds very relevant. Debbie and Eric, I'm also curious to hear how you have handled research around these three factors. 

     

    First some brief comments on the design of the workshops. One of the goals of our coding workshops is to give students the opportunity to make visible their emotional experiences with coding. We've had large, collaboratively constructed posters where students write emotion-laded hashtags to express how they feel about certain coding experiences. In bringing these affective experiences out of the shadows, we ended up with a representation of the emotional pulse of the classroom. Through their individual artwork, students also create abstract drawings of emotion and they create code poems that present rich detail about emotion. The artwork is then shared during a gallery walk. Moving forward, we are intentionally designing for even more occasions for small-group storytelling about emotion and for teachers to better understand what their students are experiencing emotionally when debugging as a way to inform their pedagogy. Maggie, would you have anything to add here? 

     

    Analyzing this data is of course a much taller order. Debbie, I appreciate your nudge to look at emotional, cognitive, and social aspects at once. I can share two analysis threads where we are (sort of) trying this. The first looks at peer-to-peer conversations about refactoring. When refactoring, students take code that works and modify it to make it even better (e.g., fewer lines of code, more elegant abstractions, more communicable code). We are finding that peer conversations surround the choice to refactor. Students have to grapple with whether it's even worth it to refactor, and they weigh the costs and benefits publicly. That's the social component. Because the refactoring experience is then already somewhat collaborative, when students do encounter a bug, they have reason to make their emotional reaction public. Students then comment on each other's affective responses to encountering these bugs. We haven't yet considered the cognitive element of refactoring. 

     

    Another context where we're looking at these factors is students' data visualizations about their coding experiences, inspired by the Dear Data project. Here, students pay attention to factors about their coding experience that they find interesting (e.g., what kinds of bugs they hit, how they react emotionally, who they talk with, etc.). The analysis of this data is tractable. We can look at what types of emotions students track, what kinds of debugging strategies they report using, and what kinds of social interactions they notice. Looking at the data challenged our assumptions about what students experienced when debugging. So, our plan for the coming summer is to use these data visualizations as the backbone project of our art space. Students can surface patterns about their experiences with coding, share them, tell stories about specific data points, and hopefully then decide with their peers and the teacher how they want to move their debugging practice forward.

  • Icon for: Maggie Dahn

    Maggie Dahn

    Co-Presenter
    May 15, 2018 | 06:32 p.m.

    Nice summary, David, and great questions, Debbie, Eric, and Alex! 

    One note of interest I would add about the design of the arts component of the learning space:

    Related to the emotional dimension of experience, during instruction students were encouraged to tap back into experiences physically and think about/channel how that experience felt in their bodies while they were making their abstract works of art. This prompting inspired students to do things like "slap" a paintbrush on the watercolor paper or crumble the paper itself to show frustration. Additional examples include students using short, quick strokes with oil pastels to show generally positive/happy feelings or splattering the paint to show excitement. It was interesting to push students to explore how to represent emotions using different elements of art like color, line, and shape. This led students to ask questions like: What kinds of lines would best represent the emotions I'm trying to express? What mix of colors can most precisely represent a feeling? Why did I choose the elements I did? How did the materials guide the creation of my artwork? We have a lot more to learn about how this arts space translates to their coding practice, but so far we see tons of possibilities especially relevant to exploring the emotional dimension of experience.

  • Icon for: Alex Lishinski

    Alex Lishinski

    Facilitator
    May 18, 2018 | 11:44 a.m.

    David, I'm happy to share my paper:

    Lishinski, A., Yadav, A., & Enbody, R. (2017, August). Students' Emotional Reactions to Programming Projects in Introduction to Programming: Measurement Approach and Influence on Learning Outcomes. In Proceedings of the 2017 ACM Conference on International Computing Education Research (pp. 30-38). ACM.

    https://dl.acm.org/citation.cfm?id=3106187

     
    2
    Discussion is closed. Upvoting is no longer available

    Diane Smoot, Ph.D.
    David DeLiema
  • May 15, 2018 | 08:56 a.m.

    It'll be interesting to see how these workshops cultivate a debugging culture; i'd love to see whether it's a model that works well for the iterative aspect of doing science in general, i.e. that it's a looping, adaptive process (rather than linear) where "not getting it right" is actually part of doing science. How to build positivity behind "negative" results, and how to make those useful foundations to continue building on to understand a topic. Thanks for sharing!

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 15, 2018 | 01:03 p.m.

    Charles, that's an interesting angle. In some science contexts, the iteration cycles are quite long. It may take a few weeks, months, even years/decades to find out if a hypothesis has panned out. In contrast, bugs in code arise constantly, and though some particularly thorny ones can take days or weeks to debug, that's more of the exception. (I'm thinking of this recent study.) We picked coding as a context for this work in part because of these rapid iteration cycles, the numerous tools available for debugging, and the engrained expectation that bugs are coming whether you're a novice or expert. There are of course also tradeoffs -- it's an emotional grind to cope with that many bugs all the time, especially for a newcomer. I wonder if scientific model-based reasoning would provide a good balance between collecting actual data and iterating quickly with a simulation of that data. I also wonder how often students experience failed inquiries in science, and whether the message that it's okay for a science experiment to fail feels authentic to students.

  • Icon for: Emily Hestness

    Emily Hestness

    Researcher
    May 15, 2018 | 12:02 p.m.

    Fascinating topic - really like the idea of creating a debugging culture in the classroom. In our work with preservice teachers and practicing teachers around computational thinking, the idea that students might get frustrated and give up often emerges as a concern for our participating teachers. Yet they clearly recognize the opportunity that this kind of thinking presents for developing students' persistence when facing challenges. Anything you can share about how what you're learning may inform teacher education and professional development (i.e., how can we support teachers in learning to support students as they work through frustrating/challenging problems)?

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 15, 2018 | 01:21 p.m.

    Emily, this is such a good question. We've wondered about this as well. Our latest thought is that we need a balance between actually getting problems fixed, fostering deep learning about the subject matter, and teaching students strategies for dealing independently with these problems. Different types of pedagogy are required for each, and each requires different time investments. What we suggest to our instructors is that they find a good balance each day for each student. That is, you can consider taking a quick route to fixing a bug, but then for the next bug, prioritize the student demonstrating that they understand the coding concept in play. Later, you may invite the student to try a new debugging strategy and return even later in the class period to see if they can implement that debugging strategy again. (The order of these could vary.) At the end of the day, instructors would know that student were not unreasonably stranded all class with bugs, they would know that the student learned something about a new coding concept, and they would also know that the student explored a new debugging strategy (and likely experienced a bit of the frustration you mention above). This strikes a balance between ensuring that students make some progress on their projects, learn foundational CS concepts, and start to develop a practice of handling bugs on their own.

  • Icon for: H Chad Lane

    H Chad Lane

    Higher Ed Faculty
    May 18, 2018 | 12:32 p.m.

    Hi, I loved the video and the approach you are taking! You're clearly presenting coding as a rich and exciting activity, and positioning debugging as something that is simply part of that richness. Honestly, I'm not sure computer science would be as meaningful if we didn't ever have to debug and overcome challenges to create what we want. I am also reminded of Bob Ross' approach to this problem - you probably know it well, but he once said "We don't make mistakes, just happy little accidents" (my son also has this on a t-shirt).

    Anyway, I am wondering if you have looked into the literature on interest development?  The title of your project mentions resilience, and I haven't seen discussion of it yet.... Renninger and Hidi (2016) write a lot about the power of promoting interest, and how as interest strengthens (from situational to individual), so does resilience, and willingness to persist through challenge and negative emotions. It's clear you have a strong theoretical framing already for the project (I paused the video on your conjecture maps - awesome!) - there may be helpful ideas in the interest literature in terms of promoting resilience and persistence.  Thanks!

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 18, 2018 | 12:50 p.m.

    Chad, thanks for your thoughtful comments. The Rob Ross quote is a gem -- I hadn't see it before. And how cool that your son wears this ethos on his sleeve!

    Your nudge for us to think carefully about interest is a good one. The students at our workshops elect to spend their weekends/summers coding, but we also know they fall in love with some projects more than others, and I'm sure there's a gradation of interest even within the well-received projects. We'll have to look closely at Renninger & Hidi (2016) and think about how to incorporate this angle.

    In order to free up more time for teacher-students conversations during debugging, and to make sure students' bugs are in their ZPDs, we've moved toward self-paced learning. Providing more room for students to pick projects that interest them could complement this approach. We're struck by the designs coming from Northwestern's FUSE project -- perhaps this could offer a template. 

    Thanks again for your comment. Also wanted to say that I appreciated your CIRCL talk about pedagogical agents a few months back.

     
    1
    Discussion is closed. Upvoting is no longer available

    H Chad Lane
  • Icon for: Maggie Dahn

    Maggie Dahn

    Co-Presenter
    May 18, 2018 | 01:12 p.m.

    Hi, Chad! I just wanted to quickly drop a note that I love that Bob Ross quote and actually considered citing it for a paper we just submitted. In the paper we talk about how students interacted with art materials in ways that allow them to have a sort of conversation with the materials to figure out the direction they want to take the work. (David, this quote shows up in an earlier draft :)-). I took it out because I couldn't find an appropriate reference! Maybe I can just use the video you shared, Chad - thank you! 

     
    1
    Discussion is closed. Upvoting is no longer available

    H Chad Lane
  • Icon for: Michelle Zhu

    Michelle Zhu

    Higher Ed Faculty
    May 19, 2018 | 08:52 p.m.

    I have to agree that learning debugging is so important for our students. I am happy that you emphasize the debugging capabilities at such an early stage of elementary and middle school. For students to participate in this project, I assume many of them already have good coding experience.  

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 21, 2018 | 12:09 p.m.

    Hi Michelle, thanks for your comment. We have a range of students in our weekend/summer coding workshops. About 1/3rd of the students have never coded, and the other 2/3rds have some experience with coding. We use differentiated curriculum that gives students the option to self-select their level of difficulty on any given day. We are also working on building a strong peer mentorship culture. But what you suggest is no doubt right -- teaching debugging to newcomers is very different from teaching debugging to students who have some coding experience under their belt.

  • May 21, 2018 | 05:33 p.m.

    Thank you for the selection of debugging books at the beginning. I anticipate that showing these to my college computing students will help them accept debugging as inevitable and necessary. I admire the way you help students express their emotional travel as they wrestle with episodic success and failure. I think grappling with this as a group must reduce impostor syndrome and promote retention. You have provided me with additional tools to use in my class.

  • Icon for: David DeLiema

    David DeLiema

    Lead Presenter
    May 21, 2018 | 06:14 p.m.

    Thanks so much, Diane. Glad to hear that some of our designs might translate to your classroom context. We all really admire how openly and proactively professional coders have approached debugging in their practice. I would be hard-pressed to find another discipline that has designed this many tools, online communities, and processes for dealing just with failure.

  • Further posting is closed as the showcase has ended.