Hi Katie! 

Thank you for your interest in our project!

I can speak to the art making component of our work. In addition to coding classes, students participated in a visual arts class as part of the project. During the class we focused on how to tell stories of struggle and success through making art. Students made various pieces–--abstract watercolor paintings, narrative story panels, data visualization postcards, code poems (I can speak more about these individually if you are interested)–––and as they made these different kinds of art, they focused on different elements of stories. For example, the abstract watercolor paintings supported students in working through and attending through specific emotional experiences as they related to their coding experiences. Although I can't embed the physical artwork here, one student's explanation of her abstract work is particularly stirring: "This stem, I guess the blue one is fear, and that's why it's in a corner, it's in its own corner, even though it's slowly branching off, it still remains in this corner because it's like scared to integrate with this, and so the gold is creativity and wanting to do better ideas and the black is just attacking it so they're fighting for dominance. While this [gold] one blossoms beautiful ideas and concepts this [black one] is kind of like negative aspect, the limitations and everything. And so these two [gold and black] are kind of connected, but they are different in their own way. Fear is like not being able, fear is like the fear of, like so this [black] one is like the fear of failing and like no, this [blue one] is fear of failing and this [black one] is kind of just into action so it's when this actually branches into this and then that's when it attacks."

As a second example, in the code poem students printed out in-progress code they had been working with and wrote their own poems (often using hashtags) to attend to a very specific experience in debugging. 

As we look to redesign for our work this summer we are interested in creating more authentic occasions for storytelling throughout the camp so students have more opportunities to share stories in small and larger groups throughout rather than just in researcher interviews at the end of camp. 

Let me know if you have any questions!  

...and it is worth noting that the narrative story panel project encouraged students to use a more traditional story structure (beginning, middle, end) to retell their failure experiences.

Hi Katie,

 

As Maggie noted, the art projects have inspired rich, varied, and personal stories about failure that beg for more in-classroom, collaborative reflection. We are just scratching the surface of the potential of art to support creative reflection on failure and success. 

 

Outside of the art space, students also generate micro stories about success and failure at an insatiable rate during coding. These are often brief, co-narrated accounts that nonetheless follow some story structure: a sequence of events that captures something about the internal lifeworld of the student during some problem. In terms of when these stories occur, we are noticing that students announce, dramatize, and propose causes of their bugs when they have the capacity to fix them and when the bug creates an occasion for a joke. The content of these stories varies widely. In their coding journals, students called out a sprawling number of debugging strategies: relying on themselves, asking peers, working with instructors, managing emotion, trying new things, working hard, experimenting, playing, focusing, observing code, comparing to working versions of code, stepping through code, using memory, experimenting with code, implementing creativity, thinking, finding the bug, preparing in advance, making mistakes, and having faith. During coding, students explain undesired outcomes and syntax errors by characterizing their own actions (“I’m doing something so dumb”) and epistemic states (“I don't even know what's wrong”) alongside attributions in multiple external spaces: the computer code (“So you only need like one function name”), the robot they are coding (“Toucan has failed me”), the computer (“The computer’s glitching”), and instructors (“[Your idea] doesn’t work, Bethany!”).

 

We still have some way to go with this analysis. What IS clear is that storytelling during coding is a somewhat constrained space because students are motivated if not a bit urgent to get the bug fixed -- the stories students tell about how they feel or what they think is causing the bug must fit into that context. The art space, on the other hand, provides a slower route, and one in which "wrong" art is even encouraged. Letting students surface stories in both spaces, listening to what they say, and deciding with students how to support them to approach failure in new ways is what we are after.

Thanks, Maggie and David, for sharing such interesting examples! I love the way kids are talking about their failures and working through their feelings about it. I think your point about the need for both storytelling during debugging and in slower spaces is important. I also find the discussion of their internal and external attributions very interesting. Inspiring work.

Hi, Irene. Thanks for your question about how objects can be used to model code. Inspired by Charles Goodwin, one of our strands of analysis involves looking at the fine details of how instructors structure opportunities for students to interact with their code and its output in consequential ways in order to engage students in authentic debugging search strategies and tactics. One way we’ve found instructors do this is by actually taking up physical objects and/or using their bodies to act out the program output at each line and asking the student to evaluate each action as they go. By doing this, students and instructors are co-constructing a debugging strategy sometimes known as ‘hand simulation’ where a programmer reviews code line by line, imagining the expected and/or desired outcome for each statement and then comparing it with the actual outcome. While there’s been a lot of work on the benefits of having students act out their own code line by line with objects and/or their bodies, we know less about how instructors and students can co-construct this process together. We’re trying to better understand what it consists of when they do it together and what affordances it may have for students’ learning debugging. We'll be presenting some of this work this summer at the International Conference of the Learning Sciences in London. 

Agree!  On both the point of appreciation and the question about emotional, cognitive, and social aspects of debugging.

I agree too, I think there is definitely a significant component of this that has to do with the emotional quality of students' experiences. I have done some research about students' emotional reactions to programming assignments and found that experiences like frustration can have a long range effect on students' outcomes in a programming class, and I think that a large part of the frustration that can surround doing programming projects comes into play when debugging. So that all being said I'm very interested to hear how you might think about the emotional character of student experiences debugging, or how you might be looking at it as a part of the process of developing debugging skills, particularly for a younger group of students.

Debbie, Eric, and Alex --

 

Thanks for your support and for your thoughtful comments. Alex, I'm wondering if you could share the reference to the paper you mentioned above. It sounds very relevant. Debbie and Eric, I'm also curious to hear how you have handled research around these three factors. 

 

First some brief comments on the design of the workshops. One of the goals of our coding workshops is to give students the opportunity to make visible their emotional experiences with coding. We've had large, collaboratively constructed posters where students write emotion-laded hashtags to express how they feel about certain coding experiences. In bringing these affective experiences out of the shadows, we ended up with a representation of the emotional pulse of the classroom. Through their individual artwork, students also create abstract drawings of emotion and they create code poems that present rich detail about emotion. The artwork is then shared during a gallery walk. Moving forward, we are intentionally designing for even more occasions for small-group storytelling about emotion and for teachers to better understand what their students are experiencing emotionally when debugging as a way to inform their pedagogy. Maggie, would you have anything to add here? 

 

Analyzing this data is of course a much taller order. Debbie, I appreciate your nudge to look at emotional, cognitive, and social aspects at once. I can share two analysis threads where we are (sort of) trying this. The first looks at peer-to-peer conversations about refactoring. When refactoring, students take code that works and modify it to make it even better (e.g., fewer lines of code, more elegant abstractions, more communicable code). We are finding that peer conversations surround the choice to refactor. Students have to grapple with whether it's even worth it to refactor, and they weigh the costs and benefits publicly. That's the social component. Because the refactoring experience is then already somewhat collaborative, when students do encounter a bug, they have reason to make their emotional reaction public. Students then comment on each other's affective responses to encountering these bugs. We haven't yet considered the cognitive element of refactoring. 

 

Another context where we're looking at these factors is students' data visualizations about their coding experiences, inspired by the Dear Data project. Here, students pay attention to factors about their coding experience that they find interesting (e.g., what kinds of bugs they hit, how they react emotionally, who they talk with, etc.). The analysis of this data is tractable. We can look at what types of emotions students track, what kinds of debugging strategies they report using, and what kinds of social interactions they notice. Looking at the data challenged our assumptions about what students experienced when debugging. So, our plan for the coming summer is to use these data visualizations as the backbone project of our art space. Students can surface patterns about their experiences with coding, share them, tell stories about specific data points, and hopefully then decide with their peers and the teacher how they want to move their debugging practice forward.

Nice summary, David, and great questions, Debbie, Eric, and Alex! 

One note of interest I would add about the design of the arts component of the learning space:

Related to the emotional dimension of experience, during instruction students were encouraged to tap back into experiences physically and think about/channel how that experience felt in their bodies while they were making their abstract works of art. This prompting inspired students to do things like "slap" a paintbrush on the watercolor paper or crumble the paper itself to show frustration. Additional examples include students using short, quick strokes with oil pastels to show generally positive/happy feelings or splattering the paint to show excitement. It was interesting to push students to explore how to represent emotions using different elements of art like color, line, and shape. This led students to ask questions like: What kinds of lines would best represent the emotions I'm trying to express? What mix of colors can most precisely represent a feeling? Why did I choose the elements I did? How did the materials guide the creation of my artwork? We have a lot more to learn about how this arts space translates to their coding practice, but so far we see tons of possibilities especially relevant to exploring the emotional dimension of experience.

David, I'm happy to share my paper:

Lishinski, A., Yadav, A., & Enbody, R. (2017, August). Students' Emotional Reactions to Programming Projects in Introduction to Programming: Measurement Approach and Influence on Learning Outcomes. In Proceedings of the 2017 ACM Conference on International Computing Education Research (pp. 30-38). ACM.

https://dl.acm.org/citation.cfm?id=3106187

Charles, that's an interesting angle. In some science contexts, the iteration cycles are quite long. It may take a few weeks, months, even years/decades to find out if a hypothesis has panned out. In contrast, bugs in code arise constantly, and though some particularly thorny ones can take days or weeks to debug, that's more of the exception. (I'm thinking of this recent study.) We picked coding as a context for this work in part because of these rapid iteration cycles, the numerous tools available for debugging, and the engrained expectation that bugs are coming whether you're a novice or expert. There are of course also tradeoffs -- it's an emotional grind to cope with that many bugs all the time, especially for a newcomer. I wonder if scientific model-based reasoning would provide a good balance between collecting actual data and iterating quickly with a simulation of that data. I also wonder how often students experience failed inquiries in science, and whether the message that it's okay for a science experiment to fail feels authentic to students.

Emily, this is such a good question. We've wondered about this as well. Our latest thought is that we need a balance between actually getting problems fixed, fostering deep learning about the subject matter, and teaching students strategies for dealing independently with these problems. Different types of pedagogy are required for each, and each requires different time investments. What we suggest to our instructors is that they find a good balance each day for each student. That is, you can consider taking a quick route to fixing a bug, but then for the next bug, prioritize the student demonstrating that they understand the coding concept in play. Later, you may invite the student to try a new debugging strategy and return even later in the class period to see if they can implement that debugging strategy again. (The order of these could vary.) At the end of the day, instructors would know that student were not unreasonably stranded all class with bugs, they would know that the student learned something about a new coding concept, and they would also know that the student explored a new debugging strategy (and likely experienced a bit of the frustration you mention above). This strikes a balance between ensuring that students make some progress on their projects, learn foundational CS concepts, and start to develop a practice of handling bugs on their own.

Chad, thanks for your thoughtful comments. The Rob Ross quote is a gem -- I hadn't see it before. And how cool that your son wears this ethos on his sleeve!

Your nudge for us to think carefully about interest is a good one. The students at our workshops elect to spend their weekends/summers coding, but we also know they fall in love with some projects more than others, and I'm sure there's a gradation of interest even within the well-received projects. We'll have to look closely at Renninger & Hidi (2016) and think about how to incorporate this angle.

In order to free up more time for teacher-students conversations during debugging, and to make sure students' bugs are in their ZPDs, we've moved toward self-paced learning. Providing more room for students to pick projects that interest them could complement this approach. We're struck by the designs coming from Northwestern's FUSE project -- perhaps this could offer a template. 

Thanks again for your comment. Also wanted to say that I appreciated your CIRCL talk about pedagogical agents a few months back.

Hi Michelle, thanks for your comment. We have a range of students in our weekend/summer coding workshops. About 1/3rd of the students have never coded, and the other 2/3rds have some experience with coding. We use differentiated curriculum that gives students the option to self-select their level of difficulty on any given day. We are also working on building a strong peer mentorship culture. But what you suggest is no doubt right -- teaching debugging to newcomers is very different from teaching debugging to students who have some coding experience under their belt.

Thanks so much, Diane. Glad to hear that some of our designs might translate to your classroom context. We all really admire how openly and proactively professional coders have approached debugging in their practice. I would be hard-pressed to find another discipline that has designed this many tools, online communities, and processes for dealing just with failure.