Reading Group
This is the homepage of the ALT+CS CS Education Reading Group.
- When: Mondays at 2:15 PM (Spring 2024)
- Where: ALT+CS Lab (DUE 2208) and via Zoom
Schedule
Helpful Information
Spring 2024 Papers
4/29/2024 - Skripchuk et. al. (2023)
Analysis of Novices’ Web-Based Help-Seeking Behavior While Programming
Abstract
Web-based help-seeking – finding and utilizing websites to solve a problem – is a critical skill during programming in both professional and academic settings. However, little work has explored how students, especially novices, engage in web-based help-seeking during programming, or what strategies they use and barriers they face. This study begins to investigate these questions through analysis of students’ web-search behaviors during programming. We collected think-aloud, screen recording, and log data as students completed a challenging programming task. Students were encouraged to use the web for help when needed, as if in an internship. We then qualitatively analyzed the data to address three research questions: 1) What events motivate students to use web search? 2) What strategies do students employ to search for, select, and learn from web pages? 3) What barriers do students face in web search, and when do they arise? Our results suggest that that novices use a variety of web-search strategies – some quite unexpected – with varying degrees of success, suggesting that web search can be a challenging skill for novice programmers. We discuss how these results inform future research and pedagogy focused on how to support students in effective web search.
4/22/2024 - Sherad et. al. (2013)
Assessment of Programming: pedagogical foundations of exams
Abstract
Previous studies of assessment of programming via written examination have focused on analysis of the examination papers and the questions they contain. This paper reports the results of a study that investigated how these final exam papers are developed, how students are prepared for these exams, and what pedagogical foundations underlie the exams. The study involved interviews of 11 programming lecturers. From our analysis of the interviews, we find that most exams are based on existing formulas that are believed to work; that the lecturers tend to trust in the validity of their exams for summative assessment; and that while there is variation in the approaches taken to writing the exams, all of the exam writers take a fairly standard approach to preparing their students to sit the exam. We found little evidence of explicit references to learning theories or models, indicating that the process is based largely on intuition and experience.
4/15/2024 - Schanzer et. al. (2024)
Integrated Data Science for Secondary Schools: Design and Assessment of a Curriculum
Abstract
We propose that secondary-school data-science curricula should be based on four key ingredients: two are technical (programming and statistics, with visualization sitting at their intersection), while two are human-facing (meaningful domains, and civic responsibility). We describe their relationship and argue for their importance. Based on this, we then present the Bootstrap:Data Science curriculum, designed for integration into multiple disciplines and settings. It achieves this by (a) being designed as a set of remix-able lessons, and (b) letting classes and students choose personally meaningful datasets. We also initiate the process of evaluating this curriculum. We create two assessment instruments, one focused on learning and the other on personalization and engagement. We provide very preliminary data gathered from students and teachers.
4/8/2024 - Ojha, Vidushi and West, Leah and Lewis, Colleen M. (2024)
Computing Self-Efficacy in Undergraduate Students: A Multi-Institutional and Intersectional Analysis
Abstract
Computing self-efficacy is an important factor in shaping students’ motivation, performance, and persistence in computer science (CS) courses. Therefore, investigating computing self-efficacy may help to improve the persistence of students from historically underrepresented groups in computing. Previous research has shown that computing self-efficacy is positively correlated with prior computing experience, but negatively correlated with some demographic identities (e.g., identifying as a woman). However, existing research has not demonstrated these patterns on a large scale while controlling for confounding variables and institutional context. In addition, there is a need to study the experiences of students with multiple marginalized identities through the lens of intersectionality. Our goal is to investigate the relationship between students’ computing self-efficacy and their prior experience in computing, demographic identities, and institutional policies. We conduct this investigation using a large, recent, and multi-institutional dataset with survey responses from 31,425 students. Our findings confirm that more computing experience positively predicts computing self-efficacy. However, identifying as Asian, Black, Native, Hispanic, non-binary, and/or a woman were statistically significantly associated with lower computing self-efficacy. The results of our work point to several future avenues for self-efficacy research in computing.
3/4/2024 - Esche & Weihe (2023)
Choosing a Didactic Basis for an Instructional Video: What Are the Implications For Novice Programmers?
Abstract
Much work on instructional videos in computing education focuses on the overall impact and technical aspects of videos, such as motivation and length. However, it might be significant how the underlying pedagogical theory, the didactic basis, determines the delivery of the content. We conducted a randomized experiment to investigate the research question: How does the didactic basis of an instructional video affect code writing performance and self-efficacy given the basic skill of novice programmers? Our data included two cohorts of 133 and 428 CS1 students from the Fall semesters of 2021/22 and 2022/23, respectively. In cohort 1, videos based on language-sensitive teaching led to significantly better results in writing code in object orientation for novices with medium basic skills than videos based on worked examples. This result could not be replicated in cohort 2. We found no effect on novice self-efficacy in either cohort.
2/26/2024 - Kumar, Becker, Pias, et. al (2023)
A Combined Knowledge and Competency (CKC) Model for Computer Science Curricula
Abstract
All prior curricular guidelines for computer science have used a knowledge model, which consists of knowledge areas, knowledge units within the knowledge areas, and learning outcomes for the topics within those knowledge units. More recently, competency models have been explored for curricular guidelines. A competency model consists of competency specifications that list the knowledge, skills and dispositions needed to complete tasks. Both knowledge models and competency models have their benefits and shortcomings. We propose a model for computer science curricular guidelines that synergistically combines knowledge and competency models, in particular, the knowledge model last proposed in CS2013 [1] and the CoLeaf competency model last proposed in an ITiCSE working group report [8,11], both modified to facilitate integration. The combined model called CKC emphasizes both ends of the learning continuum and facilitates teaching as well as evaluation. It provides both an epistemological and teleological perspective of computer science content. We provide instructions for designing computer science curricula using the CKC model.
2/19/2024 - Chen & Ward (2023)
The Value of Time Extensions in Identifying Students Abilities
Abstract
Instructors often grant students extensions or grace days to relax deadline constraints. However, researchers have yet to investigate the value of time extensions in identifying students’ abilities and why students use them in computing education. Our study shows that scheduling conflicts and underestimation of the coursework were the top two reasons why students were late, providing the very first qualitative analysis results. By categorizing students who used and did not use cost-free time extensions, we found that students who used time extensions had a significantly lower assignment and exam grades than those who chose not to use them. We first observed this phenomenon when looking at grace day usages in a final-year programming course and validated the result by looking at the usage of extended lab time in a first-year programming course. This result suggests that offering a cost-free mechanism such as grace days or a time extension can provide a very early indicator of student abilities and those likely to need assistance.
2/5/2024 - Cutts, Kallia, Anderson, et. al (2023)
Arguments for and Approaches to Computing Education in Undergraduate Computer Science Programmes
Abstract
Computing education (CE), the scientific foundation of the teaching and learning of subject matter specific to computing, has matured into a field with its own research journals and conferences as well as graduate programmes. Yet, and unlike other mature subfields of computer science (CS), it is rarely taught as part of undergraduate CS programmes. In this report, we present a gap analysis resulting from semi-structured interviews with various types of stakeholders and derive a set of arguments for teaching CE courses in undergraduate CS programmes. This analysis and the arguments highlight a number of opportunities for the discipline of CS at large, in academia, in industry, and in school education, that would be opened up with undergraduate CE courses, as well as potential barriers to implementation that will need to be overcome. We also report on the results of a Delphi process performed to elicit top- ics for such a course with various audiences in mind. The Delphi process yielded 19 high-level categories that encompass the subject matter CE courses should incorporate, tailored to the specific needs of their intended student audiences. This outcome underscores the extensive range of content that can be integrated into a comprehensive CE programme. Based on these two stakeholder interactions as well as a systematic literature review aiming to explore the current practices in teaching CE to undergraduate students, we develop two prototypical outlines of such a course, keeping in mind that departments may have different preferences and affordances resulting in different kinds of CE offerings. Overall, input from external stakeholders underscores the clear significance of undergraduate CE courses. We anticipate leveraging this valuable feedback to actively promote these courses on a broader scale.
1/29/2024 - Albrecht & Grabowski (2020)
Sometimes It’s Just Sloppiness - Studying Students’ Programming Errors and Misconceptions
Abstract
Knowledge about students’ programming errors is a valuable source to get insights into students deficiencies and misconceptions. In this paper, we use data from an introductory C programming course to identify which errors are often made by students. Previous studies often focused only on syntactic and semantic errors as they can be easily identified by compilers. Studies focusing on logic errors were often restricted to a limited set of concepts or performed for a small set of data. We manually inspect 12371 submission by 280 students and have no restrictions regarding the error types we are looking for. We classify our found errors into six categories: syntactic, conceptual, strategic, sloppiness, misinterpretation, and domain knowledge. Our results show that a big portion of errors made by students is simply caused by sloppiness. But putting sloppiness aside, students seem to have most problems with strategic knowledge, i.e., the problem solving ability. We compare our results to previous studies and provide some implications of our results for future teaching practice.
1/22/2024 - Izu & Mirolo (2023)
Exploring CS1 Student’s Notions of Code Quality
Abstract
Coding tasks combined with other activities such as Explain in Plain English or Parson Puzzles help CS1 students to develop core programming skills. Students usually receive feedback of code correctness but limited or no feedback on their code quality. Teaching students to evaluate and improve the quality of their code once it is functionally correct should be included in the curricula towards the end of CS1 or during CS2. However, little is known about the student’s perceptions of code quality at the end of a CS1 course.
This study aims to capture their developing notions of code quality, in order to tailor class activities to support code quality improvements. We directed students to think about the overall quality of small programs by asking them to rank a small set of solutions for a simple problem solving task. Their rankings and explanations have been analysed to identify the criteria underlying their quality assessments. The top quality criteria were Performance (64%), Structure (51%), Conciseness (42%) and Comprehensibility (42%). Although fast execution is a key criteria for ranking, their explanations on why a given option was fast were often flawed, indicating students need more support both to evaluate performance and to include readability or comprehensibility criteria in their assessment.
Fall 2023 Papers
9/13/2023 - Engagement and Anonymity in Online Computer Science Course Forums
Abstract
Online discussion boards, designed to facilitate learning from peers and instructors in an accessible space, are a vital part of course design, especially in large scale computer science classes. Previous work has shown that women in computer science tend to use anonymity more often than men on these boards, a trend not found in humanities, social science or business courses. In this work, we build on these findings using an intersectional lens, analyzing both gender and race/ethnicity. We find this combined analysis reveals differences in anonymity that are not apparent when examining gender alone. For example, we find a significantly greater difference in anonymity use between Hispanic men and women than would be expected from analyzing race/ethnicity and gender independently. We additionally analyze type of content (e.g., questions, answers), course, platform, and data source to characterize the many factors at play in measuring students’ choice to participate anonymously. In doing so, we show that different approaches used in prior work for eliciting information on gender — whether using registrar data, a survey, or imputing gender based on name — changes how over of students are classified, particularly affecting nonbinary students and Asian students. Understanding when students participate anonymously can help educators and platform designers to make students’ experience of online discussion boards more welcoming.
9/6/2023 - Plagiarism Deterrence in CS1 Through Keystroke Data
Abstract
Recent work in computing education has explored the idea of analyzing and grading using the process of writing a computer program rather than just the final submitted code. We build on this idea by investigating the effect on plagiarism when the process of coding, in the form of keystroke logs, is submitted for grading in addition to the final code. We report results from two terms of a university CS1 course in which students submitted keystroke logs. We find that when students are required to submit a log of keystrokes together with their written code they are less likely to plagiarize. In this paper we explore issues of implementation, adoption, deterrence, anxiety, and privacy. Our keystroke logging software is available in the form of an IDE plugin in a public plugin repository.
8/30/2023 - Neo-Piagetian Theory as a Guide to Curriculum Analysis
Abstract
The development of a coherent curriculum, encapsulating appropriate topics, learning materials and assessment, is crucial for a successful educational experience. However, designing such a curriculum is a complicated task, with challenges in tracing the development of concepts across multiple courses and ensuring that assessment is at an appropriate level at specific points in the curricula. In this paper, we introduce a curriculum mapping framework based on Neo-Piagetian theory that assists lecturers in tracing concept development and assessment throughout their courses. This framework supports the identification of prerequisite concepts, where students are already assumed to be aware of specific topics, and assessment leaps, where students are assessed at a different conceptual level than they have been taught. We illustrate the application of our framework through a case study analysing the syllabus of a sequence of three first year programming courses.
8/4/2023 - Spencer et al. (2023)
Using Programming to Express Mathematical Ideas
Abstract
Integrating programming activities into core mathematics instruction can increase children’s access to critical content. Programming gives children a language with which to express, refine, and extend their thinking.
Summer 2023 Papers
8/4/2023 - YeckehZaare et al. (2022)
Another Victim of COVID-19: Computer Science Education
Abstract
Prior literature suggests that computer science education (CSE) was less affected by the pandemic than other disciplines. However, it is unclear how the pandemic affected the quality and quantity of students’ studying in CSE. We measure the impact of the pandemic on the amount and spacing of students’ studying in a large introductory computer science course. Spacing is defined as the distribution of studying over multiple sessions, which is shown to improve long-term learning. Using multiple regression models, we analyzed the total number of students’ interactions with the eBook and the number of days they used it, as a proxy for studying amount and spacing, respectively. We compared two sequential winter semesters of the course, one during (Winter 2021) and one prior to the pandemic (Winter 2020). After controlling for possible confounders, the results show that students had 1,345.87 fewer eBook interactions and distributed their studying on 2.36 fewer days during the pandemic when compared to the previous semester prior to the pandemic. We also compared four semesters prior to the pandemic (Fall and Winter of 2018 and 2019) to two semesters during the pandemic (Fall 2020 and Winter 2021). We found, on average, students had 3,376.30 fewer interactions with the eBook and studied the eBook on 16.35 fewer days during the pandemic. Contrary to prior studies, our results indicate that the pandemic negatively affected the amount and spacing of studying in an introductory computer science course, which may have a negative impact on their education.
7/21/2023 - Shell et al. (2016)
Students’ Initial Course Motivation and Their Achievement and Retention in College CS1 Courses
Abstract
The goal of this study was to investigate how students’ entering motivation for the course in a suite of CS1 introductory computer science courses was associated with their subsequent course achievement and retention. Courses were tailored for specific student populations (CS majors, engineering majors, business-CS combined honors program). Students’ goal orientations (learning, performance, task), perceived instrumentality (endogenous, exogenous), career connectedness, self-efficacy, and mindsets (growth or fixed) were assessed at the start of the course. Grades were significantly predicted from entering motivation; but prediction was highly variable across courses, ranging from not predicted for the engineering courses to highly predictable for the business-CS honors program. Course withdrawal was significantly predicted. Likelihood of withdrawing was decreased by future time career connectedness and learning approach goal orientation and increased by having an incremental theory of intelligence. Findings suggest that CS1 students who set learning approach goals for their classes have better academic outcomes and higher retention. Other motivational beliefs were inconsistent in their impacts and varied by course and student population. Except for students in an honors program, entering motivational beliefs weakly predicted achievement and retention, suggesting that impacts of the course itself on motivation and how motivation changes during the course are perhaps more important than student’s initial motivation.
7/14/2023 - Ying et al. (2021)
CS1 Students’ Perspectives on the Computer Science Gender Gap: Achieving Equity Requires Awareness
Abstract
There are numerous initiatives to improve diversity within the computer science field. However, women still disproportionately drop CS majors and earn less than one quarter of CS bachelor’s degrees in the United States. The extent to which CS students-especially male students-are aware of this gender gap is an open question. This paper reports on a study to investigate that question. We analyzed 325 CS1 students’ survey responses and found significant differences between women’s and men’s awareness of the CS gender gap. Men were significantly less aware of the gender gap than women, and men had significantly milder beliefs about whether women experienced adversity in CS and whether there should be targeted efforts to support women. Twenty students (10 women and 10 men) participated in follow-up interviews after the survey, where they discussed their experiences and perceptions of the computer science gender gap. Even among students who were aware of the gender gap, many had a superficial understanding of its cause, believing it to be due to women having less natural interest in CS. We argue that these findings are a call-to-action: university CS curricula need to include diversity, equity, and inclusion (DEI) training so that students have a more complete understanding of this complex issue, and so that these misconceptions do not continue to be perpetuated into the workplace.
7/7/2023 - Utting et al. (2013)
A Fresh Look at Novice Programmers’ Performance and their Teachers’ Expectations
Abstract
This paper describes the results of an ITiCSE working group convened in 2013 to review and revisit the influential ITiCSE 2001 McCracken working group that reported on novice programmers’ ability to solve a specified programming problem. Like that study, the one described here asked students to implement a simple program. Unlike the original study, students’ in this study were given significant scaffolding for their efforts, including a test harness. Their knowledge of programming concepts was also assessed via a standard language-neutral survey. One of the significant findings of the original working group was that students were less successful at the programming task than their teachers expected, so in this study teachers’ expectations were explicitly gathered and matched with students’ performance. This study found a significant correlation between students’ performance in the practical task and the survey, and a significant effect on performance in the practical task attributable to the use of the test harness. The study also found a much better correlation between teachers’ expectations of their students’ performance than in the 2001 working group.
6/30/2023 - Wang (2013)
Examining the Digital Divide between Rural and Urban Schools: Technology Availability, Teachers’ Integration Level and Students’ Perception
Abstract
This study aimed to explore the gap regarding technology integration between urban and rural schools based on theWill Skill Tool model. This study was guided by three main questions: 1) Is there any significant difference in terms of technology availability between rural and urban elementary schools?; 2) Is there any significant difference in terms of teachers’ attitudes, competence, levels and experiences in technology integration between rural and urban elementary schools?; 3) Is there any significant difference in terms of students’ attitudes, competence and experiences in technology integration between rural and urban elementary schools? This was a survey study with 275 teachers and 293 students as participants in southern Taiwan. Half of the participants came from regular urban schools and the other half were from disadvantaged rural schools. T-tests and Chi-Square tests were done to examine differences. The results showed that there was a significant difference in technology availability between rural and urban schools, including the number of interactive whiteboards, desktops in labs, notebooks, netbooks, and tablet computers. There was also a difference in teacher overall high-tech integration level between rural and urban schools.Urban teachers reached the level of “familiarity and confidence” but rural teachers only stayed at the level of“understanding and application of the process.” Teachers’ experience, purpose and difficulty in technology integration between rural and urban schools were also slightly different. In addition, there was a difference in students’ experience and preference in using technology to learn, especially using interactive whiteboards in learning.
6/16/2023 - Hogan, Li & Soosai Raj (2023)
CS0 vs. CS1: Understanding Fears and Confidence amongst Non-majors in Introductory CS Courses
Abstract
Previous research has been devoted to improving the experience of non-majors in introductory CS courses. In this study, we compare the experiences of non-majors in two different introductory CS courses, specifically with respect to fears about taking the course and change in confidence levels. CS0 is a computing course intentionally designed for non-majors, and CS1 is a more traditional introductory computing course. Both of these courses were composed primarily of non-majors and were taught by the same instructor. Survey data was collected from 124 students enrolled in CS0, and 502 students enrolled in CS1. Through qualitative analysis, we found that the fears of non-major students entering both of these introductory CS courses fell into one or more of nine distinct categories (e.g., Coding, Perceiving STEM as Difficult, Managing Workload). Additionally, using students’ confidence levels at the beginning and end of the courses, we found that students in CS0 had a greater increase in confidence level than those in CS1. Finally, we explored connections between students’ fears and how their confidence changed by the end of the course. We found that students across both courses with fears related to coding, lack of preparation, and being left behind had the highest average increase in confidence levels.
6/9/2023 - Ahadi & Lister (2013)
Geek genes, prior knowledge, stumbling points and learning edge momentum: parts of the one elephant?
Abstract
Computing academics report bimodal grade distributions in their CS1 classes. Some academics believe that such a distribution is due to their being an innate talent for programming, a “geek gene”. Robins introduced the concept of learning edge momentum, which offers an alternative explanation for the purported bimodal grade distribution. In this paper, we analyze empirical data from a real introductory programming class, looking for evidence of geek genes, learning edge momentum and other possible factors.