Hour 5: Boo Boo Management - Part 1

Overview

Perhaps one of the most common actions in computer programming is making mistakes. They are common and everyone makes them. In this lesson, you will learn about mistakes, which kinds are the most common, and what happens when you inevitably make them. You will then intentionally make various kinds of errors or mistakes and learn how to adjust your code to fix them in blocks.

Goals

You have the following goals for this lesson:

• Learn about the different kinds of errors.
• Learn about the kinds of common mistakes that students make as they are learning.

Warm Up

Imagine people at different age ranges. There are young students in K-2 that are emerging readers. Students from 3-5, 6-8, or 9-12, all learning computer science as they go. There are also students in undergraduate through doctoral programs and professional programmers with potentially decades of experience. What kind of mistakes do you think each of these groups might commonly make and why?

Vocabulary

You will be learning about the following vocabulary words:

Code

There is no new code for this lesson.

CSTA Standards

This lesson covers the following standards:

• 3A-CS-03: Develop guidelines that convey systematic troubleshooting strategies that others can use to identify and fix errors.

Explore

Computer programming languages can take practice to learn and an important part of that practice is learning about mistakes. When people think of the idea of a mistake, it often has a bad connotation. Perhaps a person feels subjectively bad for making one, or is frustrated in trying to fix it.

In practice, however, while certain kinds of mistakes are harder to fix in programming than others, the good news is that humans are quite predictable in the kinds of mistakes they make and many have common solutions. Some are so commonplace that the computer can take a good reasonable guess about how to fix them. As a programmer, recognizing these mistakes and understanding why these errors occur will strengthen your problem solving skills and help you learn to invent more complex programs.

While the word "mistake" is common in English, in computer science you typically break down the concept into types of errors. The most common two kinds are compiler errors and runtime errors. Consider an example of both.

Compiler Errors

Programming languages are complicated bits of technology. You might think of them as turning human-like language into 1s and 0s, which in a literal sense they do, but in practice these languages are made for people. The basic problem though is this: what words or symbols are precise enough for a machine to understand, yet understandable enough for a human to know how to teach that machine what to do?

This question leads to a natural mismatch between humans and machines. The problem is made worse by the fact that machines have to process this text, interpret it, then reorganize it to be converted into those 1s and 0s. This is essentially what is known as a compiler. Along the way, many kinds of errors can happen. One such category of errors, called compiler errors, are those errors that can happen before a program ever runs.

Intuitively, you might think compiler errors are frustrating, because they stop your program from running. In practice, however, compiler errors are your friends. They represent a situation where the computer can calculate that, with certainty, your program may never run correctly anyway and they can tell you that in advance. This can make the problem much easier to find and you are often not so lucky.

Common Compiler Errors

There are many kinds of compiler errors and one easy way to understand them is to think through which ones are common. It is tempting to just guess, but academic research has shown that the issue really needs evidence to understand. One recent overview of what is known was conducted by Becker et. al [1] and an overview of which errors are common amongst students was conducted by Altadmri and Brown [2]. One easy way to think about them is that, if you have errors in your code, to fix the first and ignore the rest [3].

Consider an example with an output statement:

output "Hello 

First, notice that the system is being asked to output the word "Hello", except that text requires a double quote on the left and the right hand side of the system. In this case, the right hand side is missing. One of the most common errors is mismatched braces [1, 2]. This is not exactly like, but is analogous to, this situation. In this system, typing this as raw text would then give the following:

Part of the problem is that computer programming languages are not always terribly good at predicting what a human did wrong. In this case, the error message is stating that the word Hello with only a single left double quote is not a valid expression, which is true but confusing. How to make such messages better, and for whom, is an active line of research in computer science.

Besides matching, the academic literature shows students make many mistakes commonly and thus you should expect to make them too. The authors of this text still do. These include errors like [2]:

Even in block languages, it is a common misconception that no errors can be made. The errors are just shown differently and constrained in different ways. Further, in some cases the language is constrained to the point that, while it is true some errors are impossible, many kinds of programs cannot be written.

[1] Brett A. Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, Janice L. Pearce, and James Prather. 2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. In 2019 ITiCSE Working Group Reports (ITiCSE-WGR '19), July 15-17, 2019, Aberdeen, Scotland UK. ACM, New York, NY, USA, 34 pages. https://doi.org/10.1145/3344429.3372508

[2] Amjad Altadmri and Neil C.C. Brown. 2015. 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data. In SIGCSE '15. ACM, 522--527.

[3] Brett A. Becker, Cormac Murray, Tianyi Tao, Changheng Song, Robert McCartney, and Kate Sanders. 2018. Fix the First, Ignore the Rest: Dealing with Multiple Compiler Error Messages. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE '18). Association for Computing Machinery, New York, NY, USA, 634-639. https://doi.org/10.1145/3159450.3159453

Runtime Errors

Sometimes errors can happen while the code seems to be working perfectly fine. Runtime errors are different from compiler errors. Instead of a program failing before it runs at all, the computer has identified that the source code is valid. It cannot, and in fact it is impossible to, know for sure your program is correct, but it can at least know that the source code looks like a valid source in the programming language.

The problem is, not all source code gives useful results. Consider an example, which might look as so:

output 5 / 0

This code, whether typed as raw text or placed into a block, is valid and correct. The computer programming language sees that it is a valid line of code. Whether the code is so-called blocks or text is not relevant. The error would be the same. The problem is, what does a computer program do if you divide by zero? The answer is simple: it crashes. Depending on the programming language, it might give no information at all, write some cryptic numbers, or give a weird looking error, like so:

Error: class java.lang.ArithmeticException, / by zero
    file: Main.quorum,  class: Main,  action: Main,  line: 1

The details of what this means are out of scope, but the bottom line is that when a computer program is running, it crashed on line one of Main. When it crashed, it had a specific kind of reason for crashing, which is that the computer tried to divide by zero and it cannot do that. There are many kinds of runtime errors, for all sorts of situations, and you can even define your own. However, the crucial point to understand is that errors can happen before a program runs (compiler errors) and during a program's run (runtime error). Runtime errors might also be called "bugs."

Engage

For this lesson, you are going to participate in the Crazy Error Contest. This activity, modeled very loosely and philosophically on the International Obfuscated C contest, is to try and come up with the most absurd error message you can.

Directions

Computers today have an imperfect relationship with the errors they produce. The basic problem is that humans can put really anything inside of a computer program and computers cannot know their intent. Humans may not even know their own intent. It is important to remember while you are coding, the computer will be interpreting anything you type into a literal sense so it is important to produce code in a methodical manner.

Sometimes what might look like subtle differences in a program may even produce an error message that does not logically make sense given the code. Thus, for this activity, you will be living the dream to practice getting a feel for how things work: try to break things.

This activity is intended to be light hearted, but serves an important purpose: making mistakes and errors in your code is common and important when it comes to learning how to program. Part of that is that some feel anxiety around making errors and this exercise may help you understand that this is not necessary. You really can get used to the types of strange things computers say. A second consideration is that this exercise can let you practice making errors in a systematic way to help you observe what the computer does and when. Remember that even the most experienced programmers often continue to make silly mistakes and that is perfectly fine.

Past this, in terms of what is known about errors in the Quorum language specifically, consider a few facts:

• Compiler Error messages in Quorum are written at the 8th grade reading level across the entire programming language. This does not, however, mean they are easy to understand. Quorum cannot always predict, for every wrong program, what is wrong.
• Runtime error messages in Quorum are tricky and depends on how you are running Quorum and where.
• The kinds of errors humans make in Quorum are different from other programming languages. Basically, people make some kinds of errors much more commonly than others [4].

To provide an example, the most common error in Quorum (5.7% of errors) for students is trying to output a variable name. However, either the variable is misspelled, they are using a variable that does not exist, or there is a capitalization error.

The second most common error is accidentally combining items while trying to output them (4.9%). For example, to output a variable named "a" you would write "output a" but the user might type "outputa" or another phrase. Now consider an example error, which happens to be the third most common kind (4.7%):

Fish tacos

This produces the error, "I could not locate a type named Fish. Did you forget a use statement?" This makes sense, sort of, if you assume Fish is something real in the language (sadly, it is not). However, if you add one single character to the end, a semicolon (there are no semicolons in Quorum), you would have the following code:

Fish tacos;

Yet, with that single change, the error moves to, "The symbol ';' is not valid. Please remove it." This is true, but misleading in that removing the symbol would fix one problem, but not make the program correct.

The problem arises in the "Fish" type because it does not exist in the language. What you are accidentally creating is an object of "Fish" type. Recall when you created the "Math" object there were no errors. This is because there existed a library (Libraries.Compute.Math) that allowed for that variable to exist. If you tried to do the same thing with "Fish", there was no library to support it, thus creating a runtime error.

Task 1: Wrong Answers Only

Come up with a set of alternative programs with a partner. You can do this through any mechanism you wish. You can type in random characters or symbols, make up fake names, mismatch anything, or really just tinker and play. Have fun. The goal for this first task is to purposely break your program and recognize compiler errors in the system.

An example of this would be the following line of code:

Where the following error that would occur would be:

I found 2 errors. Line 1, The symbol '!' is not valid. Please remove it Press command 3 to go to the errors tab  or command G to go to the line in the editor.

Task 2: Top 3

Agree with your partner on what you perceive to be the top 3 worst, or funniest, errors you can get the system to produce. Then write down:

• The code you used to produce the error.
• The message the system says.
• What you think might be a better error message if you could change it.

For the message you suggest, try to imagine the limits of the computer. How would it know how to generate the message you suggest? Do you think it would be possible to generate such a message, and why or why not? What is the actual cause of the problem compared to what the real message or what your new one is?

Task 3: Pick a Winner

Once all groups have selected their 3 answers, share out the top choice from each group. Pick a winner, or loser depending on your point of view, across the group as a whole. The criterion for how to pick this is entirely subjective and the winner does not matter. Subjectively, the most outrageous error message seems best, but the point is to have fun with it and keep it lighthearted while learning about errors in a stress free way.

[4] Patrick Daleiden. 2020. Toward Productivity Improvements in Programming Languages through Behavioral Analytics. Ph.D. Dissertation. University of Nevada, Las Vegas. Las Vegas, NV.

Wrap up

Write down the criterion for your top three messages so they can be shared with the group after you have completed your tinkering. Provide the winner fish tacos, if that is their thing or yours.

Next Tutorial

In the next tutorial, we will discuss Data Science, Part 1, which describes data science in Quorum.