1.4.2Errors and test cases

Types of errors

Syntax errors

A syntax error breaks the grammar rules of the language, so the compiler cannot understand the code and refuses to build the program. The IDE usually underlines it in red before you even run anything.

int total = 0
System.out.println("Total: " + total);

The compiler reports something like ';' expected on the first line. Other common syntax errors are misspelt keywords (Int instead of int), missing brackets, and using a variable that was never declared. Syntax errors are the easiest to fix, because the compiler tells you where they are.

Runtime errors

The code compiles, but something impossible happens while it runs, and the program stops with an exception:

int count = 0;
int average = 100 / count;   // ArithmeticException: / by zero

String name = null;
int n = name.length();       // NullPointerException

Other examples are reading past the end of an array (ArrayIndexOutOfBoundsException) and typing letters when a number is expected (InputMismatchException). Runtime errors often depend on the input, so a program can run correctly a hundred times and then crash on unusual data. That is one reason why invalid data must be tested.

Logic errors

The program runs and finishes, but the answer is wrong. The computer cannot detect these, because the code is perfectly legal; it just does not do what you meant.

double average = a + b + c / 3;   // wrong: only c is divided by 3
double average = (a + b + c) / 3; // correct

Logic errors are the hardest to find. You find them by testing with values whose correct answers you already know, by tracing (see 1.4.1) and by using the debugger (see 1.4.3).

Designing test cases

A test case has an input, an expected output and the actual result. Always include:

• Normal data: typical valid input.
• Boundary data: values at the edge of a range (for the grade program: 49, 50, 64, 65, 74, 75, and the ends of the valid range, 0 and 100).
• Invalid data: input that should be rejected (negative marks, letters).

Testing and debugging are different

Testing finds out whether there is a problem: you run the program with chosen data and compare the actual results with the expected results. Debugging finds out where and why the problem happens, and fixes it. Testing comes first; when a test fails, you debug; and after every fix you run all the tests again, because a fix in one place can break something somewhere else.

How to choose good test data

You cannot test every possible input, so you choose a small set of values that are most likely to reveal mistakes:

• Normal data checks that the program works for everyday values. Pick at least one value from the middle of each range (one A, one B, one C and one F).
• Boundary data checks the edges, where most mistakes happen. For each limit, test the value on the limit and the value just past it: 49 and 50, 64 and 65, 74 and 75. A program that uses > where it should use >= passes every normal test, but fails at 50.
• Invalid data checks that the program rejects bad input instead of producing nonsense or crashing: -5, 101, letters, an empty input.

Writing a test case

Write the expected output before you run the test. If you run the program first and then copy what it printed into the "Expected" column, the test proves nothing. A good test case is specific: not "enter some marks" but "enter 65, expect B".

Use the lab below to run the test cases against a correct program and a program with a hidden logic error.

Levels of testing you will meet: unit testing (one method or class), integration testing (classes working together) and system testing (the whole program).