Lesson Using JUnit to Design Software Testing Programs

Learning Objectives

After this lesson, students should be able to:

• Apply the design process to test software.
• Create JUnit tests to test Java classes.
• Explain the importance of the testing phase in developing software.

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

International Technology and Engineering Educators Association - Technology

State Standards

Worksheets and Attachments

Pre-Req Knowledge

Students should have a working knowledge of Java. This lesson/activity set is intended to supplement a programming class to help students understand the importance of the testing phase in the design process, and teach them how to design tests to properly test Java classes. The teacher should have a working knowledge of Java and JUnit testing.

Introduction/Motivation

The software/system design process (shown in Figure 1) is a circular problem solving method. It has a number of steps that must be attempted to develop a solution to a problem: requirement analysis, design, implementation, testing, and evolution. In computer science, programmers are taught to analyze a problem before they create a design, implement a design and test the design. These steps are the fundamental pieces of the software/system design process. During the analysis phase, you decide, identify and frame the problem. You identify constraints and variables. With this knowledge, the design phase is entered during which you decide what algorithms and control structures might work to solve the problem. To implement the design, the software engineer writes the code using the proper syntax for the particular language being used. The next phase, testing, is critical to the design process and is often minimalized or not properly completed. In this lesson, you will learn to test a Java program thoroughly. The goal of the lesson is to learn to design and implement a test case using JUnit, which is a Java software testing tool.

So why do we test programs? A computer program can be a very complex logical creation. The computer will happily follow the instructions you give it, whether those instructions match what you actually want it to do or not. Computers are not smart enough to understand their programmers' intent—they can only follow the instructions you give, right or wrong. Since most computer programs are large, it is common for programmers to make mistakes somewhere in the process of creating a program.

With that said, how do we test programs? The most common way to test a program is simply to run the program and see whether its results match what you expect. This kind of testing can be useful, but it is not the only kind of test available. In particular, large programs are difficult to test by just running the program, because the program may contain many special cases that do not come up very often in normal use. An effective software designer anticipates and tests those special cases by designing specific tests to make sure that the program works even with unusual input.

While testing is a critical part of software design, keep in mind that testing alone cannot verify a program's correctness, because it is usually not possible to test every possible input to a program. However, a well-chosen set of tests can help diagnose and ultimately fix nearly all program errors, especially if the tests are complete and "exercise" code that is more likely to contain errors.

In this lesson, you will focus on the design process, especially the testing phase. You will be asked to consider how you have tested computer programs in the past and will learn about a new method, called JUnit, to test programs in the future. Finally, you will design tests using JUnit and implement those tests. Recall a Java program that you have recently written. Consider how you checked your program for correctness. Were those checks sufficient to ensure that the program was in fact correct? Did your tests appropriately complete the testing phase of the design process? Could that phase have been better?

JUnit is a Java testing tool that can be used to create better tests. I will now demonstrate the use of JUnit with the Java class Using JUnit: Pirate and its accompanying JUnit test Using JUnit: Pirate Test. (Teacher note: See the provided resource links and attachments. If you click on the links in the previous sentence, you will be able to view the Pirate class and the test for the Pirate class as PDF files. For working purposes, the MS Word document versions are recommended and available in the Attachments section. The teacher demonstration should show students how the JUnit tests correspond to the tested class' methods. It is recommended that the teacher project the tests as conducting them. Once you have performed the demonstration, ask students the following questions.)

• How is a test method labeled in the test class? (Answer: It has the annotation @Test.)
• How does a test method check that a given condition is true? (Answer: It calls the assertEquals() function.)
• Does the fact that the class passes all its tests ensure that the class is correct? (Answer: No, but it does ensure that the class does not contain any defects that are tested for by the test suite.)
• Can any set of tests guarantee that a program or a part of a program is correct? (Answer: No, we can never guarantee a program is correct by testing alone. However, if we choose tests well and our code passes the tests, we can greatly reduce the chance that the program contains errors.)
• If you were to change the Pirate class in some way, how hard would it be to check that the change did not break anything? (Answer: It would be very easy, only requiring that the tests be run again.)

Now that you have observed my demonstration of how JUnit functions, you will explore the JUnit testing tool and its use. Start by downloading the Using JUnit: Bank Account Java Class, which provides the BankAccount class, saving it as a Java file, and checking that it compiles correctly. (Teacher note: The MS Word document version of the BankAccount class is available in the Attachments section, while the previous link provides an easily viewable PDF version of the file.) The BankAccount class represents a bank account, providing functionality to a) track an account's account number and balance, b) permit deposits and withdrawals, and c) verify that deposits and withdrawals are sensible (positive amounts) and do not overdraw the account. What are some of the requirements of the class that you might want to test?

(Teacher note: This is the analysis phase of the design process. For example, students might test that the deposit() method correctly adds money to the account or that the withdraw() method does not allow the user to withdraw more money than is present in the account. Give students a few minutes to discuss the requirements to test the BankAccount class.)

Now that you have had some time to discuss requirements to test, you will work individually or in small groups to write JUnit tests for one or two of the requirements that you have listed.

(Teacher note: Give students time to design, create and use their tests. You may wish to have each student or group write tests for a different requirement and then combine all the tests into a more complete test set, or have students write tests for the same requirement so that they can compare their work. As students work, ask yourself or students the following questions:

1. Did students identify the major requirements listed in the BankAccount class description as testable requirements?
2. Were students able to write JUnit test cases that compile and run successfully?
3. Does the BankAccount class as provided pass the students' tests? If not, why not—does an error exist in the test case or the BankAccount class itself?

To conclude the lesson, ask students to discuss what they have learned or struggled with so that they are prepared for the associated activity, Testing Software Programs with JUnit, in which students fully implement JUnit test for a Java class given to them.)

Lesson Background and Concepts for Teachers

In this lesson and its associated activity, Testing Software Programs with JUnit, students learn how to use JUnit, which is a unit testing framework for Java. JUnit has been important in the development of test-driven programming where the tests for a Java class are created BEFORE the class is written. This programming style is different than traditional methods, but has been recognized as a powerful way to develop software applications. JUnit is linked as a Java JAR file at compiling.

(Teacher note: This following information is meant for the teacher. Teachers can provide the information below to the students as they see fit or have students "discover" it as part of the lesson. It is part of the lesson's summative assessment.)

How to Create a JUnit Test Case with NetBeans

• Right-click your project in the Projects window and choose "New -> JUnit test" from the pop-up menu that appears.
• In the dialog box that appears, give your test class a name. Typically, a test class has the same name as the class it is testing, with the word "Test" appended. For example, a test class for a class named Foo would be named FooTest. The test class' package should be set to be the same as the tested class' package. Click Finish.
• If NetBeans offers you a choice between JUnit 3.x and JUnit 4.x, choose JUnit 4.x.
• In the newly created test class, create your test methods (functions that test a function of the program). Each test method must:

1. Be preceded by the annotation @Test
2. Contain one or more assertions (calls to functions such as assertEquals, assertTrue, or assertFalse)

To run your test cases, choose Test Project from the Run menu. NetBeans displays the Test Results window, showing the results of running your test(s). Each test is classified as "passed" (it worked), "failed" (an assertion in the test did not check, or the test unexpectedly threw an exception), or "error" (the test could not be compiled).

Resources

JUnit home: http://junit.org/

JUnit documentation: https://github.com/junit-team/junit/wiki

JUnit downloads (current version is 4.11): https://github.com/junit-team/junit/wiki/Download-and-Install

Writing JUnit tests in NetBeans: https://netbeans.org/kb/docs/java/junit-intro.html

Using JUnit with Eclipse: http://onjava.com/pub/a/onjava/2004/02/04/juie.html

Associated Activities

• Testing Software Programs with JUnit - Students implement the complete design process to create and test JUnit tests for a provided Java program. They journal in an engineering notebook about each phase of the design process, including the work they did in each phase.

Vocabulary/Definitions

edge case: A test case whose input is near a "boundary" where the program changes from one behavior to another. For example, if a game allows a character to hold four items at a time, then edge cases include collecting the fourth item (the last item that the character can hold) and collecting the fifth item (an item that would exceed the character's carrying capacity).

error case: A test case whose EXPECTED result is an error. For example, for a program that divides two numbers, dividing by zero is an error case. A well-written program has a plan in place to handle unexpected data in a reasonable manner; error cases test this plan.

test case: A specific set of input for a program or part of a program, along with the expected results of running the program with that input. Note that a test case MUST contain both the input and the output! A test can only help to verify a program's correctness if the expected output can be compared to the actual output of the test case.

test suite: A collection of test cases that is intended to fully "exercise" a section of a program. A complete test suite should include edge cases and error cases as well as a representative selection of "normal" cases.

test-driven development: The software designer creates test cases for a section of code FIRST, before writing the program itself. While to many programmers this seems "backwards," writing tests first helps programmers to focus on what the code SHOULD do. By considering expected results in advance, most programmers write better code with fewer errors.

Assessment

Pre-Lesson Assessment

Opening Discussion: Question students about what they know about the design process and how it can be applied to testing software for bugs. Example questions:

• What steps are present in the design process and what does each entail? (Answer: The software development cycle has five parts: problem analysis, design, implementation, testing, and evolution. In problem analysis, the problem is framed and constraints are evaluated. In design, possible solutions are brainstormed that address the problem and constraints identified in the first step. Implementation is exactly what it sounds like; a design is put into practice. In this case it means that code is written. Testing is the focus of this lesson. Properly testing a design is critical for the development of software. The last step, evolution, is thinking about how the product can be improved or applied differently.)
• How can you apply the design process to test software? (Answer: The design process can be applied in the creation, development and implementation of the testing process. You must first decide what and how to test [problem analysis], design a solution, implement the test cases, and then see if they actually perform as they were designed [testing]. You could think of software testing as a mini design process inside of the larger software design process.)
• Why is testing software important? (Answer: If you do not properly test software, it is possible that the program will not function correctly. Computations could be wrong, the program could crash and users will not be happy with their experiences.)

Lesson Embedded Assessment

Questions: To provide a formative assessment as students work, ask yourself or the students these questions:

• Did students identify the major requirements listed in the BankAccount class description as testable requirements? (Note: The testable requirements are that the class stores the account number and balance, permits deposits/withdrawals, and does not allow overdrafts.)
• Were students able to write JUnit test cases that compile and run successfully?
• Does the BankAccount class as provided pass the students' tests? If not, why not—do errors exist in the test case or the BankAccount class itself?
• Were students able to use their programming tools to create and run JUnit tests?
• Can students explain the meaning of a test succeeding versus a test failing?
• Can students identify and use different kinds of JUnit assert statements?

Post-Lesson Assessment

Writing: Have students complete the Using JUnit: Summative Assessment, which asks students the following writing prompt. See Using JUnit: Summative Assessment Answer for the solution.

• Create a list of the necessary steps to create a JUnit test case, using the development tools used in class. Explain how the test is connected to the design process and how it could be implemented.

Copyright

Contributors

Supporting Program

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.