Lesson Java Code:
Does It Work? Test and Test Again

Learning Objectives

After this lesson, students should be able to:

• Explain the role that testing plays in the software/systems design process.
• Select a set of test cases for a section of program code.
• Evaluate test cases for completeness and code coverage.
• Analyze program code for defects based on the results of test cases.

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

NGSS: Next Generation Science Standards - Science

International Technology and Engineering Educators Association - Technology

State Standards

Worksheets and Attachments

Pre-Req Knowledge

Students must know Java Programming basics. Although their knowledge need not be advanced, students must be able to develop code that needs to be tested in order to design test cases as part of a larger design process.

Introduction/Motivation

The software/system design process — a specific use of the engineering design process — is cyclical in nature and has five steps that are necessary for the development of quality solutions (see Figure 1). In computer science classes, you are taught to analyze the problem that you wish to solve by designing a software (computer) program. After your initial analysis, you next design the code by making choices about algorithms, control structure, and more. In the implementation phase, you put your design into practice by using the proper syntax for the computer language.

One critical component of the design process that is often overlooked, and is also difficult to do well, is the testing step. In this lesson, you will learn to put a computer program, written in Java, through its paces (that is, explore all possibilities) and thoroughly test it, with the ultimate goal of learning how to properly design and implement test cases for your code. The design and implementation of test cases is a miniature software/systems process inside of the larger engineering design process.

In this lesson, you will explore the testing tool, CodingBat. Let's start by navigating to the CodingBat website (http://codingbat.com). Once there, try to write solutions to one or more of the Java problems on the website.

Note to teacher: Use the information provided in the next section [Lesson Background & Concepts for Teachers] to lead a class discussion on how CodingBat works and what different types of test cases need to be written.

Lesson Background and Concepts for Teachers

Understanding CodingBat: CodingBat is a free website with live coding problems that can be used to build coding skills in Java (and other languages). The coding problems allow users to gain a deeper understanding of coding concepts as they practice solving problems. Instructional videos about individual topics are provided as well. The website can be used by the teacher to provide homework for students, a lab setting, or to provide live lecture examples.

A unique feature of the website is that the problems are short, similar to test questions, with instant feedback, so that learning can be immediate and successful. The website provides an opportunity for students to learn how to write Java method code in an environment that helps them concentrate on coding with fewer distractions.

To properly test a program, you need to develop three different test cases. These include normal cases, edge cases and error cases. A "normal" test case consists of a specific input for a program and the expected results for that input. A normal test case must include both specific inputs and the expected results so that it can verify if the test case passed. An "edge" test case tests input near the boundary where a program changes from one behavior to another. This type of testing might be to verify if a game advances to the next level if all the criteria for the level the user is on are met, or if the level is based on points and the "level" up number of points is achieved, if the function performs correctly. An "error" test case has an output were an error is expected. For instance, if a program does computations to see if the program handles division by zero, which is an invalid operation, it should produce an error, rather than crashing the entire program. Together these test cases form a testing suite, and when implemented properly, they evaluate the performance and correctness of the software application.

Note to teacher: Depending on their Java experience, you may wish to pre-select specific problem(s) for students to solve. During this coding experience, guide a class discussion about the testing phase of the design process. Example questions:

• How does the CodingBat website determine whether submitted code is correct? (Answer: It runs the code several times, with different input values, and checks the output.)
• Is the submitted code correct or incorrect if some, but not all, tests run incorrectly? (Answer: It is incorrect. Even one incorrect test case demonstrates a problem in the code.)
• How do you think the creators of the CodingBat website decide which cases to test? (Answer: For a thorough testing phase, all cases [normal, edge and error] are tested.)

CodingBat is strongly recommended for computer science classes that teach the Java programming language, or classes that incorporate Java. It can be used by teachers and students to develop skills in writing test cases for programs.

In the associated activity, Java Programming: Testing the Edges, students develop test cases for a provided problem.

Associated Activities

• Java Programming: Testing the Edges - Students apply the software/systems design process with open-ended solutions, specifically focusing on the testing phase. Working in small groups, they create Java functions and test cases to determine whether a Java function correctly implements the requirements of the problem.

Vocabulary/Definitions

edge case: A test case whose input is near a "boundary," where the program changes from one behavior to another. For example, if a game allows a character to hold four items at a time, then edge cases would include collecting the fourth item (the last item that the character can hold) and collecting the fifth item (an item that would exceed the character's carrying capacity).

error case: A test case whose expected result is an error. For example, for a program that divides two numbers, dividing by zero is an error case. A well-written program should have a plan in place to handle unexpected data in a reasonable manner; error cases test this plan.

test case: A specific set of input for a program or part of a program, along with the expected results of running the program with that input. Note that a test case must contain both the input and the output. A test can only help to verify a program's correctness if the expected output can be compared to the actual output of the test case.

test suite: A collection of test cases that is intended to fully "exercise" a section of a program. A complete test suite should include edge cases and error cases as well as a representative selection of "normal" cases.

test-driven development: The software designer creates test cases for a section of code first, before writing the program itself. While this seems "backward" to many programmers, writing tests first helps the programmer to focus on what the code should do. By considering expected results in advance, most programmers write better code with fewer errors.

Assessment

Pre-Lesson Assessment

Class Discussion: Ask students what they know about the software development cycle (design process).

• How many steps are in the software development cycle/design process? (Answer: The software development cycle has five steps or parts.)
• What are the steps? (Answer: The steps are: problem analysis, design, implementation, testing and evolution.)
• What does each step entail? (Answer: In problem analysis, the problem is framed and constraints are evaluated. In design, possible solutions are brainstormed that address the problem and constraints identified in the first step. Implementation is putting a design into practice; or implementation means that code is written. The fourth step is testing, which is critical for the development of software. The last step, evolution, is thinking about how the product can be improved or applied differently.)

Summative Assessment

Prompted Writing Assignment: Assign students to answer the questions on the Writing Assessment handout. (Note: Alternatively, write one/all of the following prompts on the classroom board for students to answer in their engineering journals.)

• Explain how testing fits into the software/systems design process. (Example answer [answers will vary]: Testing is a critical component of the design process. To properly test software, a mini-design process must be completed to fully understand how to test and create the specific tests needed to adequately test the designed program.)
• Explain why testing is critical to the successful completion of a quality result. (Example answer [answers will vary]: Testing enables software designers to see if a newly designed solution really addresses the identified problem; it is absolutely essential that the design be adequately tested to minimize errors in the future.)
• Explain why tests for a section of a program should include normal cases, edge cases, and error cases. (Example answer [answers will vary]: For normal cases: these test that the code actually does what it is supposed to do. For edge cases: these cases are where the code is most likely to contain mistakes, so they should be tested more thoroughly than normal cases. For error cases: the test should confirm that the program does something reasonable [which might be just giving an error message] rather than trying to return a nonsensical answer.)
• How do these fit into "designing" a quality software test? (Example answer [answers will vary]: These cases are part of the analysis phase of the design process. In the analysis phase, the software designer must decide what cases are required to adequately test the software. Without thorough testing, the software is not adequately prepared for large scale consumer use.)

Copyright

Contributors

Supporting Program

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.