Hands-on Activity Java Programming of OCR

Learning Objectives

After this activity, students should be able to:

• Explain software testing and challenges involved in software testing.
• Apply optical character recognition as a Java method.
• Determine the reliability of their OCR implementations.

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

International Technology and Engineering Educators Association - Technology

State Standards

Materials List

Each group needs:

• computer (with Internet access)
• OCR algorithm, in flow chart, pseudocode or similar paper format; created in the associated lesson
• Java programming environment; install a program such as NetBeans (free) or download software such as: http://introcs.cs.princeton.edu/java/mac/ for Mac OS X (free) or https://textpad.com/ for Windows (trial copy)
• PictureEdit.java, DigitReader.java, ScoreMaker.java; students need these three .java files to get their programs running; they provide a framework and simplify the process of creating OCR software; they are provided in the ScoreMaker.zip
• (optional) access to a digital camera, to take a few images of school scoreboards in use
• Pre-Activity Quiz, one per student
• Post-Activity Quiz, one per student

Worksheets and Attachments

Pre-Req Knowledge

Students and teachers must have a strong understanding and experience with Java programming basics.

Introduction/Motivation

In this activity, you will obtain valuable hands-on experience with the steps of the engineering design process. All engineers follow some type of design process as they develop solutions for problems. As you play the part of a team of software engineers, your challenge is to create a Java program that can perform optical character recognition (OCR) on one digit of a scoreboard. This is a small part of OCR, but is a good place to start.

How do software engineers approach a new challenge? (Listen to student ideas; expect them to know this.) They first define the difficulties and constraints that affect the problem and then create design solutions that address the problem and those constraints. You did this in the associated lesson when you created your algorithms on paper. Now, you will see if what you designed will work. You will implement, test and refine your algorithms, which are the next steps in the engineering design process.

Engineers work collaboratively in teams to develop software, so you will not work alone. Remember that teams benefit from the different resources and ideas that each person brings. That means that teams have the potential to solve problems more easily than people working alone. As you work in your group, build on each other's ideas and expertise to improve your design solutions.

Procedure

Background

Optical character recognition (OCR) is the process of having a machine recognize characters (printed, scripted, etc.) and turn them into machine characters that can be manipulated on a computer (or other machine) in some way. To create a program that covers all aspects of OCR is nearly impossible due to the different nature of typefaces, scripts and individual handwriting. This means that to thoroughly test an OCR software package for all possible cases it might encounter is impossible. As part of this activity, students think about and explore the nature of this impossibility in testing. Have students consider the following questions as they work through the activity:

• How will you handle the impossibility of testing every possible case that might ever be applied to the algorithm?
• What must an algorithm do in order to perform well enough? How will you know?

Before the Activity

• Make copies of the Pre-Activity Quiz and Post-Activity Quiz, one each per student.
• Arrange for enough computers—at least one for each student group.
• Make sure each computer has a Java programming environment available, as well as the ScoreMaker software and the three Java classes (provided in ScoreMaker.zip) so students can test their OCR implementations.
• If possible, have student groups each take two or three different digital photographs of suitable scoreboards in use at the school's sports venues. Alternatively, or in addition, find images on the Internet (see Figure 2). These images will be used for "real-world" testing of their programs, after use of the three images packaged with the ScoreMaker software in ScoreMaker.zip, and using the Pixel Editor program mode to simulate scoreboard images for initial testing purposes.
• Make sure student groups have handy the OCR algorithms they created in the associated lesson.

With the Students

1. Administer the pre-quiz, as described in the Assessment section.
2. Present the Introduction/Motivation content to the class.
3. Have students sort into their same groups as in the associated lesson, groups of two to three students each. Have each group assemble at a computer with its already-developed OCR algorithm.
4. Have groups begin by using a Java programming environment to implement the OCR algorithm they created in the associated lesson.
5. Review with students the engineering challenge: To read ONE digit from a typical high school scoreboard, such as one you might find on a football field or in a gym. These digits are generally set up in a seven-segment array (see Figure 1).
6. Work with the class to help students think through the creation of their Java programs. Some example topics and points to make:

• How does a computer "see" a scoreboard? With images, the computer only sees a cluster of pixels, so we must use the colors of each pixel to determine what we are looking at.
• As a class, start to process the scoreboard images and discuss some pre-processing techniques like turning color images into black and white images. Doing this makes it easier for the next step—to determine the digit in the image.
• Talk about the seven-segment display that is common to most digital scoreboards. Work through how it might be determined whether a segment was on or off. At this point, expect groups to begin to take more independent approaches. How will your group's program determine if the top segment was on or off?

  

7. Give teams some time to test and modify their Java programs with the aim to achieve the most reliable readers possible. Direct them to test them using the ScoreMaker software package (the zip file).

• Have students use the Lights Editor to precisely control what images look like and to see how the software works on similar-looking numbers, such as 9 compared to 4, digits that are very similar except for the top segment.
• Have students test thoroughly with many different scoreboard images, such as the three available with the ScoreMaker software and simulated scoreboard images made with the Pixel Editor program mode.
• After testing in the Lights Editor, have students perform "real-world" tests by using their own digital photographs of in-use school scoreboards and images from the Internet. Expect the testing of these individual numbers to reveal situations in which groups' algorithms fail, leading them to iterate their designs further.

8. After additional testing and revising, expect all groups to have some level of optical character recognition software developed. As a class, test the different solutions alongside each other using the "Select OCR" feature; this enables testing to see how each group's software works in as many different situations as can be accomplished.
9. After each group has optimized its program to what they deem is its best potential, have each group present to the class how its program functions, its algorithm logic, how they tested, what steps its algorithm takes to find the number, and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn. Have groups explain how they decided if a segment was on or off so that everyone can see the varying levels of success with each approach.
10. As a cumulative effort, have all the groups work together to create a class "final version" of the OCR software that builds on the class best practices.
11. To conclude, hold a class discussion to talk about how the lack of a testing oracle affects the testing of an OCR implementation and what this means for a final product. Is it guaranteed to work in all cases? If not, what is good enough? How do you know? Since you cannot anticipate and test every possible scenario, you may not be certain that the software will work in all situations. What other types of software might be susceptible to this kind of uncertain results?
12. Ask students if they know how scientists and engineers generally differ in their work. (Listen to student ideas.) Make the point that scientists and engineers both contribute to the world of human knowledge, but in different ways.

• Scientists follow the steps of the scientific method to make testable explanations and predictions about what they observe and want to discover in the world. Scientists ask questions and develop experiments, or sets of experiments, to answer questions.
• Engineers follow the steps of the engineering design process to create solutions to problems and to create things that have never existed before, such as products, websites, structures and processes. Engineers identify specific needs and then create solutions that meet those needs. They think: Who need(s) what because why?

13. Administer the post-quiz, as described in the Assessment section.

Vocabulary/Definitions

Java: An object oriented computer programming language.

optical character recognition: The mechanical or electronic conversion of handwritten, typewritten or printed text into machine-encoded text. (OCR)

ScoreMaker software: A java-based software designed by the author and provided as an attachment to this activity.

test oracle: In software testing, a set of known input data and a corresponding set of known or expected output data.

Assessment

Pre-Activity Assessment

Pre-Quiz: Before starting the activity, administer the three-question Pre-Activity Quiz. The pre-quiz ask students to consider challenges in designing and testing software that does not allow for full testing. Example answers are provided on the Pre-Activity Quiz Answer Key.

Activity Embedded Assessment

Observations: As students are engaged in the activity, ask yourself the following (or similar) questions:

• Do students grasp the idea that many software programming situations do not lend themselves to be absolutely, fully tested?
• Are students able to identify techniques to test software when no oracle or known answers are available for every question?

Post-Activity Assessment

Presentations: After groups have polished their programs, have each team present to the class how its program functions, its algorithm logic, its testing discoveries and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn, so everyone sees the varying levels of success with each approach. Use this opportunity to assess group accomplishments in maximizing testing results to build robust and accommodating algorithms.

Post-Quiz: At activity end, administer the Post-Activity Quiz. Through three essay questions, assesses students individually on their understanding of software that is not able to be fully tested for all scenarios and resulting implications. Example answers are provided on the Post-Activity Quiz Answer Key.

References

Copyright

Contributors

Supporting Program

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.