Lesson 2: Computer Vision

Getting Started with Code.org: Consider watching our Getting Started with Code.org video series for an overview of how to navigate lesson plans, setup a classroom section, and other important features of the Code.org platform. Each video also has a support article if you'd prefer to read or print instructions - click here to learn more.

Setup a Classroom Section: You can use a class section in Code.org to manage your students, view their progress, and assign specific curriculum - click here to learn more.

If you are using a learning management system, there may be additional steps to sync your classes with Code.org:

• Click here for steps to setup your classes with Google Classroom
• Click Here for steps to setup your classes with Clever

Become a Verified Teacher: Lesson plans and levels have additional resources and answer keys for Verified Teachers, which is quick process that verifies your position at an educational institution. Click here to complete a form and you should have access to verified teacher resources in ~1 business day.

Supports for Teachers: You can learn more about bringing AI lessons to your classroom through our AI 101 for Teachers series. AI 101 for Teachers is a free, foundational online professional learning series for any teacher and educator interested in the groundbreaking world of artificial intelligence (AI) and its transformative potential in education.

Display: Show the image on the slide to students.

(Click to expand and see larger)

Prompt: What animal do you see in this image? Write your answer on a sheet of paper

Discussion Goal: After a brief pause, ask students to share out. Students will likely notice one of two animals: either a duck or a rabbit. Once both options are said, students may "switch" back and forth in how they see the image - sometimes it's a duck, sometimes it's a rabbit.

Prompt: If this were an argument and you wanted to convince someone else that your animal is correct, what would you say? What features in the image help your argument?

Discussion Goal: This prompt is meant to help students identify the features of the drawing that their eyes noticed to help make that decision. Both arguments will likely identify the black dot in the center as the eye of the animal. Duck arguments will identify the left feature as the bill or mouth of the duck, whereas rabbit arguments will identify those as the ears. Emphasize that students are using specific features to make their decision (ie: eye + ears + mouth = rabbit).

Display: Show slide with additional optical illusions

Illusions: The illusions are:

• The Dress: Some folks see a blue dress; some folks see a gold dress; some folks see a variety of different colors. This picture went viral several years ago with debates over the color of the dress
• Young Woman vs Older Woman: Depending on how you view it, this image either looks like a young woman looking away off to the distance, or an older woman looking down. The chin and ears of the younger woman are the nose and eyes of the older woman.
• Dogs and Bagels: Not really an optical illusion - if you look closely, some of the images are of dogs and some are of bagels. At a quick glance, it's easy to miss which one is which.

We use our eyes and sense of sight to identify objects, and sometimes an image with identical features can be interpretted in several different ways. But what about a computer - what would it "see" if it saw one of this pictures? Is the dress gold, or is it blue? Is this a young woman looking away, or an old woman looking down? Are these a bunch of dogs, or a bunch of bagels? Today we're going to talk about how computers can "see" and detect images

Video: Show the How AI Works: How Computer Vision Works video.

Videos are used throughout the curriculum to spark discussions, supplement key concepts with additional explanations and examples, and expose students to the various roles and backgrounds of individuals in computer science.

While interacting with the video, turn on closed captioning so students can also read along as they watch.

To encourage active engagement and reflection, use one or more of the strategies discussed in the Guide to Curriculum Videos.

Prompt: How would you summarize how a computer "sees" something in an image?

Discussion Goal: Listen for students describing how it breaks an image into parts, then combines them together in layers to create more complex shapes. Students may rely on the shape visuals from the video, or on the dog and muffin visuals.

Vocabulary: Display the slide with vocabulary for today's lesson

• Algorithm - a series of steps to solve a problem or make a decision
• Network - A collection of items connected together. Following these connections can represent the steps in an algorithm, like a flowchart.

Today we're going to create our own image recognition algorithm. An algorithm is a series of steps to solve a problem or make a decision. We're going to create an algorithm that uses a network to make a decision, just like in the video

Display: Show the slide with a scoreboard from a baseball game

We're helping a sports arena improve its scoreboard. We want to train a camera to detect what the scores are so it can be broadcast to a radio receiver. For this to work, it needs to detect which number is displayed on the scoreboard. Each number is represented with one of these displays, which uses 7 segments to create images.

Prompt: How would you represent your birthday using these numbers? Sketch your answer on a sheet of scratch paper

As students are sketching their responses, begin distributing the activity guide.

Distribute: Pass out the Computer Vision Network activity guide

Going Big: Consider printing out copies of the 7-segment manipulatives and having students work on whiteboards, poster paper, rolls of butcher paper, or even setting down individual items and connecting them with string or yarn.

Group: Group students into pairs to work together

Display: Show the slides giving an overview of this activity

Examples: Several examples are provided in this document: [Example] Computer Vision Network. Consider reviewing these examples before teaching the lesson to see what kinds of networks students may create.

Do This: Have students work in pairs to construct their network. The rules for the network are displayed on the slides.

Circulate: Monitor students as they complete this task, observing the different strategies students attempt to "move through" the layers. Ensure partners are collaborating together. Look for groups that are creating unique or novel networks , especially when different from some of the examples in the presentation.

Why are they out of order? Students may ask why the numbers are out of order on the right side of the activity guide. They're ordered by "number of segments" - 8 has the most active segments, and 1 does not. When it comes time to make a decision, if multiple numbers are lit up (like 9 and 7 and 4 and 1), then the top number is always chosen. This helps reduce the complexity for students as they make their algorithms - otherwise, an 8 could cause havoc in their network!

Exemplar: A few example networks are provided as an Answer Keys. These can be used as reference, or to quickly inspire students who may be stuck. They can also be used in class discussions to emphasize that there are multiple ways to solve this task.

Regroup: Have students regroup

(Optional) Gallery Walk: Consider having students perform a gallery walk to visit other networks, and have students reflect on how the networks are similar or different from their own. This is an especially engaging activity if students have "Gone Big" and drafted their networks on whiteboards or poster paper.

Now that we have our algorithm, lets test it! We're going to do 2 examples. I'm going to show a slide that has certain inputs covered with post-it notes. This means the sensor detected that particular strand was lit up. Follow your network to determine which number it is!

Display: Display the slide with the first example

(Click to expand)

Use the network, not your eyes: Students may comment that, once they have the inputs on the left-side, they already know what number it is - they can "figure it out just by looking at it". Remind students that the goal isn't for us to figure it out with our eyes and brain, but for a computer to figure it our with its sensors and algorithm. This can be useful for images that are incredibly large or complex (like satellite images of crowded markets) that we might not be able to interpret, or situations where we need to interpret images incredibly quickly which computers can do faster than we can.

You can also ask students to consider all of the steps that happened in nanoseconds once they saw these inputs - their brain needed to combine all of this information together to know what number it was. This network is doing the same thing, acting like the computer's "brain" to make a decision once it sees the inputs.

Circulate: Monitor students as they trace the path in their network

Prompt: Are there any changes you want to make to your network? You have a minute to do so!

Display: Show the slide with the second example

(Click to expand)

Optional Extensions:

• Consider having students trade networks with another pair, then giving students a third example and seeing if they can trace a neighbor's network
• Consider inviting students to trace the solution in front of the class, explaining how their network works
• Consider giving students the number 8 as an input, which will activate every single space in their network, then prompt them to consider how they can make a decision when this happens
• Consider telling students that one of the segments if "burnt out", and then giving them a pattern that doesn't exactly match with any number (for example: an 8, but the bottom segment is missing, so it looks like an A). What number will their network identify it as?
• Consider asking students to "time themselves" tracing through the network, then asking students to notice if some networks are "more efficient" than others and take less time to complete.

Prompt: What do you imagine a network could look like that could identify the image below? What do the inputs look like? What do some of the middle layers look like?

(Click to expand. Image source: Wikimedia Commons)

Discussion Goal: This is an open-ended prompt for students to apply the skills they practiced in the lesson to a new situation, but they still may struggle adapting to a new context - it's okay for students to struggle and "talk it out" as they work through this together. Students should identify the individual left and right flag positions as inputs, and the combination of the two as the output letter. The middle layer could look like two flags combined together - for example: left flag straight left + right flag straight right = R. This example network would be simpler than the class example because there are less components to build together.

Semaphores: This example comes from a historical way for ships or outposts to communicate over long distances, before radio or telephone were invented. For example, if two ships on the sea wanted to communicate, one person would wave flags according to this system and someone on another ship would record the words to piece together the message.

(Optional) Extension: Consider watching the What a Neural Network Sees video from the Experiments with Google page. The short video gives a live example of how image classification works as a neural network, where you can see the inputs and hidden layers.

We'd love to learn more about the folks teaching these lessons and the classroom experience. Please let us know in this How AI Works Teacher Survey.

If you'd like to teach additional lessons from the How AI Works video series, click here to explore additional resources

If you'd like to dive even deeper into AI and Machine Learning, consider exploring our 5-week unit on AI and Machine Learning. Click here to view the unit and learn more