Lesson 9: Saving Models in AI Lab
45 minutes
Overview
Students complete the full process of training and saving a model, then importing into App Lab. For the first time, students are able to choose the label they would like to predict and spend time deciding the features they will use to help predict their label of choice. Students also create a model card for their models in order to save them and import it into App Lab
Question of the Day: How can I use Model Cards to document my decisions when training a machine learning model?
Assessment Opportunities
-
Use data visualizations and feature iteration to train machine learning models
Each level in this lesson requires at least 75% accuracy before continuing. Students who have completed each level will have met this objective.
-
Create model cards in AI Lab to save machine learning models
Completing level 4 satisfies this objective.
Standards
BI-3 - Computers can learn from data
3-A-iii - Nature of Learning - training a model
- 3-A-iii.6-8 - Train and evaluate a classification or prediction model using machine learning on a tabular dataset
3-C-i - Datasets - feature sets
- 3-C-i.3-5 - Create a labeled dataset with explicit features of several types and use a machine learning tool to train a classifier on this data.
- 3-C-i.6-8 - Create a dataset for training a decision tree classifier or predictor and explore the impact that different feature encodings have on the decision tree.
AP - Algorithms & Programming
- 2-AP-19 - Document programs in order to make them easier to follow, test, and debug.
DA - Data & Analysis
- 3A-DA-12 - Create computational models that represent the relationships among different elements of data collected from a phenomenon or process.
- 3B-DA-05 - Use data analysis tools and techniques to identify patterns in data representing complex systems.
Agenda
Objectives
Students will be able to:
- Create model cards in AI Lab to save machine learning models
- Use data visualizations and feature iteration to train machine learning models
Preparation
- Review the Code Studio levels before the lesson to be familiar with investigating columns in AI Lab and training their model
- Print copies of the activity guide for each student
Links
Heads Up! Please make a copy of any documents you plan to share with students.
For the teachers
- Saving Models in AI Lab - Slides
For the students
- Model Cards in AI Lab - Video
- Selecting a Label - Resource
- Zoo Models - Activity Guide
Teaching Guide
Warm Up (5 minutes)
Journal
Prompt: People often post information about themselves on social media like:
- Their name and age
- The school they go to
- Where they live
- Movies they like
- Restaurants they’ve been to
- Their birthday
If all of this data was collected, what is one thing you think a machine learning model could try and predict from this data?
Discussion Goal: Answer will vary, but could include local movie or restaurant recommendations, names of people with similar interests, how much money they spend, a birthday present they should buy. There is no single correct answer to this prompt - instead, focus on the choices that students are making about what features to emphasize and what they want to predict in the first place. Even with the same set of data, students may make dramatically different choices about what a model could look like with this data.
Remarks
This is a common situation in machine learning - people have a dataset that they’ve collected, and then they have to decide what they want to predict from the data. And there are a lot of decisions that go into training a model like this! Today, we'll see how you can use model cards to help keep track of all the decisions that go into training a machine learning model
Question of the Day: How can I use Model Cards to document my decisions when training a machine learning model?
Activity (35 minutes)
Model Cards in AI Lab (5 minutes)
Video: Show students the Model Cards in AI Lab video, which summarizes Model Cards that students saw yesterday and shows how they can be saved in AI Lab.
Black Boxes: This video references how AI models are sometimes referred to as "black boxes" and how difficult it can be to uncover how a model works once it's trained. Examples of this can be seen in the ProPublica Breaking the Black Box journalism series. This is a 4-part series that includes short videos and additional resources about different investigations into machine learning algorithms. One of the videos on pricing calls out how online products can have different prices depending on the zip code of the user, which is a common issue involving bias in machine learning models where a zip code can be used as a proxy for race or socio-economic status.
These videos aren't intended for classroom use, but you may decide to share some of these videos with students to continue conversations around bias in machine learning and the need for transparency via Model Cards.
Code Studio: Have students log into Code Studio. The first three levels represent the zoo dataset in AI Lab where students can train a model.
Distribute: Pass out the Zoo Models activity guide to each student.
Display: Display the slide with the scenario for today’s task, where we've been hired by a zoo to create different apps for customers to interact with around the zoo. Have students read the slide aloud.
Zoo Predictions (20 minutes)
Do This: Have students complete the first three levels of Code Studio. Students will train and save 3 models using the Zoo dataset. They will also record some of their responses on the activity guide.
Circulate: Check in with students as they train their models. Students should spend time iterating on the features they are selecting. The "Previous Results" section of the results screen can be helpful for students to find feature combinations that will work with their label. Students should also use the activity guide to practice filling in their model card.
Selecting a Label: This is the first time students can choose the label for their model. The process is identical to selecting features, but be aware that some students may be initially surprised by this extra step in the process. Students can access a resource in the Help and Tips section of Code Studio that explains how to select a label in AI Lab
Revisiting Past Screens: Let students know it’s okay to go back and revisit past screens to improve their model or make different decisions - this is an important part of the process and something students should practice in this level. Students may find themselves starting to move fluently between the testing, feature, and label screens as they refine and improve their machine learning model.
Level 2 - Where Are The Tables? In level 2, students predict the type of animal that they are looking at (like mammal, reptile, etc). This column has too many possible values, and so many data visualization tables won't appear. This is intentional, since it challenges students to use feature iteration and the Previous Results screens intentionally to find features that work. They need to trust that, if there are patterns in the data, AI Bot will find them and produce an accurate model even when there's too much data to visualize. This also prepares students for more realistic examples where there are typically more than 4 or 5 values for a feature.
Importing Your Model (10 minutes)
Code Studio: Have students continue to the next level. This is an App Lab level where students can import their trained model and begin customizing their app.
Circulate: Help students import their app and verify that the model works correctly before continuing to level 3. Students should use the remaining time to customize their app, reinforcing the skills they learned in the last lesson on how to make their app more user friendly.
Focus on AI Lab, not App Lab: Even though students may be excited about using their models in App Lab, the larger goal of this lesson is to practice training accurate models in AI Lab and using Model Cards to document those decisions. These final levels are provided to complete the full cycle of importing a trained model to App Lab, but students are not expected to complete a fully-designed app in the time they have. Instead, tomorrow's lesson focuses entirely on App Lab which is where students can create an app more fully.
Wrap Up (5 minutes)
Journal
Reflection: Why is a model card useful when creating a model in AI Lab? How is a model card useful when creating an app in App Lab?
Students can answer this on their activity guide or as an exit ticket.
Discussion Goal: Students may relay similar points as the opening video - that a model card helps document decisions and provide transparency, and it can be used in App Lab to help add to the design - but their responses should be more grounded in their experiences in this lesson, especially after selecting their own label and training their own model.
Lesson Feedback
Find a typo? Were some of the directions unclear? Have a suggestion for how to improve the flow of this lesson? We'd love to hear it! Please use the links below to provide feedback on this lesson.
This work is available under a Creative Commons License (CC BY-NC-SA 4.0).
If you are interested in licensing Code.org materials for commercial purposes contact us.
