Lesson 7: Retrieval - Practice

Getting Familiar: These are the levels students will complete throughout the lesson. We recommend briefly exploring the levels yourself before reading the rest of the lesson plan. Some levels are meant to be completed independently by students, but others are meant to motivate class discussion or lead to moments of direct instruction in the slides. Very rarely should students be working through levels completely self-paced.

Teacher Exemplars: Verified teachers can access exemplar responses and "what success looks like" tips for each level in the Links section of the lesson plan. If the resource doesn't appear for you, you may need to become a verified teacher - learn more about this process here.

Teaching with Student Responses: Certain level types are designed to be leveraged with our Student Summary feature, specifically: Predict & Discuss and Mid-Lesson Check-In levels. You can read more about how to leverage student voices during these levels in the Teaching Guide - Incorporating Student Responses in Lessons

Teaching Tip: Building Fluency - This lesson is a strong candidate to let students work at their own pace as they build fluency and problem-solve on their own. Consider regrouping the class only if there seems to be a larger misconception that needs clarifying. Otherwise, it may be better to spend class working with students who need additional support based on yesterday’s lesson and the check-in level. Verified teachers can review exemplars and strategies for each level as a resource in the lesson plan.

Teaching Tip: Less Writing, More Copying This lesson implicitly encourages students to spend their time searching for sources and copying directly from them, rather than trying to write retrieval information themselves (which is mostly what they did yesterday). This correlates to how retrieval works in a more realistic sense with actual language models: instead of looking at individual words or statements, modern large-language models ingest entire documents or manuals or websites in order to augment their knowledgebase. For that reason, try to encourage students to put their energy into finding quality sources (like websites or books) to copy from, since this is the more realistic behavior of language models outside of this lab.

Teaching Tip: Connecting to Previous Lessons - When teaching about hallucinations, one possible direction that discussions may have gone is "misinformation" and ensuring that accurate information is available when needed. One place where this conversation appears frequently is in the context of elections, which is the context of Lesson 7 in this lesson. If you and your students spent time discussing misinformation during previous lessons, consider highlighting the role of retrieval in helping to ensure accurate information from chatbot sources. If your class is interested in continuing to explore the intersection of AI and elections, consider exploring the AI and Elections lessons from Day of AI

Ethics Opportunity: Real-World Connection - Level 5 is framed a real-world scenario where an airline used a customer-service chatbot to answer questions and the chatbot gave incorrect information to a customer. This scenario has a relatively happy ending, since the customer was able to sue the airline to make sure they honored the chatbots response (you can read the article here). However, consider asking students to imagine other scenarios where a chatbot could give incorrect information and imagine the consequences. Encourage students to consider how not every consequence has the same "weight" or impact - for example, a chatbot that tells you the wrong time for a movie may be less impactful than a chatbot that tells you the wrong time for an airline flight needed to attend an important event. A later lesson in this unit - Lesson 13: User Testing - will address these types of impacts more deeply and ask students to decide whether a chatbot should be used in certain situations at all.

Assessment Opportunity: Formative Assessment - This scenario touches on several ways that retrieval can be useful: accurate information to avoid bias based on cultural misrepresentations, specialized information from a master artist, and current information since modern practices may be different from historical ones. Look for responses that identify at least one of these aspects in the scenario and provide specific suggestions of how retrieval can help counteract these issues. After students answer this question and as they work on the remaining levels, try to check how students answered this question so you can decide if you need to make this point directly with students. Check the Generative AI - Teaching Guide for Student Responses for more information about how to use Code.org’s teacher dashboard to view this data and potentially differentiate your instruction with students.

Assessment Opportunity: Assessing Retrieval - This level can be used to assess students understanding of retrieval so far. We recommend checking just this level as opposed to every level in this lesson.