Lesson 4: Chatbots and Large Language Models (LLMs)

Getting Started with Code.org: Consider watching our Getting Started with Code.org video series for an overview of how to navigate lesson plans, setup a classroom section, and other important features of the Code.org platform. Each video also has a support article if you'd prefer to read or print instructions - click here to learn more.

Setup a Classroom Section: You can use a class section in Code.org to manage your students, view their progress, and assign specific curriculum - click here to learn more.

If you are using a learning management system, there may be additional steps to sync your classes with Code.org:

• Click here for steps to setup your classes with Google Classroom
• Click Here for steps to setup your classes with Clever

Become a Verified Teacher: Lesson plans and levels have additional resources and answer keys for Verified Teachers, which is quick process that verifies your position at an educational institution. Click here to complete a form and you should have access to verified teacher resources in ~1 business day.

Supports for Teachers: You can learn more about bringing AI lessons to your classroom through our AI 101 for Teachers series. AI 101 for Teachers is a free, foundational online professional learning series for any teacher and educator interested in the groundbreaking world of artificial intelligence (AI) and its transformative potential in education.

Prompt: Complete the following sentences with one word only:

• Thank ___.
• Please buy me a ___.
• Video games are ___.
• I enjoy playing outside in the Spring because it is ___.

Discuss: Compare your answers with your neighbors. Where did you pick the same words? Where did you pick different ones?

Discussion Goal: There are no right or wrong answers here; the idea is to collect a diversity of responses. This activity is meant to make students think about what words “probably” follow other words, and why. In this lesson, students will be learning about Large Language Models (LLMs), and how they guess likely next words based on context and what words have been close to each other in the past.

A snowball discussion protocol might work well here – give students 30 seconds to come up with their own answers, a minute to talk about them in pairs, then a minute for pairs to pair up with other pairs, until at 4 minutes you bring the whole class together.

Discuss: Why did you pick these words?

Discussion Goal: Students will likely have similar answers for the first sentence, but will diverge quickly with the other answers, such as which thing to buy. Emphasize how some prompts rely on the context of the sentence, like how students may have different opinions about video games, but they are more likely to give a positive answer to the last sentence because of the word 'enjoy’.

Optional Extension: Semantris Game: Semantris is a game developed by Google that emphasizes the same ideas in this warm-up: we naturally associate certain words with each other, which can be useful in a variety of situations. Semantris accomplishes this by generating a keyword, then the player tries to think of a word that's "close" to the generated keyword to score points. The game is engaging and fun. You could decide to play a few rounds as a class, or have students play for a bit on their own. If so, be careful of how much time you'd like to devote to this activity.

This prompt is one example of how we can use our prior knowledge to guess what words come next in a sentence. We’ve also seen that those words will not always be the same. In today’s lesson, we’re going to look at how computers can write text based on a similar principle – learning from a lot of text that has been written before in order to guess what words might follow each other.

Lesson Timing: This lesson is designed to fit within a 45 minute class period at a very brisk pace. As you look through some of the optional extensions or additional room for creativity and exploration, you may decide to extend this lesson over multiple class periods to fully explore all of the resources provided in this lesson.

Distribute: Pass out a copy of Video and Vocabulary Guide to each student

We're going to watch a video introducing the big ideas for today. These ideas can be complex so we're going to pause the video a few times and take some notes as we watch. We're going to start even before the video starts by talking about what a model is.

Vocabulary:

• Model - a computer program designed to make a decision

There are lots of ways to encourage students to retain vocabulary, and different methods will work well for different students. Encourage students to rephrase these definitions in their own words, draw pictures, or write sentences with these words. There is space on the handout for this.

Video: Show the How AI Works: Chatbots and Large Language Models (Part 1) video. At key points throughout (described below) we will be pausing the video to give students a chance to synthesize their understanding. They can keep track of their answers in the Video and Vocabulary Guide

The Shakespeare model in the video is guessing a possible next letter from a list based on what Shakespeare has said before. Let's do something similar, but with whole words, based on a written work of our choice.

Model: Open up Code Studio to the Written Voice Simulator widget.

Citing our Sources: The app is a very small version of Botnik, which uses a predictive text engine based on previous text from given tv shows, authors, video games, etc. This video provides more context around Botnik, starting around 1:20.

This widget lets us generate text similar to what we saw in the video, and we can certainly use it to generate random or non-sensical phrases. Our goal is to try and make it generate sentences or phrases that are more cohesive or even creative, similar to poetry or song lyrics. We're going to use this handout to help guide our exploration.

Distribute: Give each student a copy of the Written Voice Simulator Guide.

Code Studio: Have students log into Code Studio and begin interacting with the widget using the prompts from their activity guide.

After giving instructions, it can be helpful to check that your students understand what is expected of them – questions like “How long do we have to do this activity?” “What is expected at the end of it?” help to make sure the instructions are clear and the students are ready to get to work.

Circulate: As the students write their stories, encourage them by asking for their your thoughts and ideas, making sure they stay on task.

A link to the Written Voice Simulator User Guide resource is provided on the widget page. Students can access this for some of the advanced tasks on the worksheet that you may not have demo'd, like saving a story or loading a custom story.

There is a spot for students to share their stories with their neighbors and reflect on the similarities and differences. Make sure groups are doing this, or consider regrouping the class for a moment and ensuring all pairs get a chance to compare stories.

Regroup: Have students close out of the widget and re-focus as a whole class

Discuss: When you and your neighbor compared your "sad" stories on the bottom of the first page: What did these stories have in common? How were they different? Why?

Discussion Goal: Students have probably found that stories told by the same "voice" had more similar words, but choosing different words still meant different stories came out. This is to help students begin to have a framework around the Wrap Up Question, about whether LLMs are Creative and/or intelligent.

Language Model Games: The model that these programs build computes a “distance” between words – words that often follow each other, or are often used in place of each other, will have a shorter “distance.” This process can be simulated with several activities:

• Google’s Semantris is a game that visualizes what words Google’s model thinks are closer to each other. You play by guessing a word that is "closest" to a target word in a list.
• Semantle is a Wordle-ish variant that allows players to guess a secret word by finding words that are slowly getting closer in distance.
• Mind Meld is an improv game that has two people say two random words at the same time, then try to find the word that is most common or "between" these two other words. They repeat this process - saying both words at the same time - with the goal of both people saying the same word at the same time. This means they've successfully "closed the distance" from their starting words by successively finding words they had in common.

All of these are optional and not part of the lesson plan, but can be interesting or fun "brain breaks" during other lessons or downtime in class.

Digging Deeper: If you would like more information about the models behind Large Language Models, the YouTube channel AI Coffee Break with Letitia has a lot of great explanatory videos on this topic.

This activity may have left you with more questions than answers. How can just combining words based on probability lead to chatbots that look and sound like real people? Let's watch more of the video to learn more!

Video: Show the How AI Works: Chatbots and Large Language Models (Part 2) video. At key points throughout (described below) we will be pausing the video to give students a chance to synthesize their understanding. They can keep track of their answers in the Video and Vocabulary Guide

Video	Teacher
Pause video at 1:55 when they say “As you see, this works surprisingly well!”	Discuss: How does the system come up with better results? Discussion Goal: Use this question as a quick check for understanding. Better results come from context - in this case, all of the words that came before it.
Pause video at 2:58 when they say “this system is still just using random probabilities to choose words”	Discuss: What are the three ways a large language model is different from the letter-by-letter Shakespeare example? Discussion Goal: Use this as another quick check for understanding: Rather than looking just at Shakespeare, it looks at everything it can find on the internet. Instead of predicting letters, it predicts whole words or parts of words. There is a lot of human tuning to look for biased or dangerous responses.
Let the video end	Discuss: Based on what you've seen so far, do you think a large language model is "intelligent"? Discussion Goal: Students may share a variety of experiences, but based solely on the video and experimenting with the widget, it might be more accurate to say that LLMs generate text that sounds plausibly like a human might have written it, but may not actually be "intelligent"

Vocabulary: Review the following vocabulary from the video. Allow students to check their answers from the activity guide.

• Large Language Model (LLM): a general purpose form of AI that is trained on a very, very large body of knowledge.
• Context: Additional information that helps a model make a decision

(Optional) Additional Prompt: If it seems like students are still somewhat unsure about large language models, consider facilitating the following discussion to help tease out additional key details:

Discuss: How was the Written Voice Simulator also different from a Large Language Model?

Discussion Goal:

• An LLM is usually learning from more words, but the Written Voice Simulator was only using a single source at a time (like the Shakespeare model)
• An LLM has context and can look at entire sentences and paragraphs. The Written Voice Simulator only knew one word of context when it gives us options.
• The way humans were involved is different. With LLMs, the bot is ultimately deciding which word to choose while humans can help influence the choices it has. In the Written Voice Simulator, it's reversed - the bot influences what choices the humans can make for what word comes next.

Discuss: LLMs are learning from every word on the internet. What is a benefit of using the entire internet as input? What is a drawback?

Discussion Goal: Learning from every word on the internet gives Large Language Models a ton of information to learn from, but not all of that information is factual, and not all the wording will be kind or appropriate.

Also encourage students to consider how people with access to writing on the internet tend to be more privileged voices, and therefore voices from more marginalized communities may not be as prevalent, which only further marginalizes these groups.

Prompt: What’s an example of something you’ve interacted with that probably uses an LLM? What things can you imagine using an LLM for in the future?

This might be a good discussion to use the Whip Around Protocol.

Discussion Goal: This discussion connects the video to things the students have experienced – they have probably seen predictive text in emails or cell phones, and have probably heard of ChatGPT, for instance. Students may imagine using this technology for anything that generates text, including writing code or news articles. If students are struggling to find answers, you can provide the examples of predictive text, and / or demonstrate.

We'd love to learn more about the folks teaching these lessons and the classroom experience. Please let us know in this How AI Works Teacher Survey.

If you'd like to teach additional lessons from the How AI Works video series, click here to explore additional resources

If you'd like to dive even deeper into AI and Machine Learning, consider exploring our 5-week unit on AI and Machine Learning. Click here to view the unit and learn more

Most of the things that generate text today use a Large Language Model, but before we had the computing power to allow models to learn from the entire text of wikipedia, there were text-based AIs that were not based on Large Language Models.

Instead, these programs have rules for what sorts of text make sense at different times.

One of the most famous examples is Eliza, who was a computer program that was supposed to sound like a therapist that was invented in 1966. It was very impressive in its time. Let’s explore what Eliza is and is not capable of.

Option 1: Pull up an example Eliza displayed on the projector, and take responses from students for what to say to Eliza.

Option 2 (takes more time): Allow students to to play with Eliza on their own computers for a few minutes. Follow along in the (Optional) Eliza Activity Guide

Discuss: How did using Eliza feel different from using a Large Language Model? How does Eliza feel similar to using a Large Language Model?

Discussion Goal: Students will probably notice that the responses are more limited, though they may also notice that it can still feel real sometimes.

Real-World Examples: Chatbots like this have been used over time for things like customer service, scheduling appointments, or even emotional comfort. An overview of various chatbots and the challenges involves are presented in this TedEd Video on whether a computer can pass as a human.

When Eliza was first released, it needed warnings and disclaimers that it wasn't a real therapist and shouldn't be used for actual therapy. More recently, an Eating Disorder hotline tried to replace it's human helpline with an automated chatbot, but then needed to shut down their program when the chatbot gave inappropriate advice.