Building With Public: This is part four of a five-part series of how I've been incorporating student voices and perspectives into the early product conceptualization for my new learning app, and why the age of AI demands that we not only build in public, but build with the public. You can read the intro post here.
In today’s case study, I'll share how elementary aged students at a career fair helped me conduct product test and build AI critical thinking skills at the same time.
The Setup: Career Day at P.S. 281
A few weeks ago, I was invited to attend a Career Day at a local elementary school to talk about what it's like to build apps for a living. This ended up being the perfect opportunity for real-time AI-powered learning experiments with kids.
As I've been building MuseKat, an audio-first learning companion for families, I've found that the spillover effects of learning about building with AI to be as meaningful as the product itself.
With each new feature, I learn about a new tool or workflow. (Often, I then turn back around and teach about that process, either to other students or industry professionals through corporate trainings.)
In this way, the act of building and iterating has taught me how to form more informed opinions about my own AI usage, including what to trust and what not to trust.
This has got me wondering: Could MuseKat catalyze learning two things at once—teaching AI literacy through real-world interactions you audit on your own terms? If so, what would that look like?
So when I was invited to attend a Career Day at a P.S. 281, a public elementary school in New York City, I decided to use this as a unique opportunity to do two things at once:
Actually get my product in front of kids IRL
Run some short-burst AI training experiments with kids
Here's how it went down.
Running Small Batch AI Games with Elementary Kids
The premise of MuseKat is simple: When you're out with your kid, just take a picture of whatever you're looking at, and receive a real-time, age-adjusted audio description of that thing.
Here's what the user flow looks like in the app itself:
You can use MuseKat museums and galleries, but you can also use it in gardens and parks, or even in your own living room. Obviously, the context matters, so I've been trying to figure out how to design the best AI-powered experience.
Here are some of the questions that come up from parents, teachers, and kids who beta test the app.
Questions for Beta Users of MuseKat
Some of the questions I ask early users include:
How good is the description of each image based on what you expect to hear?
How "fun" are the stories to listen to?
How's the personality of the character, Miko (ie: "silliness" level)?
How's the speed and pacing of the story?
How's the length of each story?
How's the level of complexity and detail, based on each age?
Needless to say, this career day offered the perfect testing environment.
Kids arrived in 20-minute intervals at the fair in clusters of similarly aged kids with each class. To set up a controlled environment, I set up a "mock art gallery" environment in the school gym with a few art posters from well-known artists.
As classes arrived, I adjusted the learning age of Miko's character for their age and got a new question ready. At my table, I set up a very basic workflow.
Setting Up a Kid-Friendly Workflow for Quick Iterative Evals
How I set up a short-burst evals process with elementary-aged kids in five minute intervals:
Introduced Miko, MuseKat’s meerkat narrator.
Framed the challenge: “Miko needs help deciding if her stories are too fast, too silly, or just right. Can you be her tester?”
Let kids choose the art: They picked a poster, studied it, and got ready for Miko’s story.
Played the story aloud using a Bluetooth speaker, tailored to their age group.
Collected feedback with stickers: Kids placed stickers on paper charts to vote on things like pacing or tone.
Here's an example of what some real-time, small-batch evals looked like for two questions:
How silly was the story?
How was the speed of the story?
What stood out to was how natural it was to have opinions about this technology. Even kids as young as five know how to point out a mistake in a description about something they can plainly see, or have opinions about the way information is presented to them.
When the problem was focused and the questions specific, students instinctively practiced critical thinking: Evaluating an AI meerkat's take on the world. In that way, gamifying the feedback process turned evaluation into a game. Even when the content didn’t fully hook them, the opportunity to "help Miko" kept them engaged.
The AI Native Generation
I was surprised by how, even at an elementary level, the "AI as technology" concept isn't foreign to these kids. Some of the kids had parents who also worked with AI; one fourth grader proudly told me about some of his coding projects. When someone asked where the name MuseKat came from and I confessed that I brainstormed it with an AI, she said, "Oh, that's talking to a robot, right?"
Overall, this was an incredibly fun and effective way of getting real time, quick-hit, iterative feedback on the product in a controlled, safe way. I also really appreciated being able to bring the kids into the iterative process with me. After all, this is the AI generation.
They will be the ones shaping the narrative for what is "good" vs. "bad" AI, and they already are learning about these tools (even if it's not always happening inside of a classroom).
I'm really curious about how to continue to find ways to introduce AI concepts to young learners, so if you have other ideas about what I might be able to do (or a group of students that you'd like to conceptualize an activity around), please let me know!
Special thanks to Hannah Skriloff for inviting me to join Career Day at P.S. 281 this spring! And thanks to Greg for getting me connected.
Help Me Design Phase Two: Getting from 0 to 1 has taught me a lot about what students expect to hear from AI-powered solutions, and now I've got a few better ideas for what's possible in future iterations. But I need your help. If you work with elementary-aged learners and are interested in being a design partner for phase two, let's talk.
It's Day 4 of my mini-series, "Building WITH the Public" In today's post, I share how elementary aged students at P.S. 281 helped me conduct product tests and build critical thinking skills at the same time Turns out that kids love games. And also telling you when you're wrong. Who knew? https://hardmodefirst.xyz/building-with-the-public-getting-elementary-aged-kids-to-evaluate-ai-in-real-time (shout out to @greg for getting me invited to a career day at a local elementary school!)