The School Board Chatbot: Use this tool to ask questions about what happened at recent school board meetings. You can ask, “Did Laguna Prep come up at the last meeting?” or “Have there been discussions about school closures at recent meetings?” The tool covers 69 districts that post their meetings to YouTube. An earlier version of the chatbot only covered 13 districts and required a ChatGPT account for access.
AZ Education Answers: This is an AI-powered tool that answers users’ questions about K-12 education in Arizona. The tool prioritizes reputable sources in its responses and includes citations that take you to the original source link. You can ask questions such as, “I live in west Mesa and can’t decide where to send my kid to school. What school has the best English proficiency scores?” The tool will even make charts for you to facilitate comparisons.
Mitul Balamurugan, a master’s student at Arizona State University’s Ira A. Fulton Schools of Engineering, built these two tools. They are powered by OpenAI’s technology.
Stress-testing the School Board Chatbot
Methodology
I asked the chatbot questions about recent Scottsdale Unified School District board meetings, including whether anything had been said about the school security or safety budget and whether security issues had come up in the district. I reviewed the bot’s citations to verify answers. I also searched the transcripts for anything the bot might have missed.
What worked
In general, I was very happy with how the chatbot performed. It gave short responses that answered my questions. It was also honest when the meeting transcripts couldn’t answer a question.
Issues
It had a few challenges.
Halfway through an answer about new school resource officer and school safety officer roles, the bot started thinking out loud: “On October 4? Wait date?” It appeared to be second-guessing which meeting the information came from. It ultimately, and correctly, said the grant funding for those positions came up at the June 23 meeting.
One answer was arguably incomplete. In response to my question about whether security issues had come up at recent board meetings, the bot said the meetings addressed security mostly in terms of planning, staffing and prevention.
But it left out a policy the board reviewed at the Sept. 8 meeting: a new state law lets the board authorize the superintendent to change weapons detection systems without board approval, as long as he reports to the board within a specified time period.
Stress-testing AZ Education Answers
Methodology
To evaluate AZ Education Answers, I asked questions about specific school districts as well as more general questions. I asked how much Mesa Public Schools spends on school security, whether security systems had recently failed in public schools, which districts face the worst budget problems, which schools have the highest share of students eligible for free or reduced-price lunches, whether more students depend on free and reduced-price lunch than ever, and which districts struggle most to keep teachers.
I checked the bot’s answers against its sources, sometimes opening source links by hand and conducting a manual fact check. Other times I used a different AI tool, Claude, to compare excerpts from the bot’s responses with text or data from the sources.
What worked
It performed well. The excerpts I checked always matched the content from the sources. The bot relied on credible sources, including information from the Arizona Auditor General’s office, the Arizona Department of Education, the Annie E. Casey Foundation’s Kids Count data center and the Arizona Food Bank Network. It also showed excerpts from sources so a reader could easily verify output without having to visit a website. It was careful not to overstate its findings. In its answer on free and reduced-price lunch, it warned that more students using the program doesn’t necessarily mean a rise in poverty.
What didn’t work
It had a couple of issues.
I asked the bot how rates of student eligibility for free and reduced-price lunch have changed in recent years. Five of the 19 sources provided were never cited in the answer. When I took this concern to Balamurugan, who developed the bot, he said that on the back end there is a verification AI agent that reviews answers before they are shown to the user. If it decides a sentence is not supported by a source, it removes the sentence but retains the source, as the source may have provided additional context for the broader answer. The source is also retained as an additional resource for users.
For this same question, there was a section titled “How Arizona compares with the country.” I observed that it had the national share of students eligible in 2022-23 but not the comparable Arizona figure for that year, so it failed to make the comparison the title promised.
I also noticed that one link to an Arizona Department of Education data page would not load.
Try it yourself
Overall, I’m really excited about these tools. I think they’ll be useful for parents trying to keep up with their school boards and make data-informed decisions about education for their children. Both are free at tools.thebeam.org.
We’re constantly working to improve these tools. If you see an issue or have any other feedback, email us at thebeam@asu.edu.