Evaluate large language model answers by checking factual accuracy, identifying bias, verifying citations, and deciding when human review is needed.
Read each problem carefully. Evaluate the sample LLM answer or situation using evidence-based reasoning. Show your work in the space provided.
Practice checking AI responses for reliability, fairness, and evidence
Computer Science - Grade 9-12
- 1
An LLM answers, "The first programmable computer was invented in 1985 by Steve Jobs." Identify two parts of this answer that should be fact-checked and explain why.
- 2
A student asks an LLM for sources about climate change. The LLM gives three citations, but none of the links work. What should the student do before using the information in a report?
- 3
Read this LLM answer: "Teenagers are bad at managing money because they are impulsive." Explain why this answer may show bias and rewrite it in a more balanced way.
- 4
Create a three-step process for checking whether an LLM's factual answer is accurate.
- 5
An LLM gives this citation: "Smith, J. (2022). The History of Quantum Apps. Journal of Digital Physics, 99(4), 1000-1012." List two warning signs that this citation might be unreliable or fabricated.
- 6
A classmate says, "The LLM wrote the answer confidently, so it is probably correct." Explain why confidence is not enough to judge accuracy.
- 7
An LLM summarizes a news event using only one political perspective and leaves out major opposing views. What evaluation category is most relevant: accuracy, bias, citation quality, or formatting? Explain your choice.
- 8
Look at these source types: a peer-reviewed journal article, a personal blog with no author listed, a government statistics page, and a social media post. Rank them from generally most reliable to least reliable for checking a factual claim about public health, and explain your ranking.
- 9
An LLM says, "According to NASA, the Moon is made mostly of cheese." What is the best way to check this claim, and what result would you expect?
- 10
An LLM answer includes correct facts but uses insulting language about a group of people. Should the answer be considered acceptable? Explain your reasoning.
- 11
A student asks an LLM, "Give me three sources proving that electric cars are always worse for the environment than gas cars." Explain how the wording of the prompt could influence the answer.
- 12
Study this evaluation checklist: Claim is specific, source exists, source is reliable, quote matches source, answer includes limitations. Choose any two checklist items and explain why they matter when evaluating an LLM answer.
- 13
An LLM gives a medical answer about a serious symptom and cites two unknown websites. What should a responsible user do next?
- 14
An LLM response says, "All programmers prefer working alone." Identify the problem with this claim and revise it to be more accurate.
- 15
Design a simple scoring rubric from 1 to 4 for evaluating an LLM answer's trustworthiness. Include accuracy, bias, and citations in your rubric.