Science: Comparing Simulation Models and Real Data
Evaluating how well models match observations
Science: Comparing Simulation Models and Real Data
Evaluating how well models match observations
Science - Grade 9-12
- 1
A climate simulation predicts an average July temperature of 28.4°C for a city. The measured average July temperature is 27.1°C. Calculate the model error using simulated value minus observed value, and explain what the sign of the error means.
Use error = simulated value - observed value.
The model error is 28.4°C - 27.1°C = 1.3°C. The positive sign means the simulation overestimated the real July temperature by 1.3°C. - 2
A population model predicts that a fish population will increase from 1,000 to 1,600 in five years. Field surveys show the population increased from 1,000 to 1,250. Describe one way the model matches the data and one way it does not match the data.
The model matches the data because both show that the fish population increased over five years. The model does not match the data because it predicts a much larger increase than was actually observed. - 3
The table shows predicted and observed nitrate levels in a stream after a storm. Day 1: predicted 6 mg/L, observed 5 mg/L. Day 2: predicted 9 mg/L, observed 8 mg/L. Day 3: predicted 7 mg/L, observed 9 mg/L. Day 4: predicted 4 mg/L, observed 6 mg/L. On which day did the simulation have the smallest absolute error?
Absolute error ignores whether the prediction is too high or too low.
The smallest absolute error occurred on Day 1 and Day 2. On both days, the absolute error was 1 mg/L. - 4
A disease spread simulation assumes that every person has the same number of daily contacts. Real contact tracing data show that some people have many more contacts than others. Explain how this assumption could affect the simulation results.
Think about whether all people contribute equally to spreading an infection.
This assumption could make the simulation less realistic because disease spread often depends strongly on people with many contacts. The model might underestimate rapid outbreaks caused by highly connected people or misrepresent how infection moves through a community. - 5
A physics simulation predicts the motion of a falling ball but ignores air resistance. In a real experiment, the measured falling time is slightly longer than the simulated falling time. Explain why the simulation and real data differ.
The simulation and real data differ because air resistance slows the real ball as it falls. Since the simulation ignores that force, it predicts a shorter falling time than what is measured in the real experiment. - 6
A model of plant growth predicts heights of 12 cm, 18 cm, 24 cm, and 30 cm over four weeks. Real measurements are 11 cm, 17 cm, 22 cm, and 27 cm. Does the model show the correct trend? Does it accurately predict the values? Explain.
Separate the pattern over time from the exact numerical match.
The model shows the correct trend because both the predicted and real plant heights increase each week. It does not perfectly predict the values because the simulated heights are higher than the real measurements in every week. - 7
A student says, "The simulation is wrong because one data point does not match the real data exactly." Explain why this statement is too strong.
The statement is too strong because scientific models are simplified representations and are not expected to match every data point exactly. A model can still be useful if it captures the main pattern, stays within reasonable uncertainty, and helps make testable predictions. - 8
A graph compares simulated ocean pH and measured ocean pH from 2000 to 2020. Both lines decrease over time, but the simulated line decreases faster than the measured line. What does this suggest about the model?
Compare both the direction of change and the steepness of the lines.
This suggests that the model captures the overall direction of change but overestimates the rate of ocean acidification. The model may need adjustments to its assumptions, input data, or chemical processes. - 9
A wildfire spread simulation matches the real fire boundary well on flat land but poorly in steep terrain. Identify one likely missing or oversimplified factor in the model.
One likely missing or oversimplified factor is the effect of slope on fire spread. Fires can move faster uphill because heat rises and preheats vegetation above the flames. - 10
A simulation predicts that a solar panel will produce 5.2 kWh of energy on a certain day. The real panel produces 4.6 kWh. Calculate the percent error using absolute error divided by observed value times 100.
Use percent error = |simulated - observed| ÷ observed × 100.
The absolute error is 0.6 kWh. The percent error is 0.6 ÷ 4.6 × 100, which is about 13.0%. - 11
A weather model predicts a 70% chance of rain for a region, but no rain falls at one school in that region. Explain why this single observation does not necessarily prove the model failed.
Consider the difference between a probability forecast and a yes-or-no prediction.
This single observation does not necessarily prove the model failed because a 70% chance of rain is a probability, not a guarantee. Also, the forecast may apply to a larger region, while conditions at one school may differ from nearby locations. - 12
A simulation of traffic flow predicts average speeds accurately during normal weekdays but poorly during a holiday weekend. What does this reveal about the limits of the model?
This reveals that the model may work well under conditions similar to the data or assumptions used to build it, but it may not generalize to unusual situations. Holiday travel patterns may require different inputs or additional variables. - 13
A scatter plot compares simulated values on the x-axis with observed values on the y-axis. Most points fall close to a diagonal line where simulated value equals observed value, but three points are far away. What should a scientist investigate next?
Outliers can reveal errors, unusual events, or weaknesses in a model.
A scientist should investigate the three outliers to determine whether they were caused by measurement error, unusual real-world conditions, incorrect model assumptions, or missing variables. The scientist should also check whether the overall model performance is still acceptable. - 14
A lake temperature model was built using data from summer months only. When tested on winter data, it performs poorly. Explain why the model may not work well in winter.
Models are often most reliable under conditions similar to the data used to create them.
The model may not work well in winter because it was trained or designed using only summer conditions. Winter has different sunlight, air temperature, mixing, ice cover, and weather patterns, so the model may be applied outside the range where it is reliable. - 15
You compare a simulation with real data and find that the model consistently predicts values that are 2 units too high at every time point. Is this mainly random error or systematic bias? Explain how the model could be improved.
This is mainly systematic bias because the predictions are consistently too high in the same direction. The model could be improved by recalibrating a parameter, checking the input data, or adding a correction factor after identifying the cause of the bias.