Evaluate a scientific claim without overstating the evidence

Evaluate a scientific claim without overstating the evidence

Draft audience: Grade 10 intended, subject to teacher review of prerequisites and exact current subject, strand and term placement. This is original learning support, not an official curriculum allocation or marking scheme.

A claim can contain a useful idea and an unsupported promise in the same sentence. Consider: “Putting a cover on a cup always halves its water loss.” What counts as a cover? Over what interval? Does “halves” mean exactly 50% less, or merely less? In this paper investigation, separate those parts before judging the evidence.

The cups, conditions and masses are invented model records. No physical investigation, drinking-water test or practical equipment is required. The evidence exercise concerns evaporation, in which liquid becomes vapour. It does not certify water quality, recommend storage, or demonstrate that every missing mass in a real experiment must be water vapour.

Convert the claim into a question

Testable question: In the supplied 48-hour room-condition model, does the specified loose cover reduce the net water mass lost compared with an uncovered cup? An exact-halving claim additionally predicts a covered loss equal to one half of its matched uncovered loss.

Every cup starts with 100 g net water mass. Each pair uses equivalent cups, exposed opening dimensions, water source and starting time. One cup has the model’s standard loose cover and the other does not. Four pairs occupy different bench positions; which side is covered is assigned by a random draw within each pair. Temperature and airflow are stipulated comparable within a pair. These are modelling conditions, not observations of a real room.

The record weighs the whole cup-and-water assembly, including each cover and any water condensed on it, then subtracts its own dry assembly mass. Thus different cover masses are not mistaken for different water masses. The model excludes spills, leaks, additions and removal of water for sampling. Under those assumptions, the net loss is interpreted as water that left the assembly, principally by evaporation.

Calculate the quantity in the claim

Water loss is initial net water mass minus final net water mass. In pair 1, uncovered loss = 100 − 90 = 10 g and covered loss = 100 − 96 = 4 g. Their difference is 6 g. Comparing final masses alone without the initial masses and dry-assembly adjustment would not establish the same result.

Mean uncovered loss = 40 ÷ 4 = 10 g. Mean covered loss = 16 ÷ 4 = 4 g. The reduction relative to the uncovered mean is (10 − 4) ÷ 10 × 100 = 60%. This is a percentage based on the two mean losses, not the unweighted mean of the four individual percentage reductions. State which calculation you used.

Build claim, evidence and reasoning

Claim: the specified cover is associated with reduced loss in every supplied pair under these conditions. Evidence: covered losses are 4, 5, 3 and 4 g, below the matched uncovered losses 10, 12, 8 and 10 g. The mean reduction is 6 g, or 60% of the uncovered mean. Reasoning: matching the cups and conditions within pairs makes cover status the intended difference; reduced exchange with surrounding air is a plausible explanation. Air exchange itself was not measured.

The record does not support “exactly halves”. For pair 1, exact halving predicts 5 g, while the recorded covered loss is 4 g. Nor does it establish “always”: only one cover design, four pairs and one interval are represented. A strong conclusion keeps the supported reduction while removing the exact ratio and universal promise.

These small simulated records do not supply a population sample, statistical significance test or measurement-uncertainty budget. Do not call a 60% difference a guaranteed real-world effect. It is an accurately calculated feature of this teaching model.

Test a competing explanation

A different log places every uncovered cup beside an open window and every covered cup on a sheltered shelf. Now airflow and location change with cover status. A location difference could help explain different losses. The cover comparison is confounded even if its masses were recorded accurately. Repeating that arrangement many times would not by itself separate the two explanations.

A better comparison would match positions or distribute covered and uncovered cups across positions, keep the other stated conditions comparable and repeat. That does not guarantee every possible influence disappears, but it directly addresses the identified location explanation. The original paired model did this by keeping both cover conditions represented at each bench position.

Independent claim check

A new, separately simulated trial supplies two matched pairs. All four cups start with 100 g net water mass. The uncovered finals are 90 g and 88 g; the corresponding covered finals are 94 g and 94 g. Conditions are stipulated comparable within each pair, as before.

1. Calculate all four losses and the two mean losses.
2. Find the percentage reduction based on the mean losses, to one decimal place.
3. Evaluate “the new trial shows exactly the same 60% reduction as the first”.
4. Write a bounded conclusion and identify one further record that would help assess generality.

Explained answers and rubric

Uncovered losses are 10 and 12 g; covered losses are 6 and 6 g (2 points for both pairs). Means are 11 and 6 g (2 points). Relative reduction = (11 − 6) ÷ 11 × 100 ≈ 45.5% (2 points for denominator and result). The same-60% claim fails: the calculated mean-loss reduction is different (1 point).

A suitable conclusion is: “Both covered cups lost less water than their matched uncovered cups in this model, with a mean-loss reduction of approximately 45.5% over 48 hours” (1 point). A further matched comparison with other cover designs or independently repeated conditions would test how far the pattern extends (1 point). Explain that additional evidence could change the estimated effect rather than merely confirm a preferred answer (1 point). Maximum: 10 learning points.

Background: OpenStax on evaporation and phase transitions. The claim, assembly convention, masses, comparison and questions here are original. When reviewing an answer, check its denominator, comparison conditions and wording before rewarding confidence.