Finding 03
Finding 3 — Can we correct the computer's scores toward the teacher's?
In one sentence: Yes — for reading speed (WCPM), the computer's average error against teachers drops from -13.603 to 0.149 WCPM on recordings it has never seen.
What is being attempted
The system already measures reading speed directly: it counts the words a child got right and divides by the time taken. That measurement is honest but slightly off, because speech recognition mishears children. If it is off in a consistent way — say it always reads a little low — then that consistent error can be learned and subtracted back out.
That is all this step does. It does not decide how well a child reads. It corrects a measurement the system already made. The teacher stays the authority.
Speech model used: medium (the Experiment 1 winner). Recordings: 167, from 42 students.
How it was tested — decided before looking at any result
- Grouped by student. All of a child's recordings stay together, either in the practice set or the test set, never split. Otherwise the computer could recognise the child rather than learn the correction.
- 5 splits x 3 repeats, so no single lucky division of the data decides the answer.
- Every setting fixed in advance. Nothing was adjusted after seeing scores.
- One method named in advance as the answer:
ridge. Everything else is exploratory and is reported for interest, not as a claim.
Reading speed (words per minute)
Starting point — the uncorrected measurement is off by an average of -13.603 WCPM (typical miss: 15.037 WCPM).
| Method | Average miss | Systematic offset | Agreement range | Better than uncorrected? |
|---|---|---|---|---|
| (none — raw measurement) | 15.037 WCPM | -13.603 WCPM | -48.4 to +21.2 | |
| (guess the average) | 16.513 WCPM | +0.030 WCPM | -45.0 to +45.0 | yes |
linear | 10.038 WCPM | +0.141 WCPM | -30.1 to +30.4 | yes |
ridge ← primary | 10.043 WCPM | +0.149 WCPM | -30.1 to +30.4 | yes |
lasso | 10.045 WCPM | +0.154 WCPM | -30.1 to +30.4 | yes |
elasticnet | 10.092 WCPM | +0.193 WCPM | -30.1 to +30.4 | yes |
random_forest | 11.401 WCPM | +0.950 WCPM | -32.4 to +34.3 | yes |
hist_boosting | 11.532 WCPM | +0.797 WCPM | -32.4 to +34.0 | yes |
svr_rbf | 10.427 WCPM | -0.647 WCPM | -33.8 to +32.5 | yes |
knn | 11.67 WCPM | +1.211 WCPM | -32.0 to +34.4 | yes |
Average miss is how far off a typical recording is. Systematic offset is the bias — whether it is consistently too high or too low. The offset is the number this study is about: a method can have a similar average miss but a much smaller offset, and that is the improvement being claimed.
Does it correct each child, or just the average?
Each child, to a useful degree. R² = 0.536: the correction tracks real differences between recordings, not just the overall average. It moves stronger and weaker readings by different amounts, in the right direction.
Word reading accuracy
Starting point — the uncorrected measurement is off by an average of -19.87 % (typical miss: 19.993 %).
| Method | Average miss | Systematic offset | Agreement range | Better than uncorrected? |
|---|---|---|---|---|
| (none — raw measurement) | 19.993 % | -19.870 % | -55.2 to +15.5 | |
| (guess the average) | 4.02 % | +0.004 % | -12.3 to +12.3 | yes |
linear | 3.899 % | +0.289 % | -11.8 to +12.3 | yes |
ridge ← primary | 3.896 % | +0.288 % | -11.8 to +12.3 | yes |
lasso | 3.884 % | +0.288 % | -11.8 to +12.4 | yes |
elasticnet | 3.877 % | +0.281 % | -11.7 to +12.3 | yes |
random_forest | 3.963 % | +0.150 % | -11.8 to +12.1 | yes |
hist_boosting | 4.088 % | +0.337 % | -11.7 to +12.4 | yes |
svr_rbf | 3.512 % | +1.177 % | -10.2 to +12.6 | yes |
knn | 3.772 % | +0.857 % | -10.8 to +12.5 | yes |
Average miss is how far off a typical recording is. Systematic offset is the bias — whether it is consistently too high or too low. The offset is the number this study is about: a method can have a similar average miss but a much smaller offset, and that is the improvement being claimed.
Does it correct each child, or just the average?
Just the average. The correction scores R² = 0.001, which means it does essentially the same thing to every recording: it shifts them all by about the same amount. It removes the systematic offset — genuinely useful, since the raw measurement was badly skewed — but it does not tell the difference between one child and another on this measure.
Practical consequence: this correction is safe for reporting a class or cohort average, and not safe for ranking individual children. Say so plainly rather than letting the improved offset imply more than it should.
The headline result
For reading speed (WCPM), using ridge:
- Systematic offset: -13.603 → 0.149 WCPM (98.9% smaller)
- Typical miss: 15.037 → 10.043 WCPM
- Agreement range narrowed from 69.605 to 60.434 WCPM
Every number above comes from recordings the method had never seen, by students it had never heard.
The comparison that actually matters
Finding 01 showed a teacher's score is not a single true number — two teachers scoring the same child disagree by some amount. So the fair question is not "is the system perfect?" but "is the system within the range that two qualified teachers already differ by?"
| Comparison | Agreement within |
|---|---|
| Two teachers, same recording | ±9.6 WCPM |
| System vs teacher, after correction | ±30.2 WCPM |
| System vs teacher, before correction | ±34.8 WCPM |
Not yet. The system is about 3.2× wider than teacher-to-teacher disagreement. The correction fixed the systematic offset — the measurement is no longer skewed — but the recording-to-recording scatter is still larger than what two teachers produce.
This is the honest state of the work, and it points somewhere specific: the remaining error is scatter, not bias, so more calibration will not fix it. Narrowing it needs better underlying recognition — which is a speech-model problem, not a statistics problem.
Why the students were kept together
Each child recorded 4-8 passages. If those fall on both sides of the split, the method can recognise the child instead of learning the correction, and the scores come out flattering. Running the identical analysis and changing only that one choice:
| Measure | Grouped by student (used here) | Grouped by recording (leaky) |
|---|---|---|
| Reading speed — tracks individuals (R²) | 0.536 | 0.546 |
| Reading speed — average miss | 10.043 WCPM | 9.844 WCPM |
| Accuracy — tracks individuals (R²) | 0.001 | 0.184 |
| Accuracy — average miss | 3.896 % | 3.732 % |
Reading speed barely moves, but accuracy is where the damage shows: the leaky split reports 0.184 against the honest 0.001. It would look like the correction adapts to each child, when it does nothing of the kind — it is recognising children it has already heard.
That is why the split is grouped by student, and why the number reported here is the smaller one.
Honest limitations
- 167 recordings from 42 students is a small dataset. Treat differences between the methods as suggestive, not settled.
- Only the pre-named method is a claim. The others are exploratory — with ten methods on the table, one of them looking best can easily be chance.
- Reading speed here is derived from the teacher's word count and stopwatch time, not judged directly (see Finding 01).
- This corrects a measurement. It does not judge reading ability, and it cannot detect mispronunciation.
Generated by pipeline/calibrate.py.