One Month of Daily Score Predictions From a Chatbot: My Honest Verdict

Every morning for a month, I opened a chatbot, pasted in yesterday's practice results, and asked the same question: what score am I on track for?

This is the promise at the heart of AI exam prep—"gokaku" culture taken to its logical endpoint. Not a study helper, but an oracle. Feed it enough data and it'll tell you your future. I wanted to know if that future-telling was real, or just confidence theater with a statistical costume.

The Setup (and the Rules I Gave Myself)

The exam: a standardized test I'd been prepping for on and off. The method: one timed practice section per day, results logged in a spreadsheet, fed to an AI with my full history each morning. Every day it returned a predicted score range plus a short rationale.

Three rules kept it honest:

1. No cherry-picking. I logged every result, including the 9 p.m. disaster sessions.
2. I asked the AI to state its confidence, not just a number.
3. I recorded my emotional reaction to each prediction, because I suspected that was where the real story lived.

What It Got Right

More than I expected, honestly.

It caught trends I couldn't see. Around day nine, the AI flagged that my accuracy was fine but my timing was collapsing in the final third of every section. I hadn't noticed—I was reviewing wrong answers, not the clock. It prescribed timed drills for a week. My late-section accuracy recovered within days. That alone felt worth the experiment.

Its long-range trajectory was solid. Its day-3 forecast and its day-27 forecast for my eventual exam score landed within a few points of each other, and both were close to my actual result. Given consistent daily data, the model clearly had signal.

The rationales were better than the numbers. The predicted score was useful; the why—"your error rate on inference questions triples when you've done math sections first"—was gold. That's diagnostic power most prep books can't touch.

What It Got Painfully Wrong

It was hostage to my input quality. One week I did practice sections on low sleep, distracted, phone buzzing. The AI's forecast cratered—not because my ability dropped, but because my data got noisy. It can only model what you show it. Garbage sessions in, panic out.

Confidence didn't scale with uncertainty. Early on, with barely a week of data, it delivered predictions with the same assured tone as its day-25 forecasts. A human tutor would've said "too early to say." The chatbot just... said a number. If you're prone to anchoring on a figure, this is genuinely dangerous.

And the big one: it almost caused my worst study week. Around day 14, a strong run of results pushed its forecast up sharply. I felt great. I got complacent, skipped two days of review, coasted. The next predictions dipped accordingly, and I spent the rest of the month clawing back ground I lost to my own smugness. The AI didn't predict that outcome—and no prediction system can, because the forecast changes the behavior it's forecasting. Economists call this reflexivity. Study tools mostly ignore it.

My Verdict: Instrument, Not Oracle

Here's how I'd frame it after 30 days: an AI score predictor is a dashboard, not a crystal ball.

Treat the daily number as a weather report—useful for planning, absurd to argue with—and the real value is in the trend lines and the rationales underneath. The specific score is the least reliable part of the output, even though it's the part your brain will grab first.

Three rules for anyone trying this:

• Demand the reasoning, not just the number. If the tool won't explain its forecast, don't trust the forecast.
• Weight the trend, discount the point estimate. Direction matters; decimals are noise.
• Never let a good prediction be an excuse to rest. The moment I coasted was the moment the forecast started lying to both of us.

AI study tools are getting scarily good at reading your readiness. The failure mode isn't the model. It's the student who mistakes a forecast for a guarantee. Ask it what you're weak at. Ask it what to do today. Just don't ask it for permission to slack off—because it'll give you one, and it'll look very convincing.