I Let a Chatbot Grade My Essays for Three Weeks. My Teacher's Score Caught Up Eventually
Photo: Azikahkhan, CC BY-SA 4.0, via Wikimedia Commons
There's a specific agony in essay-based exams: you write the thing, you hand it over, and then you wait. Days, sometimes a week, before anyone tells you whether your argument held up. By the time the feedback arrives, you've already forgotten what you were trying to do in paragraph three.
So I ran an experiment. For three weeks, every practice essay I wrote—for AP English-style prompts, an SAT Writing section, and one law-school-style timed response—went to a chatbot before it went to a human grader. The question wasn't whether AI could write my essay (it can, and you shouldn't let it). The question was whether AI could grade my essay well enough to make my next draft better.
The Setup: Same Essay, Two Graders, One Rubric
My protocol was simple. I'd write the essay under timed conditions, paste it into the chatbot along with the official scoring rubric, and ask for a score plus a breakdown. Then I'd submit the same essay to my teacher or score it against an official answer key.
I did this eleven times. Here's the scoreboard:
- Exact score match: 5 of 11 essays
- Within one point: 4 of 11
- Off by two or more: 2 of 11
That's not perfect. But it's close enough that the feedback loop changed everything. Instead of waiting a week for comments, I was revising within minutes—while the essay's argument was still fresh in my head.
What the AI Graded Surprisingly Well
Thesis strength. This was the bot's superpower. It consistently flagged the difference between a thesis that states a position and one that makes a claim worth defending. My weakest drafts got caught in the first two sentences instead of the final grade.
Structural logic. Ask it to map your paragraph function—"label what each paragraph does, not what it says"—and it will show you exactly where your argument stalls. In my case, it caught that my third paragraph was evidence without analysis in six of eleven drafts. Six! I had a habit and no idea.
Rubric alignment. When I pasted the actual scoring guidelines, the bot scored against them reasonably faithfully. It's essentially a checklist machine with good reading comprehension, and rubrics are checklists.
Where It Fell Flat
Honesty time, because the misses matter more than the hits.
Tone was a blind spot. In one essay, I wrote a deliberately dry, ironic opening—the kind of thing a seasoned AP reader would reward. The bot called it "underdeveloped" and suggested I "add enthusiasm." A human grader gave it my highest score of the cycle. AI has a bias toward earnest, signposted writing, which is exactly the writing that reads as generic to experienced graders.
It graded what it saw, not what the rubric rewards. On one response, the bot docked me for a conclusion it found "abrupt." The rubric didn't even evaluate conclusions separately. Without the rubric pasted in, the bot grades like a thoughtful reader, not like the test. Always feed it the rubric. Always.
It flatters, then it folds. Early drafts got gentle scores and gentle critiques. When I added "be a harsh grader; find the three weakest elements and rank them," the feedback got dramatically more useful. Politeness is the default. Override it.
The Framework: From Critique to Revision Checklist
The raw feedback isn't the win. The win is converting it into a repeatable pre-submission routine. Here's the four-step version I settled on:
- Score first, explain second. Ask for the rubric score before any commentary. This prevents the bot from reverse-engineering a justification for a vibes-based number.
- Demand the top three. "List the three weakest elements of this essay, ranked by impact on the score." Not five suggestions. Three, ranked.
- Translate to yes/no checks. Turn each critique into a binary question for your next draft: "Does every body paragraph contain analysis after the evidence? Y/N."
- Re-grade the revision. Paste the revised draft back and ask whether each fix landed. Closing your own loop is where the score improvement actually happens.
By week three, my pre-submission checklist had six items, four of which came directly from AI critiques of my earlier drafts. My teacher's scores on my final three essays were the highest of the entire cycle. Coincidence? Maybe. But the revision speed was real either way.
The Verdict for Test Day
A chatbot is not a grader. It's a first reader—fast, tireless, occasionally tone-deaf, and weirdly good at catching the structural habits you can't see in yourself. Use it to compress the feedback loop from days to minutes, then let the human grader catch what it can't.
In Japan, exam culture has a word for the whole game: gokaku—passing. The point was never perfect feedback. The point was passing smart. A bot that's right about your thesis eight times out of eleven, instantly, beats a perfect grader you see next Tuesday.