The next exercise is the whole job
Too easy, too hard, just right
An exercise that is too easy bores a child and one that is too hard defeats them. Learning happens in between.
Give a six-year-old a page of sums she already knows and she drifts. Give her one that needs an idea she has not met and she guesses, gets it wrong three times and decides she is bad at maths.
The right exercise is one the child will mostly get right, with effort. A good teacher finds it by watching. An app has to find it from what the child has answered so far.
BigKids is a tutor for maths and English for children of about five to ten. Its exercises ask for reasoning: explain, compare, find the one that does not belong. This piece is about the part nobody sees: how it picks what comes next.
A tutor that asks before it tells
The tutor answers a question with a question, and never writes the child's sentence for them.
A tutor that gives the answer teaches the child to wait for the answer. So BigKids' tutor asks first. It gives a hint, then a smaller hint, and tells only after the child has really tried.
In English the rule is stricter. A child reads a short text, learns a few words in it, finds the evidence for an opinion and writes one good sentence. The coach may question that sentence. It may not write it.
These are behaviours you can check by reading a session. The planner behind them is harder to check, which is why it gets a test of its own.
Children who do not exist
The planner is tested on simulated learners first, because a bad plan should not be tried on a real child.
The planner is tested on simulated learners. Each one has skills it knows and skills it does not, learns at its own pace and forgets. The planner teaches it for a run of days, and everything it does is recorded.
This is a test of the planner, not of children. It asks one thing: given a learner whose knowledge we can see, did the planner make sensible choices? Because the learners are simulated, the same test gives the same result every time, so a change to the planner can be judged fairly.
What it cannot show is whether a real child learns more. That is said again under limits.
The planner's mistakes, counted
Four things are counted: lessons out of order, repeats, frustration, and how much real reasoning a child meets.
The score is built from the planner's mistakes. A lesson given before the lesson it depends on is a mistake. The same exercise shown again too soon is a mistake. A run of failures that would make a child give up is a mistake.
Against those sits what we want more of: time spent near the right level of difficulty, and a healthy share of exercises that ask for reasoning and not recall.
One number combines them, so a change that trades one mistake for another does not look like progress.
Three quarters fewer lessons out of order
Lessons given before a child was ready for them fell by about three quarters.
The clearest result: far fewer lessons out of order. The count of lessons a simulated learner received before the one they depend on fell by about three quarters.
This is the mistake that hurts most. A child handed subtraction with borrowing before place value does not fail one exercise. They fail a week of them.
Over the same work the combined score moved from below zero to above it.
Fewer repeats, and a plan that reacts in the moment
Repeated exercises fell by about a quarter, and the plan now adjusts while a child is still working.
Repeats fell by about a quarter. A child who sees the same exercise again too soon is remembering the answer, not practising the idea.
The plan also reacts sooner. When a learner starts to struggle or to coast, what comes next changes during that sitting, not the next day.
A good teacher does exactly this. She does not wait until tomorrow to notice that today's page was wrong.
What did not move
Reasoning exercises are still only about a tenth of what a learner meets, and frustration is not yet low.
Two things are not where they should be. Exercises that ask for real reasoning are still only about a tenth of what a learner meets. The product is meant to be reasoning first; by this count it is not there yet.
And simulated learners still hit runs of failure more often than a careful teacher would allow. The remaining repeats are also high.
These are stated because they are the product's own aims, measured against itself.
What cannot be claimed
No child's learning has been measured. Every figure here is about a planner teaching simulated learners.
Nothing here says a child learns more. The learners are simulated. A simulated learner is a set of assumptions about how children learn, and the planner was improved against those assumptions.
The work covers one school year of maths in depth; the rest is scaffolded. The tutor's behaviour was checked on scripted sessions, not with children. There is no comparison with any other programme.
No result here has been reproduced by a third party.
What is public, and what is on request
The shape of the result is public. The figures and the app are on request.
BigKids is proprietary. What is public is the shape of the result: which of the planner's mistakes fell, by about how much, and which did not. How a learner is modelled, how the next exercise is chosen and what was changed are not published.
The measured counts are in the full results paper. If you teach, or build for children, and want to see it, ask.
Step 2: the paper