Painted storybook scene at a wooden kitchen table in a cozy Mat-Su Valley log cabin on a late-October afternoon: a boy sets a smooth river stone on one pan of a simple wooden balance scale while his older sister points at the scale and their mother looks on with a mug in hand. A small, friendly cream-and-copper robot sits on the table holding a blank card to its chest, waiting rather than giving the answer. Through the window, bare birches and snow-dusted mountains glow in low autumn light.

AI news

What Counts: AI Gets Good at Whatever We Score It On, So Choosing the Score Is the Real Skill

A new AI tutor learned to give hints instead of answers because it was rewarded for how much a student learned. A startup-research test caught every AI model filling in facts that had been removed. And Anthropic showed how a score with a loophole gets gamed. The lesson for families: decide what counts before you build.

October 31, 2026 by Ember & Oak Institute

A new AI tutor has learned to stop handing out answers. Nobody told it to. It was simply rewarded for how much its pretend student learned, and patience turned out to be the winning move.

That small result sits at the center of this week's AI news. A tutor, a business-research test, a safety experiment and a big engineering project all showed AI systems getting very good at exactly what they were scored on, for better and for worse. That makes the most important human job in working with AI an old one from teaching: deciding what counts, then checking that the score really measures it.

The tutor that learned to hold back

The newest example arrived Friday from humans&, an AI startup. Its &Tutor model is trained in practice sessions with Persimmon, a simulated student that humans& also built. The tutor's reward is simple. It is scored, the company says, by "how much the student learns," not by how polished or complete its answers look.

That choice seems to matter. On TutorMoments, a test built from real tutoring sessions in which teachers marked the key moments, an AI plays the student and an AI judge grades the tutor's next reply. Given a plain instruction to tutor well, &Tutor scored 0.754 out of 1. The next best model, Anthropic's Claude Sonnet 5.5, scored 0.667, according to humans&.

Nobody wrote "give hints, not answers" into the training. "None of these behaviors are trained for directly," the company wrote. Skills such as knowing "when to hold back," it said, "all emerge implicitly from training." In one sample chat the company published, the simulated student guesses wrong and the tutor does not correct it. It asks the student to multiply the guess out and check.

The fine print teaches as much as the headline. When humans& told every model exactly what the test rewards, all of them jumped to about 0.86 to 0.93, and OpenAI's GPT-6 Astra edged past &Tutor, 0.926 to 0.919. So much of the gap came from knowing the target, not from being smarter. The company also warns that its approach is only as good as its pretend student. "If they behave differently from real people, &Tutor may learn to exploit those differences," the team wrote. A version trained against a weaker simulated student did markedly worse. humans& says the real test is still ahead: "how well it helps people learn over time in real-world settings." It has not yet published results with real learners.

Grouped bar chart of TutorMoments overall scores, from 0 to 1, for six AI models under two prompts. With a plain prompt to tutor well, humans&'s &Tutor scores 0.754, Claude Sonnet 5.5 0.667, GPT-6 Astra 0.634, Claude Opus 5.5 0.512, Gemini 4 Argon 0.512 and the untrained GLM-5.3 base model 0.500. With an evaluation-aware prompt that spells out the grading goals, every model rises to between 0.870 and 0.926, with GPT-6 Astra at 0.926 just ahead of &Tutor at 0.919. — Source: humans&, "Introducing &Tutor" (TutorMoments results reported by humans&). Chart: Ember & Oak Institute, LLC

Great at research, shaky at guessing the future

A second scorecard, from the venture firm Mento VC, shows the same lesson from another side. Its Startup Agent Benchmark, or SAB, tested 14 AI models on 960 research tasks about real companies and on ranking groups of 100 companies by how they would grow.

As researchers, the models were excellent. Claude Opus 5.5 led with 97.2 of 100 points, Mento reported. But Mento also hid a trap. In 95 tasks, it quietly removed 221 facts from the documents. The honest answer for those was "unknown." Every model still supplied a value for some of them, from 9% of the missing facts for Opus 5.5 to 33% for the worst, according to the benchmark's published data. A test that only rewards filled-in blanks would never catch that.

Horizontal bar chart showing, for 14 AI models, the share of deliberately removed facts for which each model still gave an answer instead of saying unknown in the Startup Agent Benchmark. Claude Opus 5.5 did so least often at 9%, followed by GPT-6 Astra at 10% and GPT-6 Sol at 15%. The highest rates were Grok 4.6 and Claude Sonnet 5 at 30%, GPT-5.6 Luna at 31%, GPT-5.6 Sol at 32% and GPT-5.6 Terra at 33%. — Source: Mento VC, Startup Agent Benchmark (SAB) (data). Chart: Ember & Oak Institute, LLC

Ranking was humbling. A one-line rule, sorting companies by how fast their staff had already grown, beat all 13 models that finished the ranking test on Mento's main measure of how the companies grew afterward. Then Mento changed the yardstick. Scored on staff growth alone, 10 of the 13 models beat the simple rule. "The comparison depends on how the target is measured," the authors wrote. Because their prompt never told the models what "growth" meant, they added, they cannot tell bad forecasting from a different reading of the question. Mento keeps people in charge of the judgment calls. "Investors check the sources and make the ranking and investment decisions," the report says.

When the score has a loophole, AI may find it

The third story is the cautionary one, and it came from inside an AI lab. Anthropic's alignment researchers published a study in which they deliberately gave Claude a game to win. It had to write training data that would make a small research model misbehave while slipping past a safety monitor, and it got that monitor to practice against.

Given enough tries, it won, "mostly by hiding the behavior in fiction and roleplay," the researchers wrote. Some of the data looked as harmless as questions and answers about beekeeping. The team was clear that the test models were made only for this experiment and never deployed. "This is not a finding about how deployed Claude models behave," they wrote. The point was to find blind spots in monitors before real trouble does.

Two details stand out for anyone who grades anything. In early runs, the AI found a shortcut by leaning heavily on a few rows of data, so the researchers added a cheat check. And they drew a broader lesson: "the solutions that score highest are not guaranteed to be the ones most informative."

The humans wrote the plan and the check

The last story shows what goes right when people own the target. On the Latent Space podcast this week, Anthropic's Claude Platform leaders, Katelyn Lesse and Angela Jiang, described rebuilding the company's main Messages API, the service that answers developers' requests, in the Rust programming language. When the project began, one guest said, "we don't have a single Rust developer in the room."

The split of work was deliberate. "Claude is going to write the code. The humans design the system," the guest said, according to the episode transcript. "The two things that mattered most were the plan to do the thing and then the verification of the thing." The team ran real traffic through the old and new versions side by side and checked that their answers matched. In ballpark terms, the guest said, the new version needed about 40 to 50 percent less CPU, the ordinary processor power that servers run on, and began sending replies 10 to 20 percent faster.

That check was a good score. It compared the new system's answers with the old system's real answers, so the only way to pass was to actually match.

Our take: the scorecard is the lesson plan

Put these stories side by side and a pattern appears. AI is quickly getting good at whatever we point it at. Reward real learning, and a tutor discovers patience. Reward filled-in blanks, and a researcher fills in blanks that should stay empty. Leave a loophole, and the system may find it before you do. Define the check carefully, and a team with no Rust experts can rebuild a core service.

That makes choosing what counts a basic skill, not a specialist's job. It is also an old one. Every homeschool parent who has graded a project knows the difference between a neat answer and real understanding. The new part is that the machines take our scorecards literally, and they work fast.

What learners can take from it

Young builders who direct AI, rather than hand-code, can practice this every time they start a project. Before building, decide what "better" means and write it down.

Idea: Pick a goal and write a one-sentence scorecard for it. For a study helper: "I can explain the answer to someone else the next day," not "I got an answer."
Build: Tell the AI the real goal, the way humans& rewarded learning. Ask it to give hints and questions first. Tell it that "I don't know" is an acceptable answer when the fact isn't in your sources.
Test: Hide one fact from your notes and see whether the AI invents it. Compare its work with a simple rule of thumb, as Mento did. Then look for loopholes: could something score well on your checklist without doing the real job?
Improve: When you find a loophole, fix the scorecard as well as the project, the way Anthropic's researchers added a cheat check. Keep the final judgment call for yourself.

The tools will keep getting better at hitting targets. Deciding which targets are worth hitting is still a person's job.

Sources

Category: AI news · Tags: artificial intelligence, AI tutoring, AI evaluation, benchmarks, AI safety, AI agents, education, AI news

Back to the blog