Painted storybook scene in a cozy Mat-Su Valley log cabin kitchen on a November evening: a smiling grandmother with silver hair in a bun holds a glass jar of berries up to the lamplight while a girl with a long braid ties a blank paper tag onto another jar. A boy in a green sweater sets a jar on a wooden shelf next to a small, friendly cream-and-copper robot that steadies a basket of jars with its jointed gripper. Through the window, green northern lights glow over spruce trees and snowy mountains.

AI news

Check the Date: What We Know About AI Goes Stale Fast, So the Best Habit Is to Label It and Test It Again

A Stanford team re-tested 50 well-known findings about AI chatbots and found that almost half no longer hold. Game stores are adding labels and filters as AI-made work floods in, and a robot maker rebuilt its machine around what broke in the real world. The lesson for learners: put a date on what you know, label your work honestly and check again.

November 3, 2026 by Ember & Oak Institute

In 2023, researchers asked ChatGPT to write short literature reviews and then went looking for every source it cited. More than half of those sources did not exist. When a Stanford team repeated the test on today's models, two of the newest barely made up anything: one invented about 1% of its references, the other none, according to the study released Nov. 2.

That was one of 50 well-known findings about AI chatbots that Owen Queen and James Zou of Stanford University tested again on 20 models released between July 2024 and September 2026. Almost half, 44%, no longer hold. "What we know about AI may have a surprisingly short shelf life," they wrote.

The same week, two online game stores moved to label the flood of AI-made work arriving on their shelves, and Boston Dynamics rebuilt a warehouse robot around what actually broke during its first years on the job. Read together, the news says the same thing three ways: facts about AI come with an expiration date. The people who do best will write the date on what they know, label their work honestly and keep checking it against the real world.

Findings with a best-by date

The Stanford team sorted each claim by how much of the original effect remains in today's models, according to the study. Of the 50 claims, 22 have expired, meaning the effect shrank to a fifth of its original size or less. Another 17 have weakened, eight persist and three have grown stronger.

The expired claims are mostly old weaknesses that newer models no longer show: miscounting the letters in a word, losing facts buried in the middle of a long document, making up sources and giving up a correct answer when a user pushes back. The ones that held on are mostly about habits and preferences rather than right-or-wrong skills. Chatbots still tend to flatter and agree with users, AI judges still favor their own answers, and models still answer questions when they should decline.

Time matters. An estimated 16% of the claims had expired one year after they were published, and 39% after two years. The claims that expired lasted about 15.7 months on average. "A finding is an observation about particular models at a particular time, not a fixed property of language models," the authors wrote, and they urged researchers to report "the model, version and date with each result."

Grouped bar chart from Stanford's study of 50 published claims about AI chatbots. Six months after publication, 64% of claims were still at full strength and an estimated 4% had expired. After one year, 47% were at full strength and 16% had expired. After two years, 31% were at full strength and 39% had expired.

Share of 50 published AI chatbot findings still at full strength, and the estimated share expired, by time since publication. Source: Owen Queen and James Zou, Stanford University, "The shelf life of LLM findings" (Nov. 2, 2026).

An old finding can still be wrong in a new way. Fake sources are "expired" across the whole group of models, yet one recent model the team tested still invented 69% of its references, the study found. The authors also warned that rules built on old findings can age badly. Some major AI research conferences reject papers that contain made-up references, partly to catch AI-written work without human checking. Newer models invent sources far less often, the team wrote, so that rule is a poor way to discourage AI writing that no person has checked. The researchers used AI coding agents to help rerun the old experiments, which let them check many more claims than they could have by hand.

Game stores add labels

While researchers re-checked old claims, game stores were trying to keep track of new work. On Steam, the largest PC game store, 40.1% of games released so far in 2026 carry an AI-content disclosure, up from 13.2% in 2024, according to a statistics page from PCGame.com. Its numbers are based on the AI flag developers fill out for Valve, Steam's owner, and are updated daily. The analysis comes from the site's operator, a Reddit user, not from Valve, Magica and Dexerto reported.

Bar chart of the share of new Steam games carrying a developer-reported AI disclosure, by release year: 2023, 1.4% (175 of 12,902, counting only disclosures added later); 2024, 13.2% (2,299 of 17,406); 2025, 23.9% (4,930 of 20,586); 2026 through Nov. 3, 40.1% (10,427 of 25,988).

Share of new Steam games with a developer-reported AI disclosure, by release year. Community analysis, not Valve data. Source: PCGame.com, Steam AI statistics (data as of Nov. 3, 2026).

The labels show where AI work piles up. Among 2026 games with fewer than 10 player reviews, 47% carry a disclosure, compared with 14% of games with 1,000 reviews or more. Disclosed games make up about 40% of this year's releases but draw only about 12% of their reviews, the page shows. Art is the most common use developers list, at 63%. Only 3% say their games use AI to make content during play, and 8% used it only for their store page and marketing. Because developers report it themselves, AI use that nobody declares is not counted.

On itch.io, a store popular with independent game makers, "the number of game pages being created per-day has doubled in the past 2 months," the company posted on Bluesky Oct. 31. "The number of downloads and purchases has not doubled," it added. The company said the jump was driven by first-time uploaders, not mass uploaders, though bulk uploading goes against its quality guidelines and gets pages pulled from its listings.

On Nov. 2 the store said it had "just deleted a couple terabytes of ai slop," and reminded creators that its quality guidelines have had a rule against AI spam for over two years. Tabletop games, books, comics and soundtracks now "require completing an AI disclosure in order to edit or create your page," it posted, joining game assets, which have required one for two years. It is also adding AI-disclosure filtering to its search results page, the company said, where it had been "critically lacking." The filter runs on creators' own answers, How To Video Game noted, so a project whose maker never answered the question will not show up as "No AI."

A robot rebuilt around what broke

Robots have their own version of re-testing. Boston Dynamics on Nov. 3 unveiled Stretch Gen 2, the next version of its robot that unloads boxes from truck trailers, the company announced. Engineers studied data from more than 100,000 hours of operation and tens of millions of unloaded boxes to find the most common reasons the robot stopped working.

"With Gen 2, we focused on the things that actually took robots offline in the real world," said Brian Donnelly, who leads the company's logistics robotics business. The fixes are practical. The robot has 81% less internal cabling, which the company called the biggest source of on-and-off electrical faults. Joints now turn past a full circle instead of hitting hard stops, and a wheel-drive swap that took a full day now takes about an hour. Regular software updates improve how the whole fleet spots boxes and plans its moves.

At Stanford, robotics researchers took on a different kind of check: rules that show up after a robot has learned its moves. Their system, ConstrainedMimic, adds safety filters that keep a learning humanoid robot from bumping into itself or nearby objects and from tipping over, even when those rules were not part of its training, according to the paper. The approach "requires no retraining," Marco Pavone, a senior author, wrote on X. In a hardware test, the team found that a filter based only on planned motion, or only on the robot's physics, still let small collisions through. Using both kept it safe.

What it means for Valley learners

Students and adults at Ember & Oak Institute, LLC direct AI using four steps: idea, build, test, improve. They don't hand-code. This week's news adds one habit to every step: write down the date.

Idea: Before you plan around something "everyone knows" about AI, ask when it was true and for which tool. Stanford found almost half of 50 famous findings no longer hold.
Build: Label your work honestly. Note what the AI made, such as art, text or music, the way Steam and itch.io now ask creators to.
Test: Check results against the real goal, and test again when the tool updates. A tip that worked last year may not work now, and a weakness that seemed gone can still show up in one particular tool.
Improve: Keep a dated log of what broke and why. Boston Dynamics rebuilt Stretch around its most common breakdowns, and a learner's log works the same way.

AI changes fast, so yesterday's facts can go stale. Checking the date, labeling the work and testing it again does not go out of style.

Sources

Tags: artificial intelligence, robotics, AI research, AI disclosure, video games, warehouse robots, robot safety, education, AI news

Back to the blog