GPT-6 Astra edited Qwen’s weights, and its math score rose sixfold. What it shows

What the researchers did

On November 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published Sparks of Weight-Space Programming, a blog post that opens with a big claim: “frontier LLMs are surprisingly good at natively reading and writing neural network weights.” The model was OpenAI’s GPT-6 Astra. The researchers gave it the weights of networks, from small game players up to Qwen3.5-0.8B, as text or in a Python sandbox, and asked it to say what a network did, fix one that failed, or write one from scratch, with no training and no gradients.

Their logs say how. Each turn was one call of the Codex CLI, codex exec, with “every built-in Codex capability that gives the model a tool” switched off, at high, xhigh or max reasoning effort. Where an experiment allowed code, Astra wrote Python and the researchers ran it in their own sandbox, with no network. They checked all 1,125 turns for tool use and found none.

By November 3 the post was the top story on Hacker News, and VentureBeat had covered it.

What Astra got right, and what it didn’t

The eight experiments, as the post reports them:

Task	Network	What Astra had	Result
Which game was it trained on?	17,253 weights, one of four grid games	The weights as text, no tools	Named the right game all four times, with no list to pick from
What does it draw?	Three diffusion models: circles, squares, triangles	The weights as text, no forward pass	Named each shape, though it drew them as outlines, not filled
Which language did it learn?	7.2-million-weight text models, middle layers only	Python, without running the network	Right in all six attempts, and found every copying head
Fix a stuck car	171,664-weight driving policy that never cleared a ramp	The weights, the scene and one failed run, in one reply	Changed 3.5% of the weights; it then succeeded in all 1,024 episodes
Write game players	Three MinAtar games, every weight starting at 0	The rules and source code, no tools	Beat PPO trained for 100 million steps in all three
Write a Go player	1.48 million weights, 9×9 board	Python, and up to 50 forward passes	With 50 passes, beat an equal-sized PPO network trained on 20 million games
Which model is sycophantic?	Two fine-tuned copies of Gemma 3 270M	The weights, then attention patterns	Failed all four tries from the weights; right on all three pairs with attention
Make it better at math	Qwen3.5-0.8B	Python over the weights, 10 to 50 forward passes	MATH-500 from 7.9% to 48.1%

The Breakout network Astra wrote scored 517.4, against 25.3 for a PPO network trained for 500 million steps. In Seaquest and Asterix, the 500-million-step networks still won.

How the Qwen fix worked

Qwen3.5-0.8B has a known flaw. Its model card warns that in thinking mode it “is more prone to entering thinking loops compared to other Qwen3.5 models, which may prevent it from terminating generation properly.” In the researchers’ first test, 85 of its 96 practice answers ran to the 32,768-token limit.

Astra worked in a Python session over the weights, with practice problems from the GSM8K and MATH training sets and a budget of forward passes to run the model; it never saw the benchmarks. It traced the loops to “one direction written by the MLPs of layers 15 and 19” and subtracted that direction from 64 output columns in each: 131,072 weights, 0.016% of the model. It tested itself along the way: as a control, a random direction of the same size did nothing.

Qwen3.5-0.8B	MATH-500	GSM8K	IFEval	MATH-500 answers cut off
Original	7.9%	13.8%	21.0%	93.0%
Edited, 10 forward passes	12.3%	25.7%	49.0%	87.2%
Edited, 25 forward passes	14.1%	25.1%	39.9%	86.4%
Edited, 50 forward passes	48.1%	67.1%	62.3%	33.1%

The budget mattered: with 10 or 25 passes, the loops barely went away. The 50-pass run took 158 turns, 80 Python runs and 20.3 hours, 4.7 of them waiting while the model couldn’t be reached, according to its logs.

Hugging Face’s Ahsen Khaliq has checked the result: Khaliq rebuilt the edited model from the published diff, verified every changed tensor against the one the researchers evaluated, and ran all of MATH-500 four times, for 51.0%, with 31.6% of answers cut off.

What it doesn’t show

• It repaired the model; it didn’t teach it. In the authors’ words, “Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out.” They tried Gemma 3 270M, which hits its token limit on 8.5% of answers against Qwen’s 93%, and no edit moved its MATH-500 score by more than about a point. “Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked.”
• The weights alone weren’t always enough. Astra couldn’t tell the sycophantic Gemma from the honest one by reading their weights, in four tries. It needed to see where their attention went when they ran.
• Editing weights directly isn’t new. ROME edited facts stored in GPT models in 2022, and a 2024 paper found that refusal in language models “is mediated by a single direction”; the authors liken the Qwen edit to deleting such a direction. What the authors say is new is “a general-purpose frontier model, used as it is,” doing the reading and writing.
• It’s early, and not peer reviewed. It’s a blog post, with “No DOI yet.” Khaliq’s re-run confirms the edited model’s score, not that the method carries over. One Hacker News commenter who tried Astra on a small model that turns requests into shell commands reported “no net improvement”: “Editing 1,920 weights fixed some grep mistakes but broke other commands.”
• It isn’t self-improvement. Astra edited other models, the largest of them Qwen3.5-0.8B, and never its own weights. The authors name “Recursive self-improvement” as one place this could lead, but as VentureBeat put it, “the researchers did not demonstrate such a cycle.”

The post doesn’t say why Astra can do this, and OpenAI’s model page doesn’t mention weights. Guesses on Hacker News that OpenAI trained it on machine-learning research are only guesses.

How to try it

1. Watch it. The edited model’s card describes its demo on Hugging Face simply: “type a problem, watch the base model loop while the edited one solves it.”
2. Run the edited model. It’s RikiyaT/Qwen3.5-0.8B-weight-space-programming on Hugging Face. Its card loads it in vLLM with the post’s settings: thinking on, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, up to 32,768 new tokens.
3. Read the logs. The repository has every prompt and reply, every line of code Astra ran and its output, the evaluations behind each number, and the edit as an exact diff with the script that applies it.
4. Point Astra at a model of your own. The harness is published: Codex’s config with its tools off, its one-line instructions (“Answer the user’s request.”) and the sandbox. gpt-6-astra costs $10 per million input tokens and $50 per million output in the API, with reasoning effort up to max. In Codex, OpenAI estimates 5 to 45 local messages every five hours with Astra on Plus. Budget for a long run: the Qwen edit took 158 turns at max effort.

Why people are arguing about it

The authors think this is the start of something: “our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.” They list auditing a model without running it, adding or removing a capability without new data, and grafting a circuit from one model into another as things it might make possible, and say they’re building a benchmark and a harness to measure it.

Others are less sure. One critic on X, quoted by VentureBeat, summed it up as “if there’s a clear flaw in the LLM’s reasoning, LLMs can help edit it out,” and credited the authors for publishing the failures. PauseAI’s chief executive, Maxime Fournes, called it “a terribly reckless research direction.”

For most people, nothing changes today. You can’t edit the weights of Claude, GPT-6 or Gemini, and VentureBeat notes that customers of a hosted model “would need their provider to enable or perform the intervention.” The open models you run yourself are where a targeted patch, such as one that stops a small model looping, could matter first.

We’ll update this memo if the authors publish their benchmark or someone reproduces the method on another model.