The number I'd look at first is the 7.9% baseline. That's very low for a reasoning model on MATH-500, and if the model card already warns about it getting stuck in loops, a lot of those misses are probably runs that never finished, not wrong math. Fix the loop and those answers come back without the model learning anything new. Finding the cause by reading weights is still impressive, but "6x better at math" oversells it. I'd want the edited model compared against the same base model with a repetition penalty or a length cap, since that's the cheap fix people already use.
The Breakout result has a similar catch. Astra got the rules and the code, so writing the weights is closer to compiling a known policy than learning one. And 25.3 after 500M PPO steps on a small Breakout is a weak baseline. Was PPO tuned at all?
The game-ID part surprises me most, since all four nets share an architecture. Did they differ in how many of the output actions they actually use? That alone would separate Freeway from Pac-Man.
Does anyone have a link to the paper? I couldn't find it.
#ai #interpretability