∇gradient updates
Papers
Four papers under review, all from 2026. Venues are not listed while reviews are pending. Each links to the de-anonymized PDF.
How Valid are Metrics of Chain of Thought Faithfulness with LLM Judges?
Cue injection is the standard way to measure whether a model's chain of thought reflects what drove its answer: plant a false hint, then check whether the answer moves and whether the CoT mentions the hint. Across four Qwen 2.5 sizes, a placebo with no false answer changes the smallest model's answer on 21 to 27 percent of questions, which is 85 to 94 percent of the effect of a real cue. Two things that should not matter, decoding parameters and the family of questions asked, move the result substantially, and an LLM judge re-scoring identical outputs disagrees with itself. The metric's headline conclusion, that smaller models are less faithful, inherits size-dependent error terms.
What Happens to a Monitor's Accuracy When You Train Against It
A linear probe on a model's hidden states predicts whether its next answer will be correct: held-out AUROC 0.982, still 0.939 with trace length stratified out, and as a best-of-16 selector it captures 89 percent of the available headroom. Used as the reward for reinforcement learning, a probe of the same representation drives true accuracy down by 31 percentage points, and the probe's own accuracy, scored at every checkpoint on fresh answers, does not register the change for about 40 training steps. An LLM judge scoring the same run shows the same lag. Accuracy is a property of a classifier and a population together, and deploying the monitor as a reward is what moves the population. A monitor built only from text features of the answer scores higher than the probe as a monitor and, as a reward, drives accuracy to zero while the answers stay well-formed. One statistic, the share of answers above a threshold fixed at the start, registers the change early and without ground truth.
How Small Can We Go? Calibrating Activation-Cache Compression for SAE Training
Training a sparse autoencoder needs a large cache of model activations, and compressing that cache changes which features the SAE learns. The paper shows that the obvious measurement is not meaningful on its own: retraining on a cache with one value in a million perturbed by the smallest fp32 step keeps only 70 percent of features, while changing the random seed keeps 23 percent. So it calibrates each site instead, with retraining on identical bytes as the ceiling and a reseed as the floor, and measures every codec against that. On GPT-2 small and Pythia-1.4B, naive per-token int4 loses more features than a reseed, standardizing channels before quantizing brings int4 back above the floor at the same file size, and a bf16 forward pass loses slightly more than rotated 8-bit storage. About 170 GPU-hours of pre-registered experiments, with every number regenerating from committed artifacts.
Coding Agents for Coding Theory
Five weeks of an LLM coding agent, run under a protocol that required verifiers to be written and tested before any search, on open cells of the quaternary edit-metric code table. Searching over codes invariant under a prescribed symmetry group improved four published lower bounds, including E4(6,3) from 114, a bound standing since 2012, to 120. The paper gives equal space to how the work went wrong: an intermediate result was written down, never rechecked, and then treated as a fact that ruled out further search, twice. SAT and ILP produced every certificate and no constructions, and the calibration check the paper proposes turns out to be necessary but not sufficient. The bound count has grown since this version was submitted.