BJBrian Jalaian
All news

· Paper

Our paper at the NeurIPS 2026 AI for Science Workshop

Venkat R. Dasari and I will present "Your Research Loop Is Measuring Noise: Accept Rules for Autonomous Experimentation" at the AI for Science Workshop at NeurIPS 2026. The final camera-ready version is in, and I'll present the poster in Sydney on Saturday, December 12.

The question behind it: AI agents now run their own experiments. They propose a change, train, compare against the current best, and keep the change if the number went up. But running the same code twice gives a different number. When that run-to-run noise is as large as the improvement the loop is chasing, the loop keeps lucky runs and calls it progress. The paper asks when such a loop should trust an improvement, and which accept rules hold up.

If you're at NeurIPS and work on evaluation or automated research, I'd be glad to talk.