A new paper, Language Models Can Rewrite Their Own Weights, reports that a language model can propose and test changes to its own numerical parameters without gradient-based training. One edit improved accuracy on previously unseen math problems by 6.6 percentage points. It raises an important question. Who decides what counts as improvement? Read the paper.
In 2022, 263 elephants were relocated to Malawi’s Kasungu National Park as part of a conservation effort. Communities subsequently reported deaths, damaged crops, and threats to their livelihoods. Their lawyers alleged inadequate safeguards; the International Fund for Animal Welfare disputed the allegations and denied liability. The dispute raises a question that extends beyond conservation: who defines success, and whose losses count against it? Community allegations; IFAW’s response.
That question, “The Elephants or Us?” came to mind while reading Language Models Can Rewrite Their Own Weights. The connection concerns how improvement is defined: which objectives receive attention, which risks are accepted, whose experience supplies the evidence, and how far into the future anyone looks. Those who establish the initial direction may exercise disproportionate influence over what follows. If subsequent improvements are judged against the same narrow measure, a system could become more effective while reinforcing an inadequate definition of success. Slow, cumulative effects might escape notice, particularly when those bearing the costs have little influence over the evaluation. How can we prevent such effects when we cannot identify every subtle influence in advance? Prevention cannot depend entirely on an affected person discovering the hidden mechanism after harm occurs. We need continuing scrutiny of outcomes, meaningful participation by affected communities, and authority to revise the objective itself—even when the system is performing exactly as intended.
“Elephants or us?” should not have to be the choice. Avoiding that choice requires more than good intentions or better performance. It requires that the people living with the consequences can challenge what counts as improvement, before the initial priorities become everyone else’s future. In all the public attention directed toward “AI Safety,” few of these critically important, fundamental distinctions are being discussed.
Signed articles reflect the views of their authors. Corrections: info@dghumana.com.
