AI Can Now Rewrite Neural Networks Directly — Transcript

Full transcript

1. 0:00A new technique called weight-space
2. 0:01programming has just entered the game,
3. 0:03and it lets strong models like Astra
4. 0:05edit the weights of smaller models like
5. 0:07Gwen directly just by prompting it. So,
6. 0:09researchers at MIT gave Astra a couple
7. 0:11of challenges. First, make Gwen, a small
8. 0:13reasoning model, better at math with no
9. 0:15gradient descent allowed. And Astra
10. 0:17found the problem on its own. Gwen kept
11. 0:18getting stuck in thinking loops. It
12. 0:20traced the loops to a single direction,
13. 0:22edited 0.016% of the weights, and the
14. 0:24math 500 score jumped from 7.9% to
15. 0:2748.1%.
16. 0:28Second, a pre-trained policy keeps
17. 0:31driving a simulated car straight at a
18. 0:32ramp and stalling because it never
19. 0:34learned to reverse first for a runway.
20. 0:35Astra's task: fix it in a single reply,
21. 0:37no tools allowed. Astra edited 3.5% of
22. 0:40the weights, and the car went from
23. 0:42clearing the jump 0% of the time to
24. 0:44100%. They're calling it weight-space
25. 0:46programming. Meaning, the LLM can reason
26. 0:48about what each part of a network
27. 0:49computes and edits the weights directly
28. 0:51with no gradient descent. What this
29. 0:53means is that we might be at the very
30. 0:55early stage of LLMs being able to edit,
31. 0:57steer, and refine neural networks
32. 0:59directly. For now, Astra is doing this
33. 1:01to other models, but what about when the
34. 1:02same skill is pointed at its own