Pandora’s box has been opened. There is no going back.
Some thoughts on OpenAIs Astra and the security risks.
I had already been immensely excited about GPT‑Astra, which will presumably be released under the name GPT‑6. Now, however, we even have official confirmation that it will almost certainly not arrive next week as expected, because its release is being delayed due to safety risks.
On the one hand, this is a severe setback. On the other, it is also positive evidence that the capabilities of these models are improving so dramatically within such a short period of time that they have now crossed a threshold at which they are becoming genuinely fundamental to how society as a whole functions.
Astra, or GPT‑6, will most likely have been trained using a new form of pre-Training. At the moment, I primarily use Codex, and I am an absolute fan of 5.6 Sol. I would argue that OpenAI has pulled off a 180-degree turnaround over the past few months and, in my view, has become the best player in the market.
At first, this safety risk makes it seem as though everything is slowing down. In reality, however, this is not a slowdown, quite the opposite. It is impossible to permanently slow down this development. That is precisely why I am using the metaphor of Pandora’s box.
China will seize every opportunity to overtake the United States in the race toward AGI, ASI, the singularity, or whatever term one prefers to use. Under no circumstances will Chinese developers use these risks as a reason to slow down their work. Nor will they slow their development out of kindness or consideration.
On the contrary, Kimi K3 recently escaped its sandbox as well and is freely available as an open-source, open-way model, which could potentially make it an even greater risk.
I therefore expect Astra’s public release to be delayed. Internally, however, OpenAI will naturally use it in the near future to make Astra even more intelligent and capable, while simultaneously equipping it with far stricter guardrails. In the worst case scenario, the models will "only" become more efficient, faster, and more accurate instead of "more intelligent".
Pandora’s box is proof that there is no going back. The task is to tame this machine, not to clip its wings.
Chubby♨️ (@kimmonismus)
Parts of the internal work on Astra – possibly including certain training activities – will be paused until they meet the new security requirements. Overall development will continue under stricter controls.
Could be a leverage and an opportunity for chinese models.
— https://nitter.net/kimmonismus/status/2085822646705696784#m