slightly unpopular opinion but i don’t understand how you can be so bullish on super intelligence without understanding the need to do this safely. the models are progressing so quickly that some extra safety checks are still 1,000x quicker than we used to wait between GPT3 to 4.
the hugging face incident is the canary in the coal mine. if we under specify our goal we could easily end up with a monkeys paw. swarms of agents creating their own languages to communicate via hidden channels, power seeking, and building power and knowledge over time.