the local hardware bet isn't justified by current pricing. if you see the progress of model development and think
"yea but you won't break even"
instead of
"in 6 months modest local hardware will probably be very useful"
then you are a SLAVE and a RETARD!
Max Weinbach (@mweinbach)
The minimum to run the model is ~$20K in hardware and you get ~20 tok/s out
~$20K gets you around 34.6B tokens at a 12:1 input to output ratio assuming good token caching
If you ran the hardware 24/7, it would take roughly 5.5 years to break even
— https://nitter.net/mweinbach/status/2068459318240837946#m