I'm finding it more or less impossible to find a model/provider combo which consistently will do >150 tok/sec. Basically everybody pumps their infra to provide that for a week, gloat on X about it, then throttles it back to 50 tok/sec next week.
I'm finding it more or less impossible to find a model/provider combo which consistently will do >150 tok/sec. Basically everybody pumps their infra to provide that for a week, gloat on X about it, then throttles it back to 50 tok/sec next week.