Generation speed in short contexts is much better, like 13.5 t/s. The generation speed number you see in the status bar is not really reliable, it is faster than that as you can see visually, but there is an upfront setup cost that must be reduced.
Generation speed in short contexts is much better, like 13.5 t/s. The generation speed number you see in the status bar is not really reliable, it is faster than that as you can see visually, but there is an upfront setup cost that must be reduced.