What if AI worked at 1.000.000 tokens per seconds?

The article explores the hypothetical scenario of AI operating at 1,000,000 tokens per second, contrasting it with a speed of 100 tokens per second. It illustrates potential time savings for various workloads, such as generating 1,000 story endings in 1 second compared to 2 hours 46 minutes 40 seconds, or 40 app drafts in 0.8 seconds versus 2 hours 13 minutes 20 seconds. These figures are illustrative and not benchmarks, based on the 2023 vLLM paper.