Your LLM Can Be Faster Than You (Can) Think