TE
TechEcho
Home
24h Top
Newest
Best
Ask
Show
Jobs
English
GitHub
Twitter
Home
TensorRT-LLM runtime now open-source
4 points
by
mmoskal
2 months ago
1 comment
mmoskal
2 months ago
Previously, the "Executor" runtime was shipped as binary blobs. This is the bit that schedules requests and manages KV cache (similar to vLLM or SGLang server).