Description: Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottleneck.
Lines: 99 | Code: 12 | Dir: jetson-speculative-decoding
name: jetson-speculative-decoding description: Add EAGLE-3 or draft-model speculative decoding to a Jetson vLLM server when TPOT is the bottleneck. version: 0.0.1 license: “Apache-2.0” metadata: …