jetson-inference-mem-tune

Jetson Inference Memory Tuning Recommends an inference runtime and the specific memory-related flags to pass to it, given the Jetson SKU/variant and the user's workload. Does not include quantization recipe selection — that lives in the model-benchmarking skill — but it does point at the precision floor each runtime can serve efficiently. Purpose Turn a live jetson-memory-audit snapshot into runtime and launch-flag recommendations for LLM/VLM serving on Jetson. Use this when the user needs to fit a model, reduce OOM risk, or switch to a lower-memory serving stack. When to use "Which serving stack should I use on Orin Nano 8 GB to run a 7B model?" "vLLM is OOMing — what should --gpu-memory-utilization and --max-model-len be?" "Same model, less memory — can I switch from vLLM to llama.cpp?" After jetson-memory-audit shows a model server is the top NvMap / PSS consumer. Prerequisites Show more Installs 688 Repository nvidia/skills GitHub Stars 2.4K First Seen Jun 22, 2026 Security Audits Gen Agent Trust Hub Pass Socket Pass Snyk Pass