tensorrt-llm
About
TensorRT-LLM is an NVIDIA library that optimizes LLM inference for maximum throughput and lowest latency on NVIDIA GPUs. It is ideal for production deployments requiring 10-100x faster performance than PyTorch, supporting features like quantization and multi-GPU scaling. Use it when you need top performance on NVIDIA hardware, opting for alternatives like vLLM for simpler setups or llama.cpp for CPU/Apple Silicon.
Quick Install
Claude Code
Recommendednpx skills add zechenzhangAGI/AI-research-SKILLs -a claude-code/plugin add https://github.com/zechenzhangAGI/AI-research-SKILLsgit clone https://github.com/zechenzhangAGI/AI-research-SKILLs.git ~/.claude/skills/tensorrt-llmCopy and paste this command in Claude Code to install this skill
GitHub Repository
Related Skills
llamaguard
OtherLlamaGuard is Meta's 7-8B parameter model for moderating LLM inputs and outputs across six safety categories like violence and hate speech. It offers 94-95% accuracy and can be deployed using vLLM, Hugging Face, or Amazon SageMaker. Use this skill to easily integrate content filtering and safety guardrails into your AI applications.
cost-optimization
OtherThis Claude Skill helps developers optimize cloud costs through resource rightsizing, tagging strategies, and spending analysis. It provides a framework for reducing cloud expenses and implementing cost governance across AWS, Azure, and GCP. Use it when you need to analyze infrastructure costs, right-size resources, or meet budget constraints.
quantizing-models-bitsandbytes
OtherThis skill quantizes LLMs to 8-bit or 4-bit precision using bitsandbytes, achieving 50-75% memory reduction with minimal accuracy loss. It's ideal for running larger models on limited GPU memory or accelerating inference, supporting formats like INT8, NF4, and FP4. The skill integrates with HuggingFace Transformers and enables QLoRA training and 8-bit optimizers.
dispatching-parallel-agents
OtherThis Claude Skill dispatches multiple agents to investigate and fix 3+ independent problems concurrently. It is designed for scenarios involving unrelated failures that can be resolved without shared state or dependencies. The core capability is parallel problem-solving, assigning one agent per independent problem domain to maximize efficiency.
