llm-inference
About
This skill enables LLM inference through Cloudflare Pages functions with OpenAI-compatible endpoints. It provides access to multiple models with different capabilities, including high-performance options like gpt-oss-120b and specialized models for various tasks. Use this when you need to integrate LLM capabilities into your applications and want the agent to select the most suitable model based on your requirements.
Documentation
LLM Inference
The Cloudflare Pages function functions/cerebras-chat.ts provides OpenAI-compatible LLM inference. See tools/cerebras-llm-inference/index.html for a working example.
Available models
| Model | Max context tokens | Requests / minute | Tokens / minute |
|---|---|---|---|
| gpt-oss-120b | 65,536 | 30 | 64,000 |
| llama-3.3-70b | 65,536 | 30 | 64,000 |
| llama3.1-8b | 8,192 | 30 | 60,000 |
| qwen-3-235b-a22b-instruct-2507 | 65,536 | 30 | 64,000 |
| qwen-3-235b-a22b-thinking-2507 | 65,536 | 30 | 60,000 |
| qwen-3-32b | 65,536 | 30 | 64,000 |
| zai-glm-4.6 | 64,000 | 10 | 150,000 |
llama3.1-8bis the fastest option.zai-glm-4.6is the most powerful option.gpt-oss-120bremains the best all rounder.
LLMs are not just for chat: they can be used to process any string in any arbitrary way. If making a tool that requires the LLM to respond in a specific way or format then be very clear and explicit in its system prompt; eg what to include/exclude, plain/markdown formatting, length, etc.
Quick Install
/plugin add https://github.com/dave1010/tools/tree/main/llm-inferenceCopy and paste this command in Claude Code to install this skill
GitHub 仓库
Related Skills
sglang
MetaSGLang is a high-performance LLM serving framework that specializes in fast, structured generation for JSON, regex, and agentic workflows using its RadixAttention prefix caching. It delivers significantly faster inference, especially for tasks with repeated prefixes, making it ideal for complex, structured outputs and multi-turn conversations. Choose SGLang over alternatives like vLLM when you need constrained decoding or are building applications with extensive prefix sharing.
evaluating-llms-harness
TestingThis Claude Skill runs the lm-evaluation-harness to benchmark LLMs across 60+ standardized academic tasks like MMLU and GSM8K. It's designed for developers to compare model quality, track training progress, or report academic results. The tool supports various backends including HuggingFace and vLLM models.
llamaguard
OtherLlamaGuard is Meta's 7-8B parameter model for moderating LLM inputs and outputs across six safety categories like violence and hate speech. It offers 94-95% accuracy and can be deployed using vLLM, Hugging Face, or Amazon SageMaker. Use this skill to easily integrate content filtering and safety guardrails into your AI applications.
langchain
MetaLangChain is a framework for building LLM applications using agents, chains, and RAG pipelines. It supports multiple LLM providers, offers 500+ integrations, and includes features like tool calling and memory management. Use it for rapid prototyping and deploying production systems like chatbots, autonomous agents, and question-answering services.