Back to Skills

qdrant-scaling-data-volume

qdrant
Updated 5 days ago
154
18
154
View on GitHub
Designdesigndata

About

This skill helps developers scale Qdrant vector database storage when data exceeds single-node capacity. It provides guidance on tenant scaling with payload partitioning and sliding time window strategies for time-series data. Use it when facing "data doesn't fit on one node" scenarios or needing to choose between vertical/horizontal scaling approaches.

Quick Install

Claude Code

Recommended
Primary
npx skills add qdrant/skills -a claude-code
Plugin CommandAlternative
/plugin add https://github.com/qdrant/skills
Git CloneAlternative
git clone https://github.com/qdrant/skills.git ~/.claude/skills/qdrant-scaling-data-volume

Copy and paste this command in Claude Code to install this skill

Documentation

Scaling Data Volume

This document covers data volume scaling scenarios, where the total size of the dataset exceeds the capacity of a single node.

Tenant Scaling

If the use case is multi-tenant, meaning that each user only has access to a subset of the data, and we never need to query across all the data, then we can use multi-tenancy patterns to scale.

The recommended way is to use multi-tenant workloads with payload partitioning, per-tenant indexes, and tiered multitenancy.

Learn more Tenant Scaling

Sliding Time Window

Some use-cases are based on a sliding time window, where only the most recent data is relevant. For example an index for social media posts, where only the last 6 months of data require fast search.

Learn more Sliding Time Window

Global Search

Most general use-cases require global search across all data. In these situations, we might need to fall back to vertical scaling, and then horizontal scaling when we reach the limits of vertical scaling.

Vertical Scaling

When data doesn't fit in a single node, the first approach is to scale the node itself — more RAM, better disk, quantization, mmap. Exhaust vertical options before going horizontal, as horizontal scaling adds permanent operational complexity.

Learn more Vertical Scaling

Horizontal Scaling

When a single node can't hold the data even with quantization and mmap, distribute data across multiple nodes via sharding.

Learn more Horizontal Scaling

GitHub Repository

qdrant/skills
Path: skills/qdrant-scaling/scaling-data-volume
0
agent-skillsai-agentsclaude-codecodexcursorembeddings

Related Skills

executing-plans

Design

Use the executing-plans skill when you have a complete implementation plan to execute in controlled batches with review checkpoints. It loads and critically reviews the plan, then executes tasks in small batches (default 3 tasks) while reporting progress between each batch for architect review. This ensures systematic implementation with built-in quality control checkpoints.

View skill

requesting-code-review

Design

This skill dispatches a code-reviewer subagent to analyze code changes against requirements before proceeding. It should be used after completing tasks, implementing major features, or before merging to main. The review helps catch issues early by comparing the current implementation with the original plan.

View skill

connect-mcp-server

Design

This skill provides a comprehensive guide for developers to connect MCP servers to Claude Code using HTTP, stdio, or SSE transports. It covers installation, configuration, authentication, and security for integrating external services like GitHub, Notion, and custom APIs. Use it when setting up MCP integrations, configuring external tools, or working with Claude's Model Context Protocol.

View skill

web-cli-teleport

Design

This skill helps developers choose between Claude Code Web and CLI interfaces based on task analysis, then enables seamless session teleportation between these environments. It optimizes workflow by managing session state and context when switching between web, CLI, or mobile. Use it for complex projects requiring different tools at various stages.

View skill