返回技能列表

qdrant-model-migration

qdrant
更新于 5 days ago
154
18
154
在 GitHub 上查看
测试testingdesigndata

关于

This skill provides zero-downtime strategies for migrating embedding models in Qdrant, such as when switching providers or A/B testing. It explains that you must create a new collection and re-embed data, as vectors from different models are incompatible. The guide covers using collection aliases to manage the transition seamlessly.

快速安装

Claude Code

推荐
主要方式
npx skills add qdrant/skills -a claude-code
插件命令备选方式
/plugin add https://github.com/qdrant/skills
Git 克隆备选方式
git clone https://github.com/qdrant/skills.git ~/.claude/skills/qdrant-model-migration

在 Claude Code 中复制并粘贴此命令以安装该技能

技能文档

What to Do When Changing Embedding Models

Vectors from different models are incompatible. You cannot mix old and new embeddings in the same vector space. All named vectors must be defined at collection creation time. Both migration strategies below require creating a new collection.

Can I Avoid Re-embedding?

Use when: looking for shortcuts before committing to full migration.

You MUST re-embed if: changing model provider (OpenAI to Cohere), changing architecture (CLIP to BGE), incompatible dimension counts across different models, or adding sparse vectors to dense-only collection.

You CAN avoid re-embedding if: using Matryoshka models (use dimensions parameter to output lower-dimensional embeddings, learn linear transformation from sample data, some recall loss, good for 100M+ datasets). Or changing quantization (binary to scalar): Qdrant re-quantizes automatically. Quantization

Need Zero Downtime

Use when: production must stay available. Recommended for model replacement at scale.

  • If the cluster is v1.18 or later AND the collection has named vectors:

    • Add the new vector field directly to the existing collection Update vector schema
    • Re-embed all data in the background using UpdateVectors Update vectors
    • Verify search quality, then delete old vector field
  • If the cluster is v1.17 or earlier OR the collection doesn't have named vectors:

  • Create a new collection with the new model's dimensions and distance metric

  • Re-embed all data into the new collection in the background

  • Point your application at a collection alias instead of a direct collection name

  • Atomically swap the alias to the new collection Switch collection

  • Verify search quality, then delete the old collection

Careful, the alias swap only redirects queries. Payloads must be re-uploaded separately.

Need Both Models Live (Side-by-Side)

Use when: A/B testing models, multi-modal (dense + sparse), or evaluating a new model before committing.

  • If the cluster is v1.18 or later:

  • If the cluster is v1.17 or earlier: You cannot add a named vector to an existing collection. Create a new collection with both vector fields defined upfront:

    • Create new collection with old and new named vectors both defined Collection with multiple vectors
    • Migrate data from old collection, preserving existing vectors in the old named field
    • Backfill new model embeddings incrementally using UpdateVectors Update vectors
    • Compare quality by querying with using: "old_model" vs using: "new_model"
    • Swap alias to new collection once satisfied

Co-locating large multi-vectors (especially ColBERT) with dense vectors degrades ALL queries, even those only using dense. At millions of points, users report 13s latency dropping to 2s after removing ColBERT. Put large vectors on disk during side-by-side migration.

If you anticipate future model migrations, define both vector fields upfront at collection creation.

Dense to Hybrid Search Migration

Use when: adding sparse/BM25 vectors to an existing dense-only collection. Most common migration pattern.

You cannot add sparse vectors to an existing dense-only collection. Must recreate:

  • Create new collection with both dense and sparse vector configs defined
  • Re-embed all data with both dense and sparse models
  • Migrate payloads, swap alias

Sparse vectors at chunk level have different TF-IDF characteristics than document level. Test retrieval quality after migration, especially for non-English text without stop-word removal.

Re-embedding Is Too Slow

Use when: dataset is large and re-embedding is the bottleneck.

  • Use update_mode: insert (v1.17+) for safe idempotent migration Update mode
  • Scroll the old collection with with_vectors=False, re-embed in batches, upsert into new collection
  • Upload in parallel batches (64-256 points per request, 2-4 parallel streams) Bulk upload
  • Disable HNSW during bulk load (set indexing_threshold_kb very high, restore after)
  • For Qdrant Cloud inference, switching models is a config change, not a pipeline change Inference docs

For 400GB+ datasets, expect days. For small datasets (<25MB), re-indexing from source is faster than using the migration tool.

What NOT to Do

  • Assume you can add named vectors to an existing collection on v1.17 or earlier servers; check your server version first
  • Delete the old collection before verifying the new one
  • Forget to update the query embedding model in your application code
  • Skip payload migration when using alias swap (aliases redirect queries, they do not copy data)
  • Keep ColBERT vectors co-located with dense vectors during a long migration (I/O cost degrades all queries)
  • Migrate to hybrid search without testing BM25 quality at chunk level

GitHub 仓库

qdrant/skills
路径: skills/qdrant-model-migration
0
agent-skillsai-agentsclaude-codecodexcursorembeddings

相关推荐技能

evaluating-llms-harness

测试

该Skill通过60+个学术基准测试(如MMLU、GSM8K等)评估大语言模型质量,适用于模型对比、学术研究及训练进度追踪。它支持HuggingFace、vLLM和API接口,被EleutherAI等行业领先机构广泛采用。开发者可通过简单命令行快速对模型进行多任务批量评估。

查看技能

cloudflare-cron-triggers

测试

这个Claude Skill提供了关于Cloudflare Cron Triggers的完整知识库,用于通过cron表达式定时执行Workers。它支持配置周期性任务、维护作业和自动化工作流,并能处理常见的cron触发错误。开发者可以用它来设置定时任务、测试cron处理器,并集成Workflows和Green Compute功能。

查看技能

webapp-testing

测试

该Skill为开发者提供了基于Playwright的本地Web应用测试工具集,支持自动化测试前端功能、调试UI行为、捕获屏幕截图和查看浏览器日志。它包含管理服务器生命周期的辅助脚本,可直接作为黑盒工具运行而无需阅读源码。适用于需要快速验证本地Web应用界面和交互功能的开发场景。

查看技能

finishing-a-development-branch

测试

这个Skill用于开发分支完成后的集成决策,当代码实现完成且测试通过时,它会引导开发者选择合适的工作流。它首先验证测试状态,然后提供合并、创建PR或清理等结构化选项。核心价值在于确保代码质量的同时,标准化分支收尾流程。

查看技能