SKILL·21866C

mutation-testing

trailofbits
Updated 15 days ago
10 views
7,515
642
7,515
View on GitHub
Testingaitestingdesign

About

This skill helps developers configure and run mutation testing campaigns using mewt or muton tools. It assists with analyzing results, identifying equivalent mutants, and finding bugs exposed by testing gaps. Use it when setting up campaigns, reviewing surviving mutants, or investigating coverage weaknesses.

Quick Install

Claude Code

Recommended
Primary
npx skills add trailofbits/skills -a claude-code
Plugin CommandAlternative
/plugin add https://github.com/trailofbits/skills
Git CloneAlternative
git clone https://github.com/trailofbits/skills.git ~/.claude/skills/mutation-testing

Copy and paste this command in Claude Code to install this skill

Documentation

Mutation Testing (mewt/muton)

Routes to the right mutation testing workflow and loads the references that workflow needs.

Note: muton and mewt share identical interfaces. Examples use mewt; substitute muton and its file names (muton.toml, muton.sqlite) for muton projects.

mewt --help and mewt <subcommand> --help are the source of truth for command-line behavior. Examples below reflect the mewt 4.x API; run --help when a flag looks unfamiliar or a command fails.

When to Use

Use this skill when the user:

  • Mentions "mewt", "muton", or "mutation testing"
  • Wants to configure, scope, or speed up a mutation testing campaign
  • Wants to analyze mutation results — surviving/uncaught mutants, equivalent mutants, kill rate
  • Wants to use mutation results to find bugs in the source code

When NOT to Use

Do not use this skill when the user asks about tests or line coverage without any mutation testing context.


Routing

Pick the workflow, then load it together with the references listed for it. Workflows and references do not load each other — that decision belongs here.

Setting up, scoping, or speeding up a campaign → workflows/configuration.md → Also load references/optimization-strategies.md when the campaign estimate is long enough to need trimming, or the user asks to make it faster.

Campaign finished, hunting for bugs in untested code → workflows/bug-hunter.md

Turning results into a formal analysis report → workflows/analyzing-results.md, plus:

Anything else → run mewt --help or mewt <subcommand> --help, then assist directly.


Essential Commands

# Set up and run
mewt init                    # Create config and database
mewt mutate [paths]          # Generate mutants without testing them
mewt run [paths]             # Generate mutants and run the campaign

# Read results
mewt status                  # Overview with per-file breakdown
mewt results                 # Uncaught mutants (default view)
mewt results --all           # Every outcome, not just uncaught
mewt results --format json   # json | sarif | ids | table

# Narrow down (these filters work on both `results` and `print mutants`)
mewt results --target 'src/auth/**'   # Quote globs so the shell does not expand them
mewt results --severity high,medium
mewt results --mutation-types ER,CR
mewt results --status Uncaught        # Uncaught | TestFail | Skipped | Timeout
mewt results --line 42

# Investigate and re-test
mewt print mutant --id [id]              # View the mutated code
mewt test --ids [ids]                    # Re-test specific mutants
mewt test --ids-file uncaught_ids.txt    # Re-test IDs from a file, or '-' for stdin

# Inspect configuration
mewt print config                        # Effective config
mewt print targets                       # Files actually mutated
mewt print mutations --language [lang]   # Mutations and severities for a language

Language labels are canonical family or family/dialect values in mewt 4.x — for example rust, javascript/ts, move/sui, move/iota.


What Results Mean

  • Caught/TestFail: tests detected the mutation (good)
  • Uncaught: tests did not detect the change. Inspect the code to distinguish a testing gap from an equivalent mutation.
  • Timeout: tests took too long — inconclusive, not evidence of coverage
  • Skipped: a less severe mutant was skipped because a more severe mutant on the same line was uncaught

Interpreting Mutation Types

mewt print mutations --language [lang] lists every mutation slug, description, and severity for a language, and is authoritative — the operator set grows with each release. What that output does not tell you is what a survivor means, which is where prioritization comes from:

SeverityRepresentative slugsWhat an uncaught mutant tells you
HighER (Error Replacement)Tests tolerate the injected error. Investigate whether the path executes, whether error handling masks the change, and whether assertions check the outcome.
MediumCR (Comment Replacement)Removing the statement does not fail the tests. Check whether its effects matter and whether assertions observe them.
MediumIF/IT (If False/True), NR (Negation Removal)Tests do not distinguish the changed condition. Both constant replacements surviving can indicate an unexecuted condition or weak assertions on the branch outcomes.
LowOperator shuffles (AOS, COS, LOS, BOS, shift/assignment variants), BL, AS, LC, WFCheck boundary inputs, arithmetic assertions, and semantic equivalence. The mutation result alone does not establish whether the code executed.

Severity ranks the mutation, not the risk. A low-severity survivor in a fee calculation matters more than a high-severity survivor in a log line — weigh what the mutated code does. Filter with --severity to work through the results in priority order.

GitHub Repository

trailofbits/skills
Path: plugins/mutation-testing/skills/mutation-testing
0
agent-skills
FAQ

Frequently asked questions

What is the mutation-testing skill?

mutation-testing is a Claude Skill by trailofbits. Skills package instructions and resources that Claude loads on demand, so Claude can perform mutation-testing-related tasks without extra prompting.

How do I install mutation-testing?

Use the install commands on this page: add mutation-testing to Claude Code as a plugin, or clone its repository into your skills directory, then restart Claude so it picks up the skill.

What category does mutation-testing belong to?

mutation-testing is in the Testing category.

Is mutation-testing free to use?

Yes. mutation-testing is listed on AIMCP and free to install.

Related Skills

evaluating-llms-harness
Testing

This Claude Skill runs the lm-evaluation-harness to benchmark LLMs across 60+ standardized academic tasks like MMLU and GSM8K. It's designed for developers to compare model quality, track training progress, or report academic results. The tool supports various backends including HuggingFace and vLLM models.

View skill
cloudflare-cron-triggers
Testing

This skill provides comprehensive knowledge for implementing Cloudflare Cron Triggers to schedule Workers using cron expressions. It covers setting up periodic tasks, maintenance jobs, and automated workflows while handling common issues like invalid cron expressions and timezone problems. Developers can use it for configuring scheduled handlers, testing cron triggers, and integrating with Workflows and Green Compute.

View skill
webapp-testing
Testing

This Claude Skill provides a Playwright-based toolkit for testing local web applications through Python scripts. It enables frontend verification, UI debugging, screenshot capture, and log viewing while managing server lifecycles. Use it for browser automation tasks but run scripts directly rather than reading their source code to avoid context pollution.

View skill
finishing-a-development-branch
Testing

This skill helps developers complete finished work by verifying tests pass and then presenting structured integration options. It guides the workflow for merging, creating PRs, or cleaning up branches after implementation is done. Use it when your code is ready and tested to systematically finalize the development process.

View skill