Back to Skills

run-puzzle-tests

pjt222
Updated 2 days ago
7 views
17
2
17
View on GitHub
Testingaitestingdesign

About

This skill runs the jigsawR test suite through WSL R execution, supporting full tests, pattern filtering, or single file runs. It interprets pass/fail/skip results and identifies failures while avoiding the --vanilla flag for proper renv activation. Developers should use it after code changes, before commits, or when debugging specific test failures.

Quick Install

Claude Code

Recommended
Primary
npx skills add pjt222/agent-almanac -a claude-code
Plugin CommandAlternative
/plugin add https://github.com/pjt222/agent-almanac
Git CloneAlternative
git clone https://github.com/pjt222/agent-almanac.git ~/.claude/skills/run-puzzle-tests

Copy and paste this command in Claude Code to install this skill

Documentation

Run Puzzle Tests

Run jigsawR test suite. Read results.

When Use

  • After modifying R source in package
  • After adding new puzzle type or feature
  • Before commit to verify nothing broken
  • Debugging specific test failure

Inputs

  • Required: Test scope (full, filtered, single)
  • Optional: Filter pattern (filtered mode, e.g. "snic", "rectangular")
  • Optional: Specific test file path (single mode)

Steps

Step 1: Choose Test Scope

ScopeUse whenDuration
FullBefore commits, after major changes~2-5 min
FilteredWorking on one puzzle type~30s
SingleDebugging a specific test file~10s

Got: Scope selected by workflow: full before commits, filtered for one type, single for one debug.

If fail: Unsure? Default to full. Slower but catches cross-type regressions.

Step 2: Create and Execute Test Script

Full suite.

Make script (e.g., /tmp/run_tests.R).

devtools::test()
R_EXE="/mnt/c/Program Files/R/R-4.5.0/bin/Rscript.exe"
cd /mnt/d/dev/p/jigsawR && "$R_EXE" -e "devtools::test()"

Filtered by pattern.

"$R_EXE" -e "devtools::test(filter = 'snic')"

Single file.

"$R_EXE" -e "testthat::test_file('tests/testthat/test-snic-puzzles.R')"

Got: Test output with pass/fail/skip counts.

If fail:

  • Do NOT use --vanilla flag; renv needs .Rprofile to activate
  • renv errors? Run renv::restore() first
  • Complex commands fail with Exit 5? Write to script file

Step 3: Interpret Results

Look for summary line.

[ FAIL 0 | WARN 0 | SKIP 7 | PASS 2042 ]
  • PASS: Tests succeeded
  • FAIL: Tests failed (need investigation)
  • SKIP: Tests skipped (usually missing optional packages like snic)
  • WARN: Warnings during tests (review but not blocking)

Got: Summary line parsed for PASS, FAIL, SKIP, WARN. FAIL = 0 = clean run.

If fail: Summary not visible? Runner crashed before completing. Check R-level errors above. Output truncated? Redirect to file: "$R_EXE" -e "devtools::test()" > test_results.txt 2>&1.

Step 4: Investigate Failures

If tests fail.

  1. Read failure msg — includes file, line, expected vs actual
  2. Check if new failure or pre-existing
  3. Assertion failures: read test + function being tested
  4. Error failures: check function signature changed
# Run just the failing test with verbose output
"$R_EXE" -e "testthat::test_file('tests/testthat/test-failing.R', reporter = 'summary')"

Got: Root cause of each failing test identified. Failure = real regression (fix code) or test env issue (missing dep, path).

If fail: Failure msg unclear? Add browser() or print() to test, re-run with testthat::test_file() for interactive debug.

Step 5: Verify Skip Reasons

Skipped tests normal when optional deps missing.

  • snic package tests skip with skip_if_not_installed("snic")
  • Tests needing specific OS skip with skip_on_os()
  • CRAN-only skips with skip_on_cran()

Confirm skip reasons legitimate, not masking real failures.

Got: All skips accounted for by legitimate reasons (optional dep, platform skip, CRAN-only). No skips masking actual failures.

If fail: Skip suspicious? Temporarily remove skip_if_*() and run test to see pass or hidden failure.

Checks

  • All tests pass (FAIL = 0)
  • No unexpected warnings
  • Skip count matches expected (only optional dep skips)
  • Test count not decreased (no tests removed by accident)

Pitfalls

  • Use --vanilla: Breaks renv activation. Never with jigsawR.
  • Complex -e strings: Shell escaping = Exit 5. Use script files.
  • Stale package state: Run devtools::load_all() or devtools::document() before testing if NAMESPACE-affecting code changed.
  • Missing test deps: Some tests need suggested packages. Check DESCRIPTION Suggests.
  • Parallel test issues: Tests interfere? Run sequential with testthat::test_file().

See Also

  • generate-puzzle — generate puzzles to verify behavior matches tests
  • add-puzzle-type — new types need comprehensive test suites
  • write-testthat-tests — general patterns for writing R tests
  • validate-piles-notation — test PILES parsing independently

GitHub Repository

pjt222/agent-almanac
Path: i18n/caveman/skills/run-puzzle-tests
0
agentsagentskillsai-assisted-developmentclaude-codeskillsteams

Related Skills

evaluating-llms-harness

Testing

This Claude Skill runs the lm-evaluation-harness to benchmark LLMs across 60+ standardized academic tasks like MMLU and GSM8K. It's designed for developers to compare model quality, track training progress, or report academic results. The tool supports various backends including HuggingFace and vLLM models.

View skill

cloudflare-cron-triggers

Testing

This skill provides comprehensive knowledge for implementing Cloudflare Cron Triggers to schedule Workers using cron expressions. It covers setting up periodic tasks, maintenance jobs, and automated workflows while handling common issues like invalid cron expressions and timezone problems. Developers can use it for configuring scheduled handlers, testing cron triggers, and integrating with Workflows and Green Compute.

View skill

webapp-testing

Testing

This Claude Skill provides a Playwright-based toolkit for testing local web applications through Python scripts. It enables frontend verification, UI debugging, screenshot capture, and log viewing while managing server lifecycles. Use it for browser automation tasks but run scripts directly rather than reading their source code to avoid context pollution.

View skill

finishing-a-development-branch

Testing

This skill helps developers complete finished work by verifying tests pass and then presenting structured integration options. It guides the workflow for merging, creating PRs, or cleaning up branches after implementation is done. Use it when your code is ready and tested to systematically finalize the development process.

View skill