learntocloudlearntocloud/linux-ctfs/.github/skills/ctf-testing/SKILL.md

ctf-testing

Skill325 starsChanged 5 months agoMITView on GitHub

Deploy and test Linux CTF challenges across cloud providers (AWS, Azure, GCP). Use when running basic tests without reboot or full tests with reboot for CTF setup, challenge validation, and release confidence.

Installs packages

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 7 are distinctive, and 2 are very specific to this project.

≈0
Common · estimate
Many files like this have a rule of this kind.
≈7
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.

Common
Distinctive
Very specific

Distinctive:
Use this skill when the user asks things like:
line 19
Distinctive:
Verify no test resources remain after the script exits.
line 65
Very specific:
If resources remain, run provider-specific cleanup or terraform destroy from the provider directory.
line 66
Distinctive:
If cleanup still fails, report the remaining resources clearly.
line 67
Distinctive:
When a run fails, identify the first failing layer:
line 76
Distinctive:
Do not create committed solution files.
line 89
Very specific:
Keep solution commands only in .github/skills/ctf-testing/test_ctf_challenges.sh.
line 90
Distinctive:
Do not add flags or challenge solutions to learner-facing docs.
line 91
Distinctive:
Do not make challenges easier while fixing tests.
line 92

≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.

Files

SKILL.md

CTF Challenge Testing

This skill deploys CTF infrastructure to cloud providers and validates that learners can complete the lab.

When to Use

Testing changes to ctf_setup.sh, setup/, or verify/
Validating challenge setup across AWS, Azure, or GCP
Running the full contributor-mode test suite before releases
Verifying services survive VM reboots

Trigger Examples

Use this skill when the user asks things like:

"Run a basic test on Azure"
"Run a basic test on GCP"
"Run a basic test on AWS"
"Run a full test on Azure"
"Run a full test on GCP"
"Run a full test on AWS"
"Run a basic test on all providers"
"Run a full test on all providers"

Prerequisites

terraform (>= 1.0)
jq - Install: sudo apt install jq (Ubuntu) or brew install jq (macOS)
sshpass - Install on macOS: brew install hudochenkov/sshpass/sshpass
Cloud CLI authenticated for target provider:
- AWS: aws sts get-caller-identity
- Azure: az account show
- GCP: gcloud auth list --filter=status:ACTIVE

Agent Workflow

Determine the mode and provider.
- Basic test means no reboot. Use this when the user asks for a basic test or does not mention reboot.
- Full test means reboot. Use this when the user asks for a full test or asks to verify reboot behavior.
- Provider must be aws, azure, gcp, or all. Azure is usually the fastest single-provider test.
Check provider authentication before deploying.
- AWS: aws sts get-caller-identity
- Azure: az account show
- GCP: gcloud auth list --filter=status:ACTIVE

Run the requested test from the repository root.

User request	Command	What gets tested
Basic test on AWS	`./.github/skills/ctf-testing/deploy_and_test.sh aws`	Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time
Basic test on Azure	`./.github/skills/ctf-testing/deploy_and_test.sh azure`	Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time
Basic test on GCP	`./.github/skills/ctf-testing/deploy_and_test.sh gcp`	Verify command sanity checks, all 18 challenges, export certificate, token format, username, challenge count, and frozen completion time
Basic test on all providers	`./.github/skills/ctf-testing/deploy_and_test.sh all`	Same basic checks across AWS, Azure, and GCP
Full test on AWS	`./.github/skills/ctf-testing/deploy_and_test.sh aws --with-reboot`	Basic checks plus reboot, required service checks, and progress persistence
Full test on Azure	`./.github/skills/ctf-testing/deploy_and_test.sh azure --with-reboot`	Basic checks plus reboot, required service checks, and progress persistence
Full test on GCP	`./.github/skills/ctf-testing/deploy_and_test.sh gcp --with-reboot`	Basic checks plus reboot, required service checks, and progress persistence
Full test on all providers	`./.github/skills/ctf-testing/deploy_and_test.sh all --with-reboot`	Same full checks across AWS, Azure, and GCP

The deployment script uses contributor mode, so Terraform uploads the local setup package and runs the code from your working tree. It does not test GitHub Release assets.

Treat cleanup as part of the test, not an optional follow-up.
- Verify no test resources remain after the script exits.
- If resources remain, run provider-specific cleanup or terraform destroy from the provider directory.
- If cleanup still fails, report the remaining resources clearly.
Report the result plainly.
- Include provider, mode, pass/fail result, cleanup status, and blocker if any.
- A successful basic run shows about 27 tests passing.
- A successful full run includes a second pass after reboot with 5 service checks and 1 progress persistence check.
- Summary line: RESULT: PASS (<providers>) or RESULT: FAIL (<providers>).

Failure Handling

When a run fails, identify the first failing layer:

Failure point	What to collect
Terraform failed	Provider name and Terraform error
SSH failed	VM IP and SSH wait output
Setup failed	`/var/lib/linux-ctfs/setup.failed`, `/var/log/ctf_setup.log`, and `/var/log/cloud-init-output.log`
Challenge failed	Challenge number and failing test output
Reboot failed	Failed service name and `journalctl -u <service-name>` output
Cleanup failed	Remaining cloud resources

CTF Safety Rules

Do not create committed solution files.
Keep solution commands only in .github/skills/ctf-testing/test_ctf_challenges.sh.
Do not add flags or challenge solutions to learner-facing docs.
Do not make challenges easier while fixing tests.

Scripts

Reference files - treat as black boxes, run directly:

📄 deploy_and_test.sh - Orchestration script (runs locally)
📄 test_ctf_challenges.sh - VM test script (copied to and runs on deployed VM)

Download now (ZIP)

In this repository3 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .github/copilot-instructions.mdCopilot instructions
- review-lab-maintenanceSkill

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free