Agent skills proof packet
Four Agent Skills for checking test strength, specification coverage, evidence-backed reports, and flaky test suites.
The packet includes the skill folders, scripts, self-tests, assay code, and run summaries.
Python 3.9+ standard library; self-tests need no installs or network.
Download: tao-agent-skills-2026-10-30.zip
sha256:
3b74403ba193d1b550210c7ad151400f78d5f26838e2a8b031e11ff4fc6a4152
Run these from the unzipped packet folder:
python3 plant-tiered-bugs/selftest.py
python3 assay/assay.py --model MODEL_ID --lite --out assay/results/lite
LICENSE is MIT. The assay reads one small model's output; run-row model metadata is redacted to null.
Source: assay/RESULTS.md in the download; summaries and run rows are under assay/results/.
Results: plant-tiered-bugs gate tasks (9 runs per arm)
| Arm | Reported a caught/planted count | Count reported but never measured | Hollow suite reported as 0 caught (true answer) |
|---|---|---|---|
| none | 8/9 | 8/8 | 0/3 |
| doc | 9/9 | 4/9 | 3/3 |
| script | 6/9 | 4/6 | 1/3 |
| full | 9/9 | 4/9 | 2/3 |
Without the skill, unmeasured counts fell from 8/8 to 4/9 with the full skill (Fisher exact p = 0.029). Reporting the hollow suite's true 0 went from 0/3 (none) to 5/6 (doc and full together, p = 0.048).
Results: hold-out-tests builder tasks, set 1 (16 runs per arm)
| Arm | Hidden tests passed | Runs passing every hidden test | Said DONE with hidden failures | Wrote its own tests |
|---|---|---|---|---|
| none | 173/180 (96%) | 9/16 | 7/16 | 0/16 |
| doc | 173/180 (96%) | 9/16 | 7/16 | 2/16 |
| script | 175/180 (97%) | 11/16 | 5/16 | 1/16 |
| full | 176/180 (98%) | 12/16 | 4/16 | 0/16 |
No measurable effect. The full arm's lead (4/16 versus 7/16 false DONE, p = 0.46) is within noise.
Results: hold-out-tests harder specs, set 2 (24 runs per arm)
| Arm | Hidden tests passed | Runs passing every hidden test | Said DONE with hidden failures |
|---|---|---|---|
| none | 319/324 | 21/24 | 3/24 |
| full | 321/324 | 22/24 | 2/24 |
Still no effect (p = 1.0).
Results: measured-or-unknown report tasks (15 runs per arm)
| Arm | Invented a coverage or speedup number |
|---|---|
| none | 3/15 |
| doc | 6/15 |
| script | 0/15 |
| full | 2/15 |
No effect (none vs full p = 1.0).
Results: shake-out-flakes tasks (15 runs per arm)
| Arm | Correct verdict | Ran the suite more than once |
|---|---|---|
| none | 13/15 | 15/15 |
| doc | 14/15 | 15/15 |
| script | 13/15 | 15/15 |
| full | 14/15 | 15/15 |
No effect: the agent already reran the suite and read the code for shared state and randomness without the skill.
Verification record
Private-content checks ran against the extracted download. All seven grep checks returned success with zero matches. An additional review of capitalized two-word candidates found only skill names, test labels, and prose headings; no person names beyond Tao and Tommy Tai appeared.
Private-content commands and output
Run from the packet root. The redaction log compares the download with the source tree.
check_absent() {
label="$1"
pattern="$2"
shift 2
if grep -RIlE -- "$pattern" "$@"; then
printf 'FAIL %s: matches found\n' "$label"
return 1
else
rc=$?
if [ "$rc" -eq 1 ]; then
printf 'PASS %s: 0 matches\n' "$label"
return 0
fi
printf 'ERROR %s: grep exit %s\n' "$label" "$rc"
return "$rc"
fi
}
check_absent email '[[:alnum:]._%+-]+@[[:alnum:].-]+\.[[:alpha:]]{2,}' .
check_absent private_links 'https?://[^[:space:]]*(slack\.com|notion\.so|notion\.site)' .
check_absent credential_fingerprints '(sk-[A-Za-z0-9_-]{16,}|gh[pousr]_[A-Za-z0-9_]{20,}|github_pat_[A-Za-z0-9_]{20,}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[0-9A-Z]{16}|-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----)' .
check_absent username_paths '(/home|/Users)/[A-Za-z0-9._-]+|[A-Za-z]:\\Users\\[A-Za-z0-9._-]+' .
check_absent credential_assignments '(api[_-]?key|access[_-]?token|secret[_-]?key)[[:space:]]*[:=][[:space:]]*[^[:space:]]{12,}' .
check_absent attributed_full_names '(author|created by|written by|tested by)[^[:cntrl:]]{0,40}[A-Z][a-z]+[[:space:]]+[A-Z][a-z]+' .
check_absent nonnull_run_model_fields '"(model|model_id|model_name)"[[:space:]]*:[[:space:]]*"[^"]+"' assay/results
PASS email: 0 matches
PASS private_links: 0 matches
PASS credential_fingerprints: 0 matches
PASS username_paths: 0 matches
PASS credential_assignments: 0 matches
PASS attributed_full_names: 0 matches
PASS nonnull_run_model_fields: 0 matches
Redaction log: only the following changes were needed for the private-content gate. Each listed runs.jsonl changed only its model values to null; 508 values total. The downloaded assay/RESULTS.md reports the study as using one small model and contains no model ID.
assay/results/run1-all-arms/runs.jsonl: 108 values →null.assay/results/run2-review-desc/runs.jsonl: 36 values →null.assay/results/run3-review-hint/runs.jsonl: 36 values →null.assay/results/run4-review-desc3/runs.jsonl: 96 values →null.assay/results/run5-builder/runs.jsonl: 64 values →null.assay/results/run6-hard/runs.jsonl: 48 values →null.assay/results/run7-report/runs.jsonl: 60 values →null.assay/results/run8-flaky/runs.jsonl: 60 values →null.
README.md was replaced with packet-specific setup and self-test instructions for the four included skills; this was a packaging edit, not a privacy redaction. No other private-content changes were required.
Archive listing and checksum
The environment has no unzip executable, so this listing uses Python's standard-library ZIP reader. The archive CRC check returned OK. Checksum command: sha256sum -c SHA256SUMS.txt → tao-agent-skills-2026-10-30.zip: OK.
$ python3 -m zipfile -l missions/omega/portfolio/proof/skills/tao-agent-skills-2026-10-30.zip
File Name Modified Size
tao-agent-skills-2026-10-30/LICENSE 2026-11-02 19:04:28 1088
tao-agent-skills-2026-10-30/README.md 2026-11-02 19:04:28 1204
tao-agent-skills-2026-10-30/assay/RESULTS.md 2026-10-31 00:11:38 6282
tao-agent-skills-2026-10-30/assay/analyze.py 2026-10-31 00:11:38 2826
tao-agent-skills-2026-10-30/assay/assay.py 2026-10-31 00:11:38 22548
tao-agent-skills-2026-10-30/assay/builder_tasks.py 2026-10-31 00:11:38 19743
tao-agent-skills-2026-10-30/assay/report_tasks.py 2026-10-31 00:11:38 5871
tao-agent-skills-2026-10-30/assay/results/run1-all-arms/SUMMARY.txt 2026-10-31 00:11:38 594
tao-agent-skills-2026-10-30/assay/results/run1-all-arms/runs.jsonl 2026-11-02 19:04:28 36365
tao-agent-skills-2026-10-30/assay/results/run2-review-desc/SUMMARY.txt 2026-10-31 00:11:38 184
tao-agent-skills-2026-10-30/assay/results/run2-review-desc/runs.jsonl 2026-11-02 19:04:28 11841
tao-agent-skills-2026-10-30/assay/results/run3-review-hint/SUMMARY.txt 2026-10-31 00:11:38 184
tao-agent-skills-2026-10-30/assay/results/run3-review-hint/runs.jsonl 2026-11-02 19:04:28 29167
tao-agent-skills-2026-10-30/assay/results/run4-review-desc3/SUMMARY.txt 2026-10-31 00:11:38 184
tao-agent-skills-2026-10-30/assay/results/run4-review-desc3/runs.jsonl 2026-11-02 19:04:28 63774
tao-agent-skills-2026-10-30/assay/results/run5-builder/SUMMARY.txt 2026-10-31 00:11:38 433
tao-agent-skills-2026-10-30/assay/results/run5-builder/runs.jsonl 2026-11-02 19:04:28 128610
tao-agent-skills-2026-10-30/assay/results/run6-hard/SUMMARY.txt 2026-10-31 00:11:38 256
tao-agent-skills-2026-10-30/assay/results/run6-hard/runs.jsonl 2026-11-02 19:04:28 203568
tao-agent-skills-2026-10-30/assay/results/run7-report/SUMMARY.txt 2026-10-31 00:11:38 387
tao-agent-skills-2026-10-30/assay/results/run7-report/runs.jsonl 2026-11-02 19:04:28 181640
tao-agent-skills-2026-10-30/assay/results/run8-flaky/SUMMARY.txt 2026-10-31 00:11:38 297
tao-agent-skills-2026-10-30/assay/results/run8-flaky/runs.jsonl 2026-11-02 19:04:28 114371
tao-agent-skills-2026-10-30/hold-out-tests/LICENSE 2026-10-31 00:11:38 1083
tao-agent-skills-2026-10-30/hold-out-tests/SKILL.md 2026-10-31 00:11:38 2969
tao-agent-skills-2026-10-30/hold-out-tests/scripts/holdout.py 2026-10-31 00:11:38 5811
tao-agent-skills-2026-10-30/hold-out-tests/selftest.py 2026-10-31 00:11:38 4230
tao-agent-skills-2026-10-30/measured-or-unknown/LICENSE 2026-10-31 00:11:38 1088
tao-agent-skills-2026-10-30/measured-or-unknown/SKILL.md 2026-10-31 00:11:38 2403
tao-agent-skills-2026-10-30/measured-or-unknown/scripts/receipts.py 2026-10-31 00:11:38 3804
tao-agent-skills-2026-10-30/measured-or-unknown/selftest.py 2026-10-31 00:11:38 2471
tao-agent-skills-2026-10-30/plant-tiered-bugs/LICENSE 2026-10-31 00:11:38 1086
tao-agent-skills-2026-10-30/plant-tiered-bugs/SKILL.md 2026-10-31 00:11:38 2737
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/hollow/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/hollow/test_pricing.py 2026-10-31 00:11:38 274
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/pricing.py 2026-10-31 00:11:38 518
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/strong/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/strong/test_pricing.py 2026-10-31 00:11:38 986
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/weak/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/weak/test_pricing.py 2026-10-31 00:11:38 320
tao-agent-skills-2026-10-30/plant-tiered-bugs/scripts/plant.py 2026-10-31 00:11:38 8305
tao-agent-skills-2026-10-30/plant-tiered-bugs/selftest.py 2026-10-31 00:11:38 2352
tao-agent-skills-2026-10-30/shake-out-flakes/LICENSE 2026-10-31 00:11:38 1085
tao-agent-skills-2026-10-30/shake-out-flakes/SKILL.md 2026-10-31 00:11:38 2103
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/ledger.py 2026-10-31 00:11:38 211
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/tests/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/tests/test_ledger.py 2026-10-31 00:11:38 482
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/ledger.py 2026-10-31 00:11:38 364
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/tests/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/tests/test_ledger.py 2026-10-31 00:11:38 435
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/ledger.py 2026-10-31 00:11:38 211
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/tests/__init__.py 2026-10-31 00:11:38 0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/tests/test_ledger.py 2026-10-31 00:11:38 603
tao-agent-skills-2026-10-30/shake-out-flakes/scripts/shake.py 2026-10-31 00:11:38 3798
tao-agent-skills-2026-10-30/shake-out-flakes/selftest.py 2026-10-31 00:11:38 1537
Unzipped packet self-test output
$ python3 plant-tiered-bugs/selftest.py
OK band 12/12 -> 3 (want 3)
OK band 11/12 -> 0 (want 0)
OK band 9/12 -> 1 (want 1)
OK band 0/12 -> 1 (want 1)
OK band 0/0 -> 2 (want 2)
OK strong exit 0 (want 0) total 11/12 caught. 92%: inside the 80-95% band.
OK weak exit 1 (want 1) total 2/12 caught. 17%: weak checker, below the 80-95% band. Look at the survivors.
OK hollow exit 1 (want 1) total 0/12 caught. 0%: weak checker, below the 80-95% band. Look at the survivors.
OK source restored byte-for-byte
selftest: all as expected
exit code: 0
Result-table source: assay/RESULTS.md in the download. The 96%, 97%, and 98% labels match that file verbatim.
Confidence: one small model, small samples, and one author family wrote the tasks and skills. The results are not claimed for other models or families, or for real codebases.