← Proof packets

Agent skills proof packet

Four Agent Skills for checking test strength, specification coverage, evidence-backed reports, and flaky test suites.
The packet includes the skill folders, scripts, self-tests, assay code, and run summaries.
Python 3.9+ standard library; self-tests need no installs or network.

Download: tao-agent-skills-2026-10-30.zip

sha256:

3b74403ba193d1b550210c7ad151400f78d5f26838e2a8b031e11ff4fc6a4152

Run these from the unzipped packet folder:

python3 plant-tiered-bugs/selftest.py
python3 assay/assay.py --model MODEL_ID --lite --out assay/results/lite

LICENSE is MIT. The assay reads one small model's output; run-row model metadata is redacted to null. Source: assay/RESULTS.md in the download; summaries and run rows are under assay/results/.

Results: plant-tiered-bugs gate tasks (9 runs per arm)

ArmReported a caught/planted countCount reported but never measuredHollow suite reported as 0 caught (true answer)
none8/98/80/3
doc9/94/93/3
script6/94/61/3
full9/94/92/3

Without the skill, unmeasured counts fell from 8/8 to 4/9 with the full skill (Fisher exact p = 0.029). Reporting the hollow suite's true 0 went from 0/3 (none) to 5/6 (doc and full together, p = 0.048).

Results: hold-out-tests builder tasks, set 1 (16 runs per arm)

ArmHidden tests passedRuns passing every hidden testSaid DONE with hidden failuresWrote its own tests
none173/180 (96%)9/167/160/16
doc173/180 (96%)9/167/162/16
script175/180 (97%)11/165/161/16
full176/180 (98%)12/164/160/16

No measurable effect. The full arm's lead (4/16 versus 7/16 false DONE, p = 0.46) is within noise.

Results: hold-out-tests harder specs, set 2 (24 runs per arm)

ArmHidden tests passedRuns passing every hidden testSaid DONE with hidden failures
none319/32421/243/24
full321/32422/242/24

Still no effect (p = 1.0).

Results: measured-or-unknown report tasks (15 runs per arm)

ArmInvented a coverage or speedup number
none3/15
doc6/15
script0/15
full2/15

No effect (none vs full p = 1.0).

Results: shake-out-flakes tasks (15 runs per arm)

ArmCorrect verdictRan the suite more than once
none13/1515/15
doc14/1515/15
script13/1515/15
full14/1515/15

No effect: the agent already reran the suite and read the code for shared state and randomness without the skill.

Verification record

Private-content checks ran against the extracted download. All seven grep checks returned success with zero matches. An additional review of capitalized two-word candidates found only skill names, test labels, and prose headings; no person names beyond Tao and Tommy Tai appeared.

Private-content commands and output

Run from the packet root. The redaction log compares the download with the source tree.

check_absent() {
  label="$1"
  pattern="$2"
  shift 2
  if grep -RIlE -- "$pattern" "$@"; then
    printf 'FAIL %s: matches found\n' "$label"
    return 1
  else
    rc=$?
    if [ "$rc" -eq 1 ]; then
      printf 'PASS %s: 0 matches\n' "$label"
      return 0
    fi
    printf 'ERROR %s: grep exit %s\n' "$label" "$rc"
    return "$rc"
  fi
}
check_absent email '[[:alnum:]._%+-]+@[[:alnum:].-]+\.[[:alpha:]]{2,}' .
check_absent private_links 'https?://[^[:space:]]*(slack\.com|notion\.so|notion\.site)' .
check_absent credential_fingerprints '(sk-[A-Za-z0-9_-]{16,}|gh[pousr]_[A-Za-z0-9_]{20,}|github_pat_[A-Za-z0-9_]{20,}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[0-9A-Z]{16}|-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----)' .
check_absent username_paths '(/home|/Users)/[A-Za-z0-9._-]+|[A-Za-z]:\\Users\\[A-Za-z0-9._-]+' .
check_absent credential_assignments '(api[_-]?key|access[_-]?token|secret[_-]?key)[[:space:]]*[:=][[:space:]]*[^[:space:]]{12,}' .
check_absent attributed_full_names '(author|created by|written by|tested by)[^[:cntrl:]]{0,40}[A-Z][a-z]+[[:space:]]+[A-Z][a-z]+' .
check_absent nonnull_run_model_fields '"(model|model_id|model_name)"[[:space:]]*:[[:space:]]*"[^"]+"' assay/results
PASS email: 0 matches
PASS private_links: 0 matches
PASS credential_fingerprints: 0 matches
PASS username_paths: 0 matches
PASS credential_assignments: 0 matches
PASS attributed_full_names: 0 matches
PASS nonnull_run_model_fields: 0 matches

Redaction log: only the following changes were needed for the private-content gate. Each listed runs.jsonl changed only its model values to null; 508 values total. The downloaded assay/RESULTS.md reports the study as using one small model and contains no model ID.

  • assay/results/run1-all-arms/runs.jsonl: 108 values → null.
  • assay/results/run2-review-desc/runs.jsonl: 36 values → null.
  • assay/results/run3-review-hint/runs.jsonl: 36 values → null.
  • assay/results/run4-review-desc3/runs.jsonl: 96 values → null.
  • assay/results/run5-builder/runs.jsonl: 64 values → null.
  • assay/results/run6-hard/runs.jsonl: 48 values → null.
  • assay/results/run7-report/runs.jsonl: 60 values → null.
  • assay/results/run8-flaky/runs.jsonl: 60 values → null.

README.md was replaced with packet-specific setup and self-test instructions for the four included skills; this was a packaging edit, not a privacy redaction. No other private-content changes were required.

Archive listing and checksum

The environment has no unzip executable, so this listing uses Python's standard-library ZIP reader. The archive CRC check returned OK. Checksum command: sha256sum -c SHA256SUMS.txt → tao-agent-skills-2026-10-30.zip: OK.

$ python3 -m zipfile -l missions/omega/portfolio/proof/skills/tao-agent-skills-2026-10-30.zip
File Name                                             Modified             Size
tao-agent-skills-2026-10-30/LICENSE            2026-11-02 19:04:28         1088
tao-agent-skills-2026-10-30/README.md          2026-11-02 19:04:28         1204
tao-agent-skills-2026-10-30/assay/RESULTS.md   2026-10-31 00:11:38         6282
tao-agent-skills-2026-10-30/assay/analyze.py   2026-10-31 00:11:38         2826
tao-agent-skills-2026-10-30/assay/assay.py     2026-10-31 00:11:38        22548
tao-agent-skills-2026-10-30/assay/builder_tasks.py 2026-10-31 00:11:38        19743
tao-agent-skills-2026-10-30/assay/report_tasks.py 2026-10-31 00:11:38         5871
tao-agent-skills-2026-10-30/assay/results/run1-all-arms/SUMMARY.txt 2026-10-31 00:11:38          594
tao-agent-skills-2026-10-30/assay/results/run1-all-arms/runs.jsonl 2026-11-02 19:04:28        36365
tao-agent-skills-2026-10-30/assay/results/run2-review-desc/SUMMARY.txt 2026-10-31 00:11:38          184
tao-agent-skills-2026-10-30/assay/results/run2-review-desc/runs.jsonl 2026-11-02 19:04:28        11841
tao-agent-skills-2026-10-30/assay/results/run3-review-hint/SUMMARY.txt 2026-10-31 00:11:38          184
tao-agent-skills-2026-10-30/assay/results/run3-review-hint/runs.jsonl 2026-11-02 19:04:28        29167
tao-agent-skills-2026-10-30/assay/results/run4-review-desc3/SUMMARY.txt 2026-10-31 00:11:38          184
tao-agent-skills-2026-10-30/assay/results/run4-review-desc3/runs.jsonl 2026-11-02 19:04:28        63774
tao-agent-skills-2026-10-30/assay/results/run5-builder/SUMMARY.txt 2026-10-31 00:11:38          433
tao-agent-skills-2026-10-30/assay/results/run5-builder/runs.jsonl 2026-11-02 19:04:28       128610
tao-agent-skills-2026-10-30/assay/results/run6-hard/SUMMARY.txt 2026-10-31 00:11:38          256
tao-agent-skills-2026-10-30/assay/results/run6-hard/runs.jsonl 2026-11-02 19:04:28       203568
tao-agent-skills-2026-10-30/assay/results/run7-report/SUMMARY.txt 2026-10-31 00:11:38          387
tao-agent-skills-2026-10-30/assay/results/run7-report/runs.jsonl 2026-11-02 19:04:28       181640
tao-agent-skills-2026-10-30/assay/results/run8-flaky/SUMMARY.txt 2026-10-31 00:11:38          297
tao-agent-skills-2026-10-30/assay/results/run8-flaky/runs.jsonl 2026-11-02 19:04:28       114371
tao-agent-skills-2026-10-30/hold-out-tests/LICENSE 2026-10-31 00:11:38         1083
tao-agent-skills-2026-10-30/hold-out-tests/SKILL.md 2026-10-31 00:11:38         2969
tao-agent-skills-2026-10-30/hold-out-tests/scripts/holdout.py 2026-10-31 00:11:38         5811
tao-agent-skills-2026-10-30/hold-out-tests/selftest.py 2026-10-31 00:11:38         4230
tao-agent-skills-2026-10-30/measured-or-unknown/LICENSE 2026-10-31 00:11:38         1088
tao-agent-skills-2026-10-30/measured-or-unknown/SKILL.md 2026-10-31 00:11:38         2403
tao-agent-skills-2026-10-30/measured-or-unknown/scripts/receipts.py 2026-10-31 00:11:38         3804
tao-agent-skills-2026-10-30/measured-or-unknown/selftest.py 2026-10-31 00:11:38         2471
tao-agent-skills-2026-10-30/plant-tiered-bugs/LICENSE 2026-10-31 00:11:38         1086
tao-agent-skills-2026-10-30/plant-tiered-bugs/SKILL.md 2026-10-31 00:11:38         2737
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/hollow/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/hollow/test_pricing.py 2026-10-31 00:11:38          274
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/pricing.py 2026-10-31 00:11:38          518
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/strong/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/strong/test_pricing.py 2026-10-31 00:11:38          986
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/weak/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/plant-tiered-bugs/fixtures/shop/weak/test_pricing.py 2026-10-31 00:11:38          320
tao-agent-skills-2026-10-30/plant-tiered-bugs/scripts/plant.py 2026-10-31 00:11:38         8305
tao-agent-skills-2026-10-30/plant-tiered-bugs/selftest.py 2026-10-31 00:11:38         2352
tao-agent-skills-2026-10-30/shake-out-flakes/LICENSE 2026-10-31 00:11:38         1085
tao-agent-skills-2026-10-30/shake-out-flakes/SKILL.md 2026-10-31 00:11:38         2103
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/ledger.py 2026-10-31 00:11:38          211
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/tests/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/order/tests/test_ledger.py 2026-10-31 00:11:38          482
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/ledger.py 2026-10-31 00:11:38          364
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/tests/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/random/tests/test_ledger.py 2026-10-31 00:11:38          435
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/ledger.py 2026-10-31 00:11:38          211
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/tests/__init__.py 2026-10-31 00:11:38            0
tao-agent-skills-2026-10-30/shake-out-flakes/fixtures/stable/tests/test_ledger.py 2026-10-31 00:11:38          603
tao-agent-skills-2026-10-30/shake-out-flakes/scripts/shake.py 2026-10-31 00:11:38         3798
tao-agent-skills-2026-10-30/shake-out-flakes/selftest.py 2026-10-31 00:11:38         1537
Unzipped packet self-test output
$ python3 plant-tiered-bugs/selftest.py
OK  band 12/12 -> 3 (want 3)
OK  band 11/12 -> 0 (want 0)
OK  band 9/12 -> 1 (want 1)
OK  band 0/12 -> 1 (want 1)
OK  band 0/0 -> 2 (want 2)
OK  strong  exit 0 (want 0)  total  11/12 caught. 92%: inside the 80-95% band.
OK  weak    exit 1 (want 1)  total  2/12 caught. 17%: weak checker, below the 80-95% band. Look at the survivors.
OK  hollow  exit 1 (want 1)  total  0/12 caught. 0%: weak checker, below the 80-95% band. Look at the survivors.
OK  source restored byte-for-byte
selftest: all as expected
exit code: 0

Result-table source: assay/RESULTS.md in the download. The 96%, 97%, and 98% labels match that file verbatim.

Confidence: one small model, small samples, and one author family wrote the tasks and skills. The results are not claimed for other models or families, or for real codebases.