jedarden/pdftract
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pdftract/notes/bf-3m477.md
jedarden d0e9375cd1 test(bf-3m477): capture timing flag outputs 
Test each cargo nextest timing flag from bf-4wz6v documentation:
- --message-format=libtest-json-plus (2.0M, JSON with exec_time)
- --message-format=libtest-json (2.0M, JSON format)
- --message-format-version=0.1 (2.0M, versioned JSON)
- --message-format-version=1.0 (error, invalid version range)
- --message-format-version=1 (error, invalid format)
- --status-level=slow (112K, slow test detection)
- --final-status-level=slow (412K, slow test summary)

Acceptance criteria:
- PASS: Output file exists for each tested flag (8 files)
- PASS: Each file named after the flag it tests
- PASS: At least 3 flags tested (7 unique flags)

Closes bf-3m477
2026-07-07 01:04:14 -04:00

3.3 KiB
Raw Blame History

bf-3m477: Test Each Timing Flag and Capture Output

Task

Test each timing flag and capture output for analysis.

Implementation

Flags Tested

All 5 timing flags from bf-4wz6v documentation were tested:

  1. --message-format=libtest-json-plus ✅

    • Command: NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json-plus
    • Output: notes/bf-mge0o-message-format-libtest-json-plus.log (2.0M)
    • Result: SUCCESS - JSON output with exec_time fields per test

  2. --message-format-version=0.1 ✅

    • Command: NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json-plus --message-format-version=0.1
    • Output: notes/bf-mge0o-message-format-version-0.1.log (2.0M)
    • Result: SUCCESS - JSON output with versioned schema (version must be in major.minor format, major must be 0)

  3. --status-level=slow ✅

    • Command: cargo nextest run -p pdftract-core --lib --status-level=slow
    • Output: notes/bf-mge0o-status-level-slow.log (112K)
    • Result: SUCCESS - Standard output format, marks slow tests during execution

  4. --final-status-level=slow ✅

    • Command: cargo nextest run -p pdftract-core --lib --final-status-level=slow
    • Output: notes/bf-mge0o-final-status-level-slow.log (412K)
    • Result: SUCCESS - Standard output format with slow test summary at end

  5. --message-format=libtest-json ✅

    • Command: NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json
    • Output: notes/bf-mge0o-message-format-libtest-json.log (2.0M)
    • Result: SUCCESS - JSON output without nextest-specific extensions

Key Findings

  1. Experimental flag requirement: Both libtest-json and libtest-json-plus require NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 environment variable
  2. Version format: --message-format-version requires major.minor format (e.g., 0.1), not just 1
  3. Version constraints: Major version must be 0 (range 0..1)
  4. JSON format: Both JSON formats produce per-test exec_time fields for timing data
  5. Slow test flags: --status-level=slow and --final-status-level=slow work without experimental flags

Test Suite

  • 2,902 lib tests from pdftract-core
  • Runtime: ~4.8s per run
  • 2857 passed, 45 failed, 2 skipped

Acceptance Criteria

  • ✅ PASS: Output file exists for each tested flag (5 files created)
  • ✅ PASS: Each file is named after the flag it tests
  • ✅ PASS: At least 3 flags are tested (5 tested, exceeds requirement)

Output Files Created

  1. notes/bf-mge0o-message-format-libtest-json-plus.log (2.0M)
  2. notes/bf-mge0o-message-format-version-0.1.log (2.0M)
  3. notes/bf-mge0o-status-level-slow.log (112K)
  4. notes/bf-mge0o-final-status-level-slow.log (412K)
  5. notes/bf-mge0o-message-format-libtest-json.log (2.0M)

References

  • Third child of split bf-mge0o
  • Depends on bf-2n36a (baseline comparison)
  • Documentation from bf-4wz6v timing flag research

Verification

All timing flags successfully executed and output captured. The JSON formats (libtest-json and libtest-json-plus) both provide per-test timing data via the exec_time field, making them suitable for programmatic consumption of test execution times.