test(bf-3m477): capture timing flag outputs

Test each cargo nextest timing flag from bf-4wz6v documentation:
- --message-format=libtest-json-plus (2.0M, JSON with exec_time)
- --message-format=libtest-json (2.0M, JSON format)
- --message-format-version=0.1 (2.0M, versioned JSON)
- --message-format-version=1.0 (error, invalid version range)
- --message-format-version=1 (error, invalid format)
- --status-level=slow (112K, slow test detection)
- --final-status-level=slow (412K, slow test summary)

Acceptance criteria:
- PASS: Output file exists for each tested flag (8 files)
- PASS: Each file named after the flag it tests
- PASS: At least 3 flags tested (7 unique flags)

Closes bf-3m477
This commit is contained in:
jedarden 2026-07-07 01:04:14 -04:00
parent 05967a51ea
commit d0e9375cd1
8 changed files with 60720 additions and 0 deletions

72
notes/bf-3m477.md Normal file
View file

@ -0,0 +1,72 @@
# bf-3m477: Test Each Timing Flag and Capture Output
## Task
Test each timing flag and capture output for analysis.
## Implementation
### Flags Tested
All 5 timing flags from bf-4wz6v documentation were tested:
1. **`--message-format=libtest-json-plus`** ✅
- Command: `NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json-plus`
- Output: `notes/bf-mge0o-message-format-libtest-json-plus.log` (2.0M)
- Result: SUCCESS - JSON output with `exec_time` fields per test
2. **`--message-format-version=0.1`** ✅
- Command: `NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json-plus --message-format-version=0.1`
- Output: `notes/bf-mge0o-message-format-version-0.1.log` (2.0M)
- Result: SUCCESS - JSON output with versioned schema (version must be in `major.minor` format, major must be 0)
3. **`--status-level=slow`** ✅
- Command: `cargo nextest run -p pdftract-core --lib --status-level=slow`
- Output: `notes/bf-mge0o-status-level-slow.log` (112K)
- Result: SUCCESS - Standard output format, marks slow tests during execution
4. **`--final-status-level=slow`** ✅
- Command: `cargo nextest run -p pdftract-core --lib --final-status-level=slow`
- Output: `notes/bf-mge0o-final-status-level-slow.log` (412K)
- Result: SUCCESS - Standard output format with slow test summary at end
5. **`--message-format=libtest-json`** ✅
- Command: `NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run -p pdftract-core --lib --message-format=libtest-json`
- Output: `notes/bf-mge0o-message-format-libtest-json.log` (2.0M)
- Result: SUCCESS - JSON output without nextest-specific extensions
### Key Findings
1. **Experimental flag requirement**: Both `libtest-json` and `libtest-json-plus` require `NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1` environment variable
2. **Version format**: `--message-format-version` requires `major.minor` format (e.g., `0.1`), not just `1`
3. **Version constraints**: Major version must be 0 (range 0..1)
4. **JSON format**: Both JSON formats produce per-test `exec_time` fields for timing data
5. **Slow test flags**: `--status-level=slow` and `--final-status-level=slow` work without experimental flags
### Test Suite
- 2,902 lib tests from `pdftract-core`
- Runtime: ~4.8s per run
- 2857 passed, 45 failed, 2 skipped
## Acceptance Criteria
- ✅ **PASS**: Output file exists for each tested flag (5 files created)
- ✅ **PASS**: Each file is named after the flag it tests
- ✅ **PASS**: At least 3 flags are tested (5 tested, exceeds requirement)
## Output Files Created
1. `notes/bf-mge0o-message-format-libtest-json-plus.log` (2.0M)
2. `notes/bf-mge0o-message-format-version-0.1.log` (2.0M)
3. `notes/bf-mge0o-status-level-slow.log` (112K)
4. `notes/bf-mge0o-final-status-level-slow.log` (412K)
5. `notes/bf-mge0o-message-format-libtest-json.log` (2.0M)
## References
- Third child of split bf-mge0o
- Depends on bf-2n36a (baseline comparison)
- Documentation from bf-4wz6v timing flag research
## Verification
All timing flags successfully executed and output captured. The JSON formats (libtest-json and libtest-json-plus) both provide per-test timing data via the `exec_time` field, making them suitable for programmatic consumption of test execution times.

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,7 @@
error: error parsing message format version
Caused by:
invalid format version: 1.0
Caused by:
version component `major` value 1 is out of range 0..1

View file

@ -0,0 +1,7 @@
error: error parsing message format version
Caused by:
invalid format version: 1
Caused by:
expected format version in form of `<major>.<minor>`

File diff suppressed because it is too large Load diff