evaluate2.sh: Check output of warmup run and abort early if failed (#333)

* refactor: replace xtrace with "print_and_execute" function

* nit: stylize error messages

* replace out_expected.txt with measurements_1B.out

* print

* prevent errors on cleanup

* run tests and check warmup run output before running benchmark

* move "git diff" pretty diff output to test.sh

* Ensure "set -e" is re-enabled if we followed a "continue" branch

* add timeouts to test.sh invocations

* use diff with tocsv.sh to show differences on failed test

* add --quiet mode to test.sh

* move prepare_$fork.sh invocation to right below hyperfine since test.sh also invokes it

* Revert "add --quiet mode to test.sh"

This reverts commit 13e9fb7f395c1bd64a62528b8349803bc1366941.

* use tee to capture test output to a temp file and print contents on failure

---------

Co-authored-by: Jason Nochlin <hundredwatt@users.noreply.github.com>
This commit is contained in:
Jason Nochlin
2024-01-13 04:19:29 -07:00
committed by GitHub
parent df67791039
commit eff73db9fe
2 changed files with 84 additions and 57 deletions

View File

@@ -45,7 +45,7 @@ for sample in $(ls $INPUT); do
rm -f measurements.txt
ln -s $sample measurements.txt
diff <("./calculate_average_$FORK.sh") ${sample%.txt}.out
diff <("./calculate_average_$FORK.sh" | ./tocsv.sh) <(./tocsv.sh < ${sample%.txt}.out)
done
rm measurements.txt