Playground/1brc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
09e0311e09
1brc/test_all.sh

80 lines
2.3 KiB
Bash
Raw Normal View History

test: add script to test all implementations The script tests all implementations and prints PASS or FAIL status. In case of failure it also prints implementation output to stderr. This will be handy for adding new test samples. Show test statuses and omit failing output: ```sh $ ./test_all.sh 2>/dev/null PASS artsiomkorzun PASS baseline PASS bjhara PASS criccomini FAIL ddimtirov FAIL ebarlas PASS filiphr FAIL itaske PASS khmarbaise FAIL kuduwa-keshavram FAIL lawrey PASS padreati FAIL palmr PASS richardstartin FAIL royvanrijn FAIL seijikun PASS spullara PASS truelive ``` Show only passing implementations: ``` $ ./test_all.sh 2>/dev/null | grep PASS | cut -d' ' -f2 artsiomkorzun baseline bjhara criccomini filiphr khmarbaise padreati richardstartin spullara truelive ``` For #61
2024-01-04 20:36:20 +01:00
#!/bin/bash
#
# Copyright 2023 The original authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
set -euo pipefail
Update test*.sh to support input file pattern This is useful for testing fork(s) against subset of test samples
2024-01-10 17:01:33 +01:00
INPUT=${1:-""}
if [ "$INPUT" = "-h" ] || [ "$#" -gt 1 ]; then
echo "Usage: ./test_all.sh [input file pattern]"
echo
echo "For each available fork run ./test.sh <fork name> [input file pattern]."
echo "Note that optional <input file pattern> should be quoted if contains wild cards."
echo
echo "Examples:"
echo "./test_all.sh"
echo "./test_all.sh 2>/dev/null"
echo "./test_all.sh src/test/resources/samples/measurements-1.txt"
echo "./test_all.sh 'src/test/resources/samples/measurements-*.txt'"
exit 1
fi
test_all.sh: configure fork test timeout Example output: ``` $ ./test_all.sh 2>/dev/null PASS abfrmblr PASS AbstractKamen PASS ags313 PASS alesj PASS algirdasrascius PASS anandmattikopp PASS arjenvaneerde FAIL armandino TIME artpar PASS artsiomkorzun PASS asun PASS baseline PASS berry120 PASS bjhara PASS charlibot PASS coolmineman PASS couragelee PASS criccomini PASS davecom PASS davery22 PASS ddimtirov PASS deemkeen PASS ebarlas PASS entangled90 FAIL fatroom PASS felix19350 PASS filiphr PASS flippingbits PASS fragmede PASS gabrielreid PASS gamlerhart PASS gnmathur PASS hchiorean FAIL hundredwatt PASS imrafaelmerino PASS isolgpus PASS itaske PASS iziamos PASS japplis PASS jgrateron PASS jotschi PASS kevinmcmurtrie PASS kgeri PASS khmarbaise PASS kuduwa-keshavram PASS kumarsaurav123 FAIL lawrey PASS maeda6uiui PASS MeanderingProgrammer PASS merykitty FAIL moysesb PASS mudit-saxena PASS obourgain PASS padreati PASS palmr PASS pedestrianlove PASS phd3 PASS raipc PASS rby PASS richardstartin FAIL roman-r-m PASS royvanrijn PASS rprabhu PASS santanu PASS seijikun PASS semotpan PASS spullara PASS thomaswue PASS truelive FAIL twobiers PASS Ujjwalbharti PASS yavuztas PASS yehwankim23 FAIL yemreinci PASS zerninv ```
2024-01-11 13:40:21 +01:00
if [ -t 1 ]; then
GREEN='\033[0;32m'
RED='\033[0;31m'
RESET='\033[0m'
else
GREEN=""
RED=""
RESET=""
fi
WITH_TIMEOUT=""
if [ -x "$(command -v timeout)" ]; then
WITH_TIMEOUT="timeout -s KILL 5s"
elif [ -x "$(command -v gtimeout)" ]; then # MacOS from `brew install coreutils`
WITH_TIMEOUT="gtimeout -s KILL 5s"
else
echo "$0: timeout command not available, tests may run indefinitely long." 1>&2
fi
test: add script to test all implementations The script tests all implementations and prints PASS or FAIL status. In case of failure it also prints implementation output to stderr. This will be handy for adding new test samples. Show test statuses and omit failing output: ```sh $ ./test_all.sh 2>/dev/null PASS artsiomkorzun PASS baseline PASS bjhara PASS criccomini FAIL ddimtirov FAIL ebarlas PASS filiphr FAIL itaske PASS khmarbaise FAIL kuduwa-keshavram FAIL lawrey PASS padreati FAIL palmr PASS richardstartin FAIL royvanrijn FAIL seijikun PASS spullara PASS truelive ``` Show only passing implementations: ``` $ ./test_all.sh 2>/dev/null | grep PASS | cut -d' ' -f2 artsiomkorzun baseline bjhara criccomini filiphr khmarbaise padreati richardstartin spullara truelive ``` For #61
2024-01-04 20:36:20 +01:00
for impl in $(ls calculate_average_*.sh | sort); do
noext="${impl%%.sh}"
Update test*.sh to support input file pattern This is useful for testing fork(s) against subset of test samples
2024-01-10 17:01:33 +01:00
fork=${noext##calculate_average_}
test: add script to test all implementations The script tests all implementations and prints PASS or FAIL status. In case of failure it also prints implementation output to stderr. This will be handy for adding new test samples. Show test statuses and omit failing output: ```sh $ ./test_all.sh 2>/dev/null PASS artsiomkorzun PASS baseline PASS bjhara PASS criccomini FAIL ddimtirov FAIL ebarlas PASS filiphr FAIL itaske PASS khmarbaise FAIL kuduwa-keshavram FAIL lawrey PASS padreati FAIL palmr PASS richardstartin FAIL royvanrijn FAIL seijikun PASS spullara PASS truelive ``` Show only passing implementations: ``` $ ./test_all.sh 2>/dev/null | grep PASS | cut -d' ' -f2 artsiomkorzun baseline bjhara criccomini filiphr khmarbaise padreati richardstartin spullara truelive ``` For #61
2024-01-04 20:36:20 +01:00
test_all.sh: configure fork test timeout Example output: ``` $ ./test_all.sh 2>/dev/null PASS abfrmblr PASS AbstractKamen PASS ags313 PASS alesj PASS algirdasrascius PASS anandmattikopp PASS arjenvaneerde FAIL armandino TIME artpar PASS artsiomkorzun PASS asun PASS baseline PASS berry120 PASS bjhara PASS charlibot PASS coolmineman PASS couragelee PASS criccomini PASS davecom PASS davery22 PASS ddimtirov PASS deemkeen PASS ebarlas PASS entangled90 FAIL fatroom PASS felix19350 PASS filiphr PASS flippingbits PASS fragmede PASS gabrielreid PASS gamlerhart PASS gnmathur PASS hchiorean FAIL hundredwatt PASS imrafaelmerino PASS isolgpus PASS itaske PASS iziamos PASS japplis PASS jgrateron PASS jotschi PASS kevinmcmurtrie PASS kgeri PASS khmarbaise PASS kuduwa-keshavram PASS kumarsaurav123 FAIL lawrey PASS maeda6uiui PASS MeanderingProgrammer PASS merykitty FAIL moysesb PASS mudit-saxena PASS obourgain PASS padreati PASS palmr PASS pedestrianlove PASS phd3 PASS raipc PASS rby PASS richardstartin FAIL roman-r-m PASS royvanrijn PASS rprabhu PASS santanu PASS seijikun PASS semotpan PASS spullara PASS thomaswue PASS truelive FAIL twobiers PASS Ujjwalbharti PASS yavuztas PASS yehwankim23 FAIL yemreinci PASS zerninv ```
2024-01-11 13:40:21 +01:00
# ./test.sh calls ./prepare_$fork.sh e.g. to build native image
# which may take some time.
# Here we run it upfront, assuming that prepare result is cached
# to avoid timeout due to long preparation.
if [ -f "./prepare_$fork.sh" ]; then
if ! output=$("./prepare_$fork.sh" 2>&1); then
echo "$output" 1>&2
echo "FAIL $fork"
continue
fi
fi
if output=$($WITH_TIMEOUT ./test.sh "$fork" "$INPUT" 2>&1); then
echo -e "${GREEN}PASS${RESET} $fork"
elif [ $? -eq 137 ]; then
echo -e "${RED}TIME${RESET} $fork"
test: add script to test all implementations The script tests all implementations and prints PASS or FAIL status. In case of failure it also prints implementation output to stderr. This will be handy for adding new test samples. Show test statuses and omit failing output: ```sh $ ./test_all.sh 2>/dev/null PASS artsiomkorzun PASS baseline PASS bjhara PASS criccomini FAIL ddimtirov FAIL ebarlas PASS filiphr FAIL itaske PASS khmarbaise FAIL kuduwa-keshavram FAIL lawrey PASS padreati FAIL palmr PASS richardstartin FAIL royvanrijn FAIL seijikun PASS spullara PASS truelive ``` Show only passing implementations: ``` $ ./test_all.sh 2>/dev/null | grep PASS | cut -d' ' -f2 artsiomkorzun baseline bjhara criccomini filiphr khmarbaise padreati richardstartin spullara truelive ``` For #61
2024-01-04 20:36:20 +01:00
else
echo "$output" 1>&2
test_all.sh: configure fork test timeout Example output: ``` $ ./test_all.sh 2>/dev/null PASS abfrmblr PASS AbstractKamen PASS ags313 PASS alesj PASS algirdasrascius PASS anandmattikopp PASS arjenvaneerde FAIL armandino TIME artpar PASS artsiomkorzun PASS asun PASS baseline PASS berry120 PASS bjhara PASS charlibot PASS coolmineman PASS couragelee PASS criccomini PASS davecom PASS davery22 PASS ddimtirov PASS deemkeen PASS ebarlas PASS entangled90 FAIL fatroom PASS felix19350 PASS filiphr PASS flippingbits PASS fragmede PASS gabrielreid PASS gamlerhart PASS gnmathur PASS hchiorean FAIL hundredwatt PASS imrafaelmerino PASS isolgpus PASS itaske PASS iziamos PASS japplis PASS jgrateron PASS jotschi PASS kevinmcmurtrie PASS kgeri PASS khmarbaise PASS kuduwa-keshavram PASS kumarsaurav123 FAIL lawrey PASS maeda6uiui PASS MeanderingProgrammer PASS merykitty FAIL moysesb PASS mudit-saxena PASS obourgain PASS padreati PASS palmr PASS pedestrianlove PASS phd3 PASS raipc PASS rby PASS richardstartin FAIL roman-r-m PASS royvanrijn PASS rprabhu PASS santanu PASS seijikun PASS semotpan PASS spullara PASS thomaswue PASS truelive FAIL twobiers PASS Ujjwalbharti PASS yavuztas PASS yehwankim23 FAIL yemreinci PASS zerninv ```
2024-01-11 13:40:21 +01:00
echo -e "${RED}FAIL${RESET} $fork"
test: add script to test all implementations The script tests all implementations and prints PASS or FAIL status. In case of failure it also prints implementation output to stderr. This will be handy for adding new test samples. Show test statuses and omit failing output: ```sh $ ./test_all.sh 2>/dev/null PASS artsiomkorzun PASS baseline PASS bjhara PASS criccomini FAIL ddimtirov FAIL ebarlas PASS filiphr FAIL itaske PASS khmarbaise FAIL kuduwa-keshavram FAIL lawrey PASS padreati FAIL palmr PASS richardstartin FAIL royvanrijn FAIL seijikun PASS spullara PASS truelive ``` Show only passing implementations: ``` $ ./test_all.sh 2>/dev/null | grep PASS | cut -d' ' -f2 artsiomkorzun baseline bjhara criccomini filiphr khmarbaise padreati richardstartin spullara truelive ``` For #61
2024-01-04 20:36:20 +01:00
fi
done
Reference in New Issue Copy Permalink