* feat(flippingbits): Improve parsing of station names
* chore(flippingbits): Remove obsolete import
* feat(flippingbits): Use custom hash map
* feat(flippingbits): Use UNSAFE
* fix(flippingbits): Support very small files
* chore(flippingbits): Few cleanups
* chore(flippingbits): Align names
* fix(flippingbits): Initialize hash with first byte
* fix(flippingbits): Fix initialization of hash value
* Update create_measurements.py
Added license header to the python script to avoid breaking the build.
* Update src/main/python/create_measurements.py
---------
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* added python script to build test data
* moved create_measurements.py to src/main/python and updated paths for file io
* Updated readme to include blurb about python script to generate measurements
* Linear probe for city indexing. Beats current leader spullara 2.2 vs 3.8 elapsed time.
* Straightforward impl using bytebuffers. Turns out memorysegments were slower than used mappedbytebuffers.
* A initial submit-worthy entry
Comparison to select entries (averaged over 3 runs)
* spullara 1.66s [5th on leaderboard currently]
* vemana (this submission) 1.65s
* artsiomkorzun 1.64s [4th on leaderboard currently]
Tests: PASS
Impl Class: dev.morling.onebrc.CalculateAverage_vemana
Machine specs
* 16 core Ryzen 7950X
* 128GB RAM
Description
* Decompose the full file into Shards of memory mapped files and process
each independently, outputting a TreeMap: City -> Statistics
* Compose the final answer by merging the individual TreeMap outputs
* Select 1 Thread per available processor as reported by the JVM
* Size to fit all datastructure in 0.5x L3 cache (4MB/core on the
evaluation machines)
* Use linear probing hash table, with identity of city name = byte[] and
hash code computed inline
* Avoid all allocation in the hot path and instead use method
parameters. So, instead of passing a single Object param called Point(x, y, z),
pass 3 parameters for each of its components. It is ugly, but this
challenge is so far from Java's idioms anyway
* G1GC seems to want to interfere; use ParallelGC instead (just a quick
and dirty hack)
Things tried that did not work
* MemorySegments are actually slower than MappedByteBuffers
* Trying to inline everything: not needed; the JIT compiler is pretty
good
* Playing with JIT compiler flags didn't yield clear wins. In
particular, was surprised that using a max level of 3 and reducing
compilation threshold did nothing.. when the jit logs print that none
of the methods reach level 4 and stay there for long
* Hand-coded implementation of Array.equals(..) using
readLong(..) & bitmask_based_on_length from a bytebuffer instead of byte by byte
* Further tuning to compile loop methods: timings are now consistenctly ahead of artsiomkorzun in 4th place.
There are methods on the data path that were being interpreted for far
too long. For example, the method that takes a byte range and simply
calls one method per line was taking a disproportionate amount of time.
Using `-XX:+AlwaysCompileLoopMethods` option improved completion time by 4%.
============= vemana ===============
[20:55:22] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh vemana;
done;
Using java version 21.0.1-graal in this shell.
real 0m1.581s
user 0m34.166s
sys 0m1.435s
Using java version 21.0.1-graal in this shell.
real 0m1.593s
user 0m34.629s
sys 0m1.470s
Using java version 21.0.1-graal in this shell.
real 0m1.632s
user 0m35.893s
sys 0m1.340s
Using java version 21.0.1-graal in this shell.
real 0m1.596s
user 0m33.074s
sys 0m1.386s
Using java version 21.0.1-graal in this shell.
real 0m1.611s
user 0m35.516s
sys 0m1.438s
============= artsiomkorzun ===============
[20:56:12] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh
artsiomkorzun; done;
Using java version 21.0.1-graal in this shell.
real 0m1.669s
user 0m38.043s
sys 0m1.287s
Using java version 21.0.1-graal in this shell.
real 0m1.679s
user 0m37.840s
sys 0m1.400s
Using java version 21.0.1-graal in this shell.
real 0m1.657s
user 0m37.607s
sys 0m1.298s
Using java version 21.0.1-graal in this shell.
real 0m1.643s
user 0m36.852s
sys 0m1.392s
Using java version 21.0.1-graal in this shell.
real 0m1.644s
user 0m36.951s
sys 0m1.279s
============= spullara ===============
[20:57:55] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh spullara;
done;
Using java version 21.0.1-graal in this shell.
real 0m1.676s
user 0m37.404s
sys 0m1.386s
Using java version 21.0.1-graal in this shell.
real 0m1.652s
user 0m36.509s
sys 0m1.486s
Using java version 21.0.1-graal in this shell.
real 0m1.665s
user 0m36.451s
sys 0m1.506s
Using java version 21.0.1-graal in this shell.
real 0m1.671s
user 0m36.917s
sys 0m1.371s
Using java version 21.0.1-graal in this shell.
real 0m1.634s
user 0m35.624s
sys 0m1.573s
========================== Running Tests ======================
[21:17:57] [lsv@vemana]$ ./runTests.sh vemana
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10000-unique-keys.txt
Using java version 21.0.1-graal in this shell.
real 0m0.150s
user 0m1.035s
sys 0m0.117s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10.txt
Using java version 21.0.1-graal in this shell.
real 0m0.114s
user 0m0.789s
sys 0m0.116s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-1.txt
Using java version 21.0.1-graal in this shell.
real 0m0.115s
user 0m0.948s
sys 0m0.075s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-20.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.926s
sys 0m0.066s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-2.txt
Using java version 21.0.1-graal in this shell.
real 0m0.110s
user 0m0.734s
sys 0m0.078s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-3.txt
Using java version 21.0.1-graal in this shell.
real 0m0.114s
user 0m0.870s
sys 0m0.095s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-boundaries.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.843s
sys 0m0.084s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-complex-utf8.txt
Using java version 21.0.1-graal in this shell.
real 0m0.121s
user 0m0.852s
sys 0m0.171s
* Improve by a few % more; now, convincingly faster than 6th place
submission. So far, only algorithms and tuning; no bitwise tricks yet.
Improve chunking implementation to avoid allocation and allow
finegrained chunking for the last X% of work. Work now proceeds in two
stages: big chunk stage and small chunk stage. This is to avoid
straggler threads holding up result merging.
Tests pass
[07:14:49] [lsv@vemana]$ ./test.sh vemana
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10000-unique-keys.txt
Using java version 21.0.1-graal in this shell.
real 0m0.152s
user 0m0.973s
sys 0m0.107s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.840s
sys 0m0.060s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-1.txt
Using java version 21.0.1-graal in this shell.
real 0m0.107s
user 0m0.681s
sys 0m0.085s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-20.txt
Using java version 21.0.1-graal in this shell.
real 0m0.105s
user 0m0.894s
sys 0m0.068s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-2.txt
Using java version 21.0.1-graal in this shell.
real 0m0.099s
user 0m0.895s
sys 0m0.068s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-3.txt
Using java version 21.0.1-graal in this shell.
real 0m0.098s
user 0m0.813s
sys 0m0.050s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-boundaries.txt
Using java version 21.0.1-graal in this shell.
real 0m0.095s
user 0m0.777s
sys 0m0.087s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-complex-utf8.txt
Using java version 21.0.1-graal in this shell.
real 0m0.112s
user 0m0.904s
sys 0m0.069s
* Merge results from finished threads instead of waiting for all threads
to finish.
Not a huge difference overall but no reason to wait.
Also experiment with a few other compiler flags and attempt to use
jitwatch to understand what the jit is doing.
* Move to prepare_*.sh format and run evaluate2.sh locally.
Shows 7th place in leaderboard
| # | Result (m:s.ms) | Implementation | JDK | Submitter | Notes
|
|---|-----------------|--------------------|-----|---------------|-----------|
| 1 | 00:01.588 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_thomaswue.java)|
21.0.1-graal | [Thomas Wuerthinger](https://github.com/thomaswue) |
GraalVM native binary |
| 2 | 00:01.866 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_merykitty.java)|
21.0.1-open | [Quan Anh Mai](https://github.com/merykitty) | |
| 3 | 00:01.904 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_royvanrijn.java)|
21.0.1-graal | [Roy van Rijn](https://github.com/royvanrijn) | |
| | 00:02.398 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_ebarlas.java)|
21.0.1-graal | [Elliot Barlas](https://github.com/ebarlas) | |
| | 00:02.724 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_obourgain.java)|
21.0.1-open | [Olivier Bourgain](https://github.com/obourgain) | |
| | 00:02.771 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_algirdasrascius.java)|
21.0.1-open | [Algirdas Ra__ius](https://github.com/algirdasrascius) |
|
| | 00:02.842 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_vemana.java)|
21.0.1-graal | [Vemana](https://github.com/vemana) | |
| | 00:02.902 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_spullara.java)|
21.0.1-graal | [Sam Pullara](https://github.com/spullara) | |
| | 00:02.906 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_artsiomkorzun.java)|
21.0.1-graal | [artsiomkorzun](https://github.com/artsiomkorzun) | |
| | 00:02.970 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_isolgpus.java)|
21.0.1-open | [Jamie Stansfield](https://github.com/isolgpus) | |
* Tune chunksize to get another 2% improvement for 8 processors as used by
the evaluation script.
* Read int at a time for city name length detection; speeds up by 2% in local testing.
* Improve reading temperature double by exiting loop quicker; no major
tricks (like reading an int) yet, but good for 5th place on leaderboard in local testing.
This small change has caused a surprising gain in performance by about 4%.
I didn't expect such a big change, but perhaps in combination with the
earlier change to read int by int for the city name, temperature reading
is dominating that aspect of the time. Also, perhaps the quicker exit
(as soon as you see '.' instead of reading until '\n') means you get to
simply skip reading the '\n' across each of the lines. Since the lines
are on average like 15 characters, it may be that avoiding reading the \n
is a meaningful saving. Or maybe the JIT found a clever optimization for
reading the temperature.
Or maybe it is simply the case that the number of multiplications is now
down to 2 from the previous 3 is what's causing the performance gain?
| # | Result (m:s.ms) | Implementation | JDK | Submitter | Notes
|
|---|-----------------|--------------------|-----|---------------|-----------|
| 1 | 00:01.531 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_thomaswue.java)|
21.0.1-graal | [Thomas Wuerthinger](https://github.com/thomaswue) |
GraalVM native binary |
| 2 | 00:01.794 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_royvanrijn.java)|
21.0.1-graal | [Roy van Rijn](https://github.com/royvanrijn) | |
| 3 | 00:01.956 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_merykitty.java)|
21.0.1-open | [Quan Anh Mai](https://github.com/merykitty) | |
| | 00:02.346 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_ebarlas.java)|
21.0.1-graal | [Elliot Barlas](https://github.com/ebarlas) | |
| | 00:02.673 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_vemana.java)|
21.0.1-graal | [Subrahmanyam](https://github.com/vemana) | |
| | 00:02.689 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_obourgain.java)|
21.0.1-open | [Olivier Bourgain](https://github.com/obourgain) | |
| | 00:02.785 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_algirdasrascius.java)|
21.0.1-open | [Algirdas Ra__ius](https://github.com/algirdasrascius) |
|
| | 00:02.926 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_isolgpus.java)|
21.0.1-open | [Jamie Stansfield](https://github.com/isolgpus) | |
| | 00:02.928 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_artsiomkorzun.java)|
21.0.1-graal | [Artsiom Korzun](https://github.com/artsiomkorzun) | |
| | 00:02.932 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_spullara.java)|
21.0.1-graal | [Sam Pullara](https://github.com/spullara) | |
* Reduce one multiplication when temperature is +ve.
* Linear probe for city indexing. Beats current leader spullara 2.2 vs 3.8 elapsed time.
* Straightforward impl using bytebuffers. Turns out memorysegments were slower than used mappedbytebuffers.
* A initial submit-worthy entry
Comparison to select entries (averaged over 3 runs)
* spullara 1.66s [5th on leaderboard currently]
* vemana (this submission) 1.65s
* artsiomkorzun 1.64s [4th on leaderboard currently]
Tests: PASS
Impl Class: dev.morling.onebrc.CalculateAverage_vemana
Machine specs
* 16 core Ryzen 7950X
* 128GB RAM
Description
* Decompose the full file into Shards of memory mapped files and process
each independently, outputting a TreeMap: City -> Statistics
* Compose the final answer by merging the individual TreeMap outputs
* Select 1 Thread per available processor as reported by the JVM
* Size to fit all datastructure in 0.5x L3 cache (4MB/core on the
evaluation machines)
* Use linear probing hash table, with identity of city name = byte[] and
hash code computed inline
* Avoid all allocation in the hot path and instead use method
parameters. So, instead of passing a single Object param called Point(x, y, z),
pass 3 parameters for each of its components. It is ugly, but this
challenge is so far from Java's idioms anyway
* G1GC seems to want to interfere; use ParallelGC instead (just a quick
and dirty hack)
Things tried that did not work
* MemorySegments are actually slower than MappedByteBuffers
* Trying to inline everything: not needed; the JIT compiler is pretty
good
* Playing with JIT compiler flags didn't yield clear wins. In
particular, was surprised that using a max level of 3 and reducing
compilation threshold did nothing.. when the jit logs print that none
of the methods reach level 4 and stay there for long
* Hand-coded implementation of Array.equals(..) using
readLong(..) & bitmask_based_on_length from a bytebuffer instead of byte by byte
* Further tuning to compile loop methods: timings are now consistenctly ahead of artsiomkorzun in 4th place.
There are methods on the data path that were being interpreted for far
too long. For example, the method that takes a byte range and simply
calls one method per line was taking a disproportionate amount of time.
Using `-XX:+AlwaysCompileLoopMethods` option improved completion time by 4%.
============= vemana ===============
[20:55:22] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh vemana;
done;
Using java version 21.0.1-graal in this shell.
real 0m1.581s
user 0m34.166s
sys 0m1.435s
Using java version 21.0.1-graal in this shell.
real 0m1.593s
user 0m34.629s
sys 0m1.470s
Using java version 21.0.1-graal in this shell.
real 0m1.632s
user 0m35.893s
sys 0m1.340s
Using java version 21.0.1-graal in this shell.
real 0m1.596s
user 0m33.074s
sys 0m1.386s
Using java version 21.0.1-graal in this shell.
real 0m1.611s
user 0m35.516s
sys 0m1.438s
============= artsiomkorzun ===============
[20:56:12] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh
artsiomkorzun; done;
Using java version 21.0.1-graal in this shell.
real 0m1.669s
user 0m38.043s
sys 0m1.287s
Using java version 21.0.1-graal in this shell.
real 0m1.679s
user 0m37.840s
sys 0m1.400s
Using java version 21.0.1-graal in this shell.
real 0m1.657s
user 0m37.607s
sys 0m1.298s
Using java version 21.0.1-graal in this shell.
real 0m1.643s
user 0m36.852s
sys 0m1.392s
Using java version 21.0.1-graal in this shell.
real 0m1.644s
user 0m36.951s
sys 0m1.279s
============= spullara ===============
[20:57:55] [lsv@vemana]$ for i in 1 2 3 4 5; do ./runTheir.sh spullara;
done;
Using java version 21.0.1-graal in this shell.
real 0m1.676s
user 0m37.404s
sys 0m1.386s
Using java version 21.0.1-graal in this shell.
real 0m1.652s
user 0m36.509s
sys 0m1.486s
Using java version 21.0.1-graal in this shell.
real 0m1.665s
user 0m36.451s
sys 0m1.506s
Using java version 21.0.1-graal in this shell.
real 0m1.671s
user 0m36.917s
sys 0m1.371s
Using java version 21.0.1-graal in this shell.
real 0m1.634s
user 0m35.624s
sys 0m1.573s
========================== Running Tests ======================
[21:17:57] [lsv@vemana]$ ./runTests.sh vemana
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10000-unique-keys.txt
Using java version 21.0.1-graal in this shell.
real 0m0.150s
user 0m1.035s
sys 0m0.117s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10.txt
Using java version 21.0.1-graal in this shell.
real 0m0.114s
user 0m0.789s
sys 0m0.116s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-1.txt
Using java version 21.0.1-graal in this shell.
real 0m0.115s
user 0m0.948s
sys 0m0.075s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-20.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.926s
sys 0m0.066s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-2.txt
Using java version 21.0.1-graal in this shell.
real 0m0.110s
user 0m0.734s
sys 0m0.078s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-3.txt
Using java version 21.0.1-graal in this shell.
real 0m0.114s
user 0m0.870s
sys 0m0.095s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-boundaries.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.843s
sys 0m0.084s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-complex-utf8.txt
Using java version 21.0.1-graal in this shell.
real 0m0.121s
user 0m0.852s
sys 0m0.171s
* Improve by a few % more; now, convincingly faster than 6th place
submission. So far, only algorithms and tuning; no bitwise tricks yet.
Improve chunking implementation to avoid allocation and allow
finegrained chunking for the last X% of work. Work now proceeds in two
stages: big chunk stage and small chunk stage. This is to avoid
straggler threads holding up result merging.
Tests pass
[07:14:49] [lsv@vemana]$ ./test.sh vemana
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10000-unique-keys.txt
Using java version 21.0.1-graal in this shell.
real 0m0.152s
user 0m0.973s
sys 0m0.107s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-10.txt
Using java version 21.0.1-graal in this shell.
real 0m0.113s
user 0m0.840s
sys 0m0.060s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-1.txt
Using java version 21.0.1-graal in this shell.
real 0m0.107s
user 0m0.681s
sys 0m0.085s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-20.txt
Using java version 21.0.1-graal in this shell.
real 0m0.105s
user 0m0.894s
sys 0m0.068s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-2.txt
Using java version 21.0.1-graal in this shell.
real 0m0.099s
user 0m0.895s
sys 0m0.068s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-3.txt
Using java version 21.0.1-graal in this shell.
real 0m0.098s
user 0m0.813s
sys 0m0.050s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-boundaries.txt
Using java version 21.0.1-graal in this shell.
real 0m0.095s
user 0m0.777s
sys 0m0.087s
Validating calculate_average_vemana.sh --
src/test/resources/samples/measurements-complex-utf8.txt
Using java version 21.0.1-graal in this shell.
real 0m0.112s
user 0m0.904s
sys 0m0.069s
* Merge results from finished threads instead of waiting for all threads
to finish.
Not a huge difference overall but no reason to wait.
Also experiment with a few other compiler flags and attempt to use
jitwatch to understand what the jit is doing.
* Move to prepare_*.sh format and run evaluate2.sh locally.
Shows 7th place in leaderboard
| # | Result (m:s.ms) | Implementation | JDK | Submitter | Notes
|
|---|-----------------|--------------------|-----|---------------|-----------|
| 1 | 00:01.588 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_thomaswue.java)|
21.0.1-graal | [Thomas Wuerthinger](https://github.com/thomaswue) |
GraalVM native binary |
| 2 | 00:01.866 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_merykitty.java)|
21.0.1-open | [Quan Anh Mai](https://github.com/merykitty) | |
| 3 | 00:01.904 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_royvanrijn.java)|
21.0.1-graal | [Roy van Rijn](https://github.com/royvanrijn) | |
| | 00:02.398 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_ebarlas.java)|
21.0.1-graal | [Elliot Barlas](https://github.com/ebarlas) | |
| | 00:02.724 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_obourgain.java)|
21.0.1-open | [Olivier Bourgain](https://github.com/obourgain) | |
| | 00:02.771 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_algirdasrascius.java)|
21.0.1-open | [Algirdas Ra__ius](https://github.com/algirdasrascius) |
|
| | 00:02.842 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_vemana.java)|
21.0.1-graal | [Vemana](https://github.com/vemana) | |
| | 00:02.902 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_spullara.java)|
21.0.1-graal | [Sam Pullara](https://github.com/spullara) | |
| | 00:02.906 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_artsiomkorzun.java)|
21.0.1-graal | [artsiomkorzun](https://github.com/artsiomkorzun) | |
| | 00:02.970 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_isolgpus.java)|
21.0.1-open | [Jamie Stansfield](https://github.com/isolgpus) | |
* Tune chunksize to get another 2% improvement for 8 processors as used by
the evaluation script.
* Read int at a time for city name length detection; speeds up by 2% in local testing.
* Improve reading temperature double by exiting loop quicker; no major
tricks (like reading an int) yet, but good for 5th place on leaderboard in local testing.
This small change has caused a surprising gain in performance by about 4%.
I didn't expect such a big change, but perhaps in combination with the
earlier change to read int by int for the city name, temperature reading
is dominating that aspect of the time. Also, perhaps the quicker exit
(as soon as you see '.' instead of reading until '\n') means you get to
simply skip reading the '\n' across each of the lines. Since the lines
are on average like 15 characters, it may be that avoiding reading the \n
is a meaningful saving. Or maybe the JIT found a clever optimization for
reading the temperature.
Or maybe it is simply the case that the number of multiplications is now
down to 2 from the previous 3 is what's causing the performance gain?
| # | Result (m:s.ms) | Implementation | JDK | Submitter | Notes
|
|---|-----------------|--------------------|-----|---------------|-----------|
| 1 | 00:01.531 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_thomaswue.java)|
21.0.1-graal | [Thomas Wuerthinger](https://github.com/thomaswue) |
GraalVM native binary |
| 2 | 00:01.794 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_royvanrijn.java)|
21.0.1-graal | [Roy van Rijn](https://github.com/royvanrijn) | |
| 3 | 00:01.956 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_merykitty.java)|
21.0.1-open | [Quan Anh Mai](https://github.com/merykitty) | |
| | 00:02.346 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_ebarlas.java)|
21.0.1-graal | [Elliot Barlas](https://github.com/ebarlas) | |
| | 00:02.673 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_vemana.java)|
21.0.1-graal | [Subrahmanyam](https://github.com/vemana) | |
| | 00:02.689 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_obourgain.java)|
21.0.1-open | [Olivier Bourgain](https://github.com/obourgain) | |
| | 00:02.785 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_algirdasrascius.java)|
21.0.1-open | [Algirdas Ra__ius](https://github.com/algirdasrascius) |
|
| | 00:02.926 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_isolgpus.java)|
21.0.1-open | [Jamie Stansfield](https://github.com/isolgpus) | |
| | 00:02.928 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_artsiomkorzun.java)|
21.0.1-graal | [Artsiom Korzun](https://github.com/artsiomkorzun) | |
| | 00:02.932 |
[link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_spullara.java)|
21.0.1-graal | [Sam Pullara](https://github.com/spullara) | |
* Reduce one multiplication when temperature is +ve.
* Added some documentation on the approach.
---------
Co-authored-by: vemana <vemana.github@gmail.com>
* first version
* second version (0m59s)
* third version (0m46s)
* fourth version (0m39s)
* fifth version (0m18s)
* follow naming conventions from project structure
* fix rounding (see /issues/49)
* formatting changes from build
* name should case-match github username
* sixth version (14s)
* seventh version (11s)
* potential fix for other systems?
* no need for sdk install
* binary should go to ./target
* building native-image only when not existing yet
* Calculate average by vaidhy
* Calculate average by vaidhy
* More changes
* remove worker log
* Pass -Dparellelism and switch back to open
* Try out mmap
* Improve mmap solution
* no copy version
* reduce threads
* hash code computed on the fly
* Reuse the char (Do not know if it helps)
* primitive hash map
* Primite HashMap
* Micro optimizations to push for optimizations
* Revert "Micro optimizations to push for optimizations"
This reverts commit ea333e2821ebb5c1d6d71a4e87e569a8f2f8f7f0.
* Micro optimizations to get the juice
* floorMod fixes
* findSemi and findNewLine as separate functions
* Optimized parseDouble
* More micro changes
* Aligned equal check
* more small changes
* XOR instead of compare
* Reduce loop length
* Revert changes
* Loop optimization and added native build
* Hand unrolled findSemi loop.
* Remove incorrect comments
* Taking care fo PR comments
* Add prepare script
* Missing header error fix
* remove wrong comment
---------
Co-authored-by: Anita S V <anitasvasu@gmail.com>
Co-authored-by: Anita SV <anitvasu@amazon.com>
* Use graal
* Use dynamic cores computer
* Use stream API to cleanup code
* Use max processors
* Use hash to avoid init string
* optimize concurrentmap init
* Smaller hash size
* Avoid checking concurrentmap
* Optimize data type
* string dedup
* Faster write
* Change base
* Remove time
* Use mul instead of div
* refactor: replace xtrace with "print_and_execute" function
* nit: stylize error messages
* replace out_expected.txt with measurements_1B.out
* print
* prevent errors on cleanup
* run tests and check warmup run output before running benchmark
* move "git diff" pretty diff output to test.sh
* Ensure "set -e" is re-enabled if we followed a "continue" branch
* add timeouts to test.sh invocations
* use diff with tocsv.sh to show differences on failed test
* add --quiet mode to test.sh
* move prepare_$fork.sh invocation to right below hyperfine since test.sh also invokes it
* Revert "add --quiet mode to test.sh"
This reverts commit 13e9fb7f395c1bd64a62528b8349803bc1366941.
* use tee to capture test output to a temp file and print contents on failure
---------
Co-authored-by: Jason Nochlin <hundredwatt@users.noreply.github.com>
* remove unneeded check
* slightly improved hash code perf
* Use unsafe to access memory + untangle the code a bit
* Adhoc cache that works a bit better
* Store station names as offset into the memory segment + length; slightly change how the hash is calculated
