- It avoids creating unnecessary Strings objects and handles with the station names with its djb2 hashes instead
- Initializes hashmaps with capacity and load factor
- Adds -XX:+AlwaysPreTouch
* final comit
changing using mappedbytebuffer
changes before using unsafe address
using unsafe
* using graalvm,correct unsafe mem implementation
---------
Co-authored-by: Karthikeyans <karthikeyan.sn@zohocorp.com>
* Version 3
* Use SWAR algorithm from netty for finding a symbol in a string
* Faster equals - store the remainder in a long field (- 0.5s)
* optimise parsing numbers - prep
* Keep tweaking parsing logic
* Rewrote number parsing
may be a tiby bit faster it at all
* Epsilon GC
* 1bc challenge, but one that will run using jdk 8 without unsafe and still do reasonably well.
* Better hashtable
* the fastest GC is no GC
* cleanups
* increased hash size
* removed Playground.java
* collision-handling allocation free hashmap
* formatting
on automatic closing of ByteBuffers.. previously, a straggler could hold
up closing the ByteBuffers.
Also
- Improve Tracing code
- Parametrize additional options to aid in tuning
Our previous PR was surprising; parallelizing munmap() call did not
yield anywhere near the performance gain I expected. Local machine had
10% gain while testing machine only showed 2% gain. I am still not clear
why it happened and the two best theories I have are
1) Variance due to stragglers (that this change addresses)
2) munmap() is either too fast or too slow relative to the other
instructions compared to our local machine. I don't know which. We'll
have to use adaptive tuning, but that's in a different change.
* Version 3
* trying to optimize memory access (-0.2s)
- use smaller segments confined to thread
- unload in parallel
* Only call MemorySegment.address() once (~200ms)
* Squashing a bunch of commits together.
Commit#2; Uplift of 7% using native byteorder from ByteBuffer.
Commit#1: Minor changes to formatting.
* Commit #4: Parallelize munmap() and reduce completion time further by
10%. As the jvm exits with exit(0) syscall, the kernel reclaims the
memory mappings via munmap() call. Prior to this change. all the unmap()
calls were happening right at the end as the JVM exited. This led to
serial execution of about 350ms out of 2500 ms right at the end after
each shard completed its work. We can parallelize it by exposing the
Cleaner from MappedByteBuffer and then ensure that it is truly parallel
execution of munmap() by using a non-blocking lock (SeqLock). The
optimal strategy for when each thread must call unmap() is an interesting math problem with an exact solution and this code roughly reflects it.
Commit #3: Tried out reading long at a time from bytebuffer and
checking for presence of ';'.. it was slower compared to just reading int().
Removed the code for reading longs; just retaining the
hasSemicolonByte(..) check code
Commit #2: Introduce processLineSlow() and processRangeSlow() for the
tial part.
Commit #1: Create a separate tail piece of work for the last few lines to be
processed separately from the main loop. This allows the main loop to
read past its allocated range (by a 'long' if we reserve atleast 8 bytes
for the tail piece of work.)
* Golang implementation
* Speed up by avoiding copying the lines
* Memory mapping
* Add script for testing
* Now passing most of the tests
* Refactor to composed method
* Now using integer math throughout
* Now using a state machine for parsing!
* Refactoring state names
* Enabling profiling
* Running in parallel!
* Fully parallel!
* Refactor
* Improve type safety of methods
* The rounding problem is due to difference between Javas and Gos printf implementation
* Converting my solution to Java
* Merging results
* Splitting the file in several buffers
* Made it parallel!
* Removed test file
* Removed go implementation
* Removed unused files
* Add header to .sh file
---------
Co-authored-by: Matteo Vaccari <mvaccari@thoughtworks.com>
* Modify baseline version to improve performance
- Consume and process stream in parallel with memory map buffers, parsing it directly
- Use int instead of float/double to store values
- Use Epsilon GC and graal
* Update src/main/java/dev/morling/onebrc/CalculateAverage_adriacabeza.java
* Update calculate_average_adriacabeza.sh
---------
Co-authored-by: Gunnar Morling <gunnar.morling@googlemail.com>
* - Read file in multiple threads if available: 17" -> 15" locally
- Changed String to BytesText with cache: 12" locally
* - Fixed bug
- BytesText to Text
- More checks when reading the file
* - Combining measurements should be thread safe
- More readability changes
