fdd92dd5f7
Make it more like libraries.rs, tried reading whole file into vec, but it's very slow, maybe slice is better
main
Fabian Schmidt2024-12-31 12:58:04 +01:00
45b3014cbb
Store station name as byte slice and only convert to string once at the end, but using u64 hash as key is still faster
Fabian Schmidt2024-12-31 11:24:31 +01:00
98cd6e930c
Make faster by splitting borrows of byte slice
Fabian Schmidt2024-12-31 11:13:34 +01:00
7b8943976f
Revert last commit Accidentally kept the hashing and when I removed it the solution slowed down again
Fabian Schmidt2024-08-28 14:16:38 +02:00
aaa11c7b94
Worse than fx (despite me hashing the station names there too) better than vanilla
Fabian Schmidt2024-08-28 14:07:16 +02:00
07a8e7fc69
add solution I found which helped me get faster, unfortunately the solution itself is false (?)
Fabian Schmidt2024-08-28 13:09:47 +02:00
608cbb59e5
managed to get faster again by searching hashmap using &str and only converting to String on insertion. Removed FxHashMap again
Fabian Schmidt2024-08-27 10:57:23 +02:00
d246c54cd9
removed polars example because it wouldn't compile anymore, even when updating to latest version. It also massively reduced the number of downloaded crates
Fabian Schmidt2024-08-19 13:55:19 +02:00
7add8793a5
managed to get library version on par with the reference implementation by using memmap2. I'm understanding scoped threads a little better now... I think The fixed line length solution is still just as slow, even using memmap
Fabian Schmidt2024-08-19 10:39:19 +02:00
b1d7ebaaea
Create multi threaded version using smol for async reading of the file ~1 seconds faster
Fabian Schmidt2024-08-13 14:14:42 +02:00
bbc89aa2b3
Create single threaded version using smol for async reading of the file 46 -> 40 seconds
Fabian Schmidt2024-08-13 13:50:49 +02:00
b53212103b
Tried looking at what the performance would be if I read the data from a file where every line would have the same length ie. not having to read until eol. But despite not having to search the \n byte (or ; because every station name and temperatures are padded with null bytes) and having a fixed size array instead of vec this is slower (the normal read_until version is actually still just as fast, while the new one is 10x slower)
Fabian Schmidt2024-08-12 10:48:07 +02:00
8ffea918c4
either it's slightly faster or it's a measurement error
Fabian Schmidt2024-08-05 12:52:57 +02:00
0ffbff4cbf
added 2 new implementations from users who commented on my reddit post as a comparison
Fabian Schmidt2024-07-31 13:58:42 +02:00
25d20169aa
some improvements by reducing the casts, implemented a get_pos function to get position of byte in byteslice by bitmasking... slightly slower
Fabian Schmidt2024-07-31 12:49:16 +02:00
5aa94e67d1
use read_until method instead of custom function because it is faster than mine using the Bytes struct
Fabian Schmidt2024-07-25 15:35:10 +02:00
e230a5ce2c
My multi-treaded version is now faster than polars and takes less time to compile. It's a little more complex though
Fabian Schmidt2024-07-25 10:03:37 +02:00
dcaca0cc65
managed to make my solution super fast... but it's also incorrect
Fabian Schmidt2024-07-24 15:19:17 +02:00
16cf4ca2ca
moved from reading String to reading bytes. A little faster, still need to implement for multithreaded solution
Fabian Schmidt2024-07-23 16:30:50 +02:00
91adbf4c1c
Actually I just needed to add target-cpu=native to [build] in Cargo.toml, did that and create HashMap with capacity, also added reference implementation (which uses libraries unlike my solution)
Fabian Schmidt2024-07-23 13:23:26 +02:00
00096647ee
The polars solution I saw on reddit a few months ago is also super slow. I'm thinking it might be my macbook
Fabian Schmidt2024-07-23 13:23:26 +02:00
fe1053b74a
Multi threaded works now but it's slower than single threaded...
Fabian Schmidt2024-07-23 13:23:26 +02:00
3ffed9099c
Still broken but it compiles. For some reason lines are not properly being read and only one thread is being run
Fabian Schmidt2024-07-23 13:23:26 +02:00
65df621cf6
use hashbrown instead of std hashmap, no performance improvements
Fabian Schmidt2024-07-23 13:23:26 +02:00
Fabian Schmidt
Antonio Goncalves
Ruslan Kovtun
Daniel Patrick
Gunnar Morling
Eugene Huang
Martin
Smoofie
Chris Bellew
JurenIvan
yourwass
Anita SV
Diego Parra
Panagiotis Drakatos
gonix
tivrfoa