Timeline for Need something that is faster than "wc -l"
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 22, 2019 at 14:54 | comment | added | Petr Skocik | @TobySpeight Yeah, multithreading might speed it up. Also looking scanning two bytes at a time via a 2^16 look up tables provided a pretty good speed up last time I played with it. | |
| Mar 2, 2016 at 17:59 | history | edited | Faheem Mitha | CC BY-SA 3.0 | deleted 1 character in body |
| Mar 2, 2016 at 17:34 | comment | added | Barmar | The read() version may benefit from read-ahead. | |
| Mar 2, 2016 at 12:52 | comment | added | Toby Speight | There may be some benefit to reading different parts of the file in separate threads (e.g. with an OpenMP for loop) so that some progress might be made while one thread is stalled waiting for input. But on the other hand, it might hamper I/O scheduling, so all I can recommend is to try it, and measure! | |
| Mar 2, 2016 at 5:08 | history | edited | muru | CC BY-SA 3.0 | added 48 characters in body |
| Mar 2, 2016 at 1:12 | comment | added | jthill | mmap is going to get vastly better results on linux because it'll map to huge pages these days, and TLB misses are sloooowwwwwww. | |
| Mar 2, 2016 at 0:27 | comment | added | Petr Skocik | I was surprised at the mmap result TBH. I used to think mmaping was faster than read/write, but then I saw some linux benchmarks that showed the opposite. Looks like it is very true in this case. | |
| Mar 2, 2016 at 0:19 | history | edited | Petr Skocik | CC BY-SA 3.0 | added 84 characters in body |
| Mar 2, 2016 at 0:11 | history | edited | Petr Skocik | CC BY-SA 3.0 | added 880 characters in body |
| Mar 1, 2016 at 22:53 | history | edited | Petr Skocik | CC BY-SA 3.0 | added 141 characters in body |
| Mar 1, 2016 at 22:46 | history | answered | Petr Skocik | CC BY-SA 3.0 |