Skip to main content

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

6
  • Are you sure the CPU would be the bottleneck? On my 12 year old 4 core 8 thread laptop with a fast SSD, time printf '%s\0' /var/lib/apt/lists/*lz4 | xargs -r0 -n 1 -P8 lz4 -dc (so even without the GNU parallel overhead) is not faster than lz4 -dmc /var/lib/apt/lists/*lz4 for me. Commented Nov 3, 2023 at 19:04
  • Did you consider the --tmpdir and/or --compress options? Commented Nov 3, 2023 at 19:36
  • @xhienne, the --tmpdir and --compress options imply using temporary files, which is what I asked to prevent in this question. Commented Nov 3, 2023 at 19:39
  • @StéphaneChazelas, added a comparision of raw serial and parallel decompression performance to the question. Commented Nov 3, 2023 at 19:45
  • 1
    @xhienne, sorry, but I was asking this question in order to maximize throughput. The general solution to this objective is to use streaming and prevent from using file I/O. Of course, the above is an artificial test case while the real task was more complicated. Commented Nov 3, 2023 at 19:52