Home/ Questions/Q 9040615
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-16T10:02:04+00:00

i’m using Apache Commons Compress for Java to compress multiple log files to a

  • 0

i’m using Apache Commons Compress for Java to compress multiple log files to a single tar.bz2 archive.

However, it takes really long (> 12 hours) to compress, because i compress around 20GB of files a day.

As this library compresses files mono-threaded, i’d like to know if there is a way to do this multi-threaded.

I found many solutions (Commandline pbzip2 or some C++ libraries) but all i found for java is this blog post:

https://plus.google.com/117421466255362255970/posts/3jfKVu325zh

It seems that i can’t use it in my Java application.

Is there anything out there? What would you recommend? Or is there another faster solution with similar compression rates like bzip2 ?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    As you have multiple files, you can compress each file in a different thread. As your process is CPU bound, I suggest creating a fixed size thread pool i.e. an ExecutorService, and adding a task for each file to compress.

    Note: if pbzip2 does what you want, I would call it from Java. You might find it is fast for even one thread as the BZIP2 libraries I have seen for Java are natively implemented (unlike JAR, ZIP and GZIP)

    • 0