Question

Is there a text compression library for Erlang? When working with very long strings, it may be advantageous to compress the character data. Has anyone compressed text or thought of a way to do it in Erlang?

I was thinking of using the zip module, but instead of working with files, I work in-memory like this:

compress(LargeText)->
    Binary = list_to_binary(LargeText),
    {ok,{_,Zipped}} = zip:zip("ZipName",[{"Name",Binary}],[memory]),
    Zipped.

Then I would unzip the text back into memory when I need it. Like this:

{ok,[{"Name",Binary}]} = zip:unzip(Zipped,[memory]).

My Erlang application is supposed to be part of a middle tier in which large text may have to pass through to, and out of a storage system. The storage system is intended on storing large text. To optimize the storage, there is need to compress it before sending it. Assume that the text value is like a CLOB data type in Oracle Database. I am thinking that if I combine the zipping and the erlang:garbage_collect/0, i can pull it off.

Or if it’s not possible in Erlang, perhaps it is possible using a system call via os:cmd({Some UNIX script}) and then I would grab the output in Erlang? If there’s a better way, please show it.

Answer 1

There is a zlib module for Erlang, which supports in-memory compression and decompression.