Question

Haskell’s Network.Browser module seems to not do any compression. How can I configure it so that it does gzip compression assuming that the server supports it (or fall back to no compression if it doesn’t) ?

Answer 1

Here’s a quick version of the “rather easy” solution rkhayrov refers to:

import Codec.Compression.GZip (decompress)
import Control.Arrow (second)
import Control.Monad (liftM)
import qualified Data.ByteString.Lazy as B
import Network.Browser
import Network.HTTP (Request, Response, getRequest, getResponseBody, rspBody)
import Network.HTTP.Headers (HasHeaders, HeaderName (..), findHeader, replaceHeader)
import Network.TCP (HStream, HandleStream)
import Network.URI (URI, parseURI)

gzipRequest :: URI -> BrowserAction (HandleStream B.ByteString) (URI, Response B.ByteString)
gzipRequest
  = liftM (second unzipIfNeeded)
  . request
  . replaceHeader HdrAcceptEncoding "gzip"
  . defaultGETRequest_
  where
    unzipIfNeeded rsp
      | isGz rsp  = rsp { rspBody = decompress $ rspBody rsp }
      | otherwise = rsp
      where
        isGz rsp = maybe False (== "gzip") $ findHeader HdrContentEncoding rsp

I ran a couple of tests with the following:

main = print =<< rspBody . snd <$> (getResponse =<< head <$> getArgs)
  where
    getResponse = browse . gzipRequest . fromJust . parseURI

It works as expected on both the Yahoo (compressed) and Google (uncompressed) home pages.