Question

I implemented a small function bruteforce, using lazy evaluation to find the first valid solution for a problem:

import Data.Maybe

bruteforce :: (a -> Bool) -> [a] -> Maybe a
bruteforce f xs
  | null result = Nothing
  | otherwise = Just $ head result
  where
    result = mapMaybe bruteforce' xs
    -- test one instance
    bruteforce' x
      | f x = Just x
      | otherwise = Nothing

generatorString :: Int -> [String]
generatorString 0 = [""]
generatorString deep = concatMap (\x -> map (\ys -> (x:ys)) nextgen) ['a'..'z']
  where nextgen = generatorString (deep - 1)


main :: IO ()
main = do
  putStrLn $ fromJust $ bruteforce ((==) "zabcde") (generatorString 6)

also avaiable as a gist. yes, the test function is stupid, but it’s enough to show what’s the problem. It works as expected, however memory consumption is quite high:

$ ghc --make -O2 brute.hs -o brute -rtsopts
$ ./brute +RTS -s
zabcde
  24,752,866,120 bytes allocated in the heap
  15,748,430,224 bytes copied during GC
     581,741,352 bytes maximum residency (22 sample(s))
      18,464,056 bytes maximum slop
            1719 MB total memory in use (0 MB lost due to fragmentation)
[...]

so I tried to force the RTS using less memory (i.e. call the GC more often). 200 MB should be enough?

$ ./brute +RTS -s -M200M
Heap exhausted;
Current maximum heap size is 209715200 bytes (200 MB);
use `+RTS -M<size>' to increase it.

well, nope (I wouldn’t like this approach anyway…)

Any pointers how one could rewrite bruteforce to be more memory friendly?

Answer 1

If the memory consumption is the primary concern, stop sharing nextgen. That’s a huge list.

So replace

generatorString deep = concatMap (\x -> map (\ys -> (x:ys)) nextgen) ['a'..'z']
  where nextgen = generatorString (deep - 1)

with

generatorString deep = concatMap (\x -> map (\ys -> (x:ys)) $ generatorString (deep - 1)) ['a'..'z']

and tell the compiler that you’re serious about not sharing it:

$ ghc -O2 -fno-full-laziness -rtsopts bruteforce

That runs in

 $ ./bruteforce +RTS -s
 zabcde
 189,448,835,904 bytes allocated in the heap
  18,301,350,520 bytes copied during GC
          29,504 bytes maximum residency (16901 sample(s))
          37,248 bytes maximum slop
               2 MB total memory in use (0 MB lost due to fragmentation)

small resident memory. Of course the recomputation means the total allocation is much higher, and it also takes much longer to compute the result.

A better algorithm could reduce space and time consumption.