Question

My application, which turns given data into a tree representation, is using way too much memory. As it manages to turn around 200-300MB of memory into roughly 3GB before crashing.

I now want to figure out where the leak is, which part of the program causes that.

Therefore I do now wonder what the most common and efficient technique for memory-profiling in common-lisp, using sbcl is?

I already looked at (room) and (time) but its output is way to verbose, all I need is a wrapper which will say “After the execution the overall memory usage was +1000Byte”, this would do the deal as I just want to know where the memory is used. Another criteria is that it has to work “on the fly” as the application will most likely crash, due to no remaining RAM.

Something looking like this:

(dotimes (i 4)
  (profiler-wrapper :messg "After execution memory ~a~%"  (execute-me i) ))

After execution memory +100Mb
After execution memory +100Mb
After execution memory +100Mb
After execution memory +100Mb
NIL

Answer 1

I wrote my own version of such an macro, it is most likely not elegant and it does slow the code significantly due to (sb-ext:gc :full t) but it does give some perspective of the memory usage after the execution of a given body.

(defparameter *last-profile-step* 0)

(defmacro profile-it ((name gc-on) &body body)
    `(let ((*last-profile-step* ,(if gc-on
                     `(progn
                    (sb-ext:gc :full t)
                    (sb-kernel::dynamic-usage))
                     `(sb-kernel::dynamic-usage))))
       (unwind-protect 
        (progn
          ,@body)
     (progn
       (FORMAT t "After execution of ~a : ~a byte~%" ,name
           (- ,(if gc-on
               `(progn
                  (sb-ext:gc :full t)
                  (sb-kernel::dynamic-usage))
               `(sb-kernel::dynamic-usage))
              *last-profile-step* ))))))