Home/ Questions/Q 8610375
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-12T04:06:54+00:00

I have a program, written in C, it uses Berkeley-Db, glib-2, and libxml-2. It

  • 0

I have a program, written in C, it uses Berkeley-Db, glib-2, and libxml-2. It is a search indexer which makes NLP with syntactic and semantic analysis, and stores conceptual structures in search indices instead of words.

However, I have a problem — after a day or so, my program runs out of memory.

When I try to run my program under valgrind, it works very slow, it seems to me, that 10 times slower than with normal run.

All my attempts to check for memory-leaks (made for weeks under valgrind or so) didn’t show any leaks.

Does anybody know some tools that can help me in this situation?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    my solution would be:

    #define malloc(size) mymalloc(size, __FILE__, __FUNCTION__, __LINE__)
... (realloc, calloc, free)

void *mymalloc(size_t size, char *file, char *function, int line)
{
  void *data = malloc(size);
  addList(data, size, file, function, line);
  return data;
}

void myfree(void *data)
{
  removeList(data);
  free(data);
}

void debugalloc()
{
  printList();
}

    the removeList() would search for “data” in the global list and remove it, or gives a warning. with “debugalloc();” you should print the whole list (and normally it should be empty, or well filled with what you expect it to have).

    Another option:
    your memory management gets fragmented, … (upps, Alexey Frunze wrote that just a second ago in the comments section.)

    EDITH: XML

    I made a comment about DOM and SAX, and I have the feeling just to expand this comment:

    <head><data>...</data><data>...</data>......<data>...</data></head>

    is an excellent candidate for SAX. In DOM, the whole XML will be read and converted into memory, which can lead very quickly to fragmented memory, if < data > itself is structured with different sized values. it may happen, that your operating system’s memory modell will work against mallocing small amounts of memory.

    With SAX, you only load portions of data into memory and get rid of it as soon as it is consumed. But your implementation must change from traversing the Document into a kind of State Machine with feeding (< data >) and consuming/processing (< / data >).

    • 0