I wrote a C program on linux to process large amount of data, it

Question

0

Asked: May 31, 20262026-05-31T21:50:37+00:00 2026-05-31T21:50:37+00:00

I wrote a C program on linux to process large amount of data, it

0

I wrote a C program on linux to process large amount of data, it run OK most of the time. But when processing a set of particular data it always told me the ‘segmentation fault’ error, I tried to use gdb without luck (see my another question How to deal segmentation fault with GDB under particular circumstance?), so I tried to use valgrind. It didn’t tell me the ‘segmentation fault’ error when run with valgrind. So how to find the bug according to these valgrind output?

==2441== Invalid write of size 4
==2441==    at 0x404893: nodes_term32_flush (tyn_indexer.c:227)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid write of size 4
==2441==    at 0x4048D8: nodes_term32_flush (tyn_indexer.c:254)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x40450D: nodes_term32_flush (tyn_indexer.c:260)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x404878: nodes_term32_flush (tyn_indexer.c:224)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid write of size 4
==2441==    at 0x40487F: nodes_term32_flush (tyn_indexer.c:224)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x4048BD: nodes_term32_flush (tyn_indexer.c:251)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c094 is not stack'd, malloc'd or (recently) free'd
==2441==
==2441== Invalid write of size 4
==2441==    at 0x4048C4: nodes_term32_flush (tyn_indexer.c:251)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c094 is not stack'd, malloc'd or (recently) free'd
==2441==
==2441== Invalid read of size 4
==2441==    at 0x4093A0: tyn_p4d_encode32 (tyn_coder.c:645)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x409490: tyn_p4d_encode32 (tyn_coder.c:669)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x4094A7: tyn_p4d_encode32 (tyn_coder.c:667)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c088 is 8 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x409574: tyn_p4d_encode32 (tyn_coder.c:694)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x4095A6: tyn_p4d_encode32 (tyn_coder.c:708)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c088 is 8 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x409524: tyn_p4d_encode32 (tyn_coder.c:697)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c08c is 12 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x40953A: tyn_p4d_encode32 (tyn_coder.c:700)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1233c08c is 12 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x409552: tyn_p4d_encode32 (tyn_coder.c:702)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1243c2fc is 12 bytes after a block of size 16 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x40785B: tyn_exsorter_sort (tyn_exsorter.c:106)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x407CB7: tyn_iS16_encode32 (tyn_coder.c:96)
==2441==    by 0x409621: tyn_p4d_encode32 (tyn_coder.c:725)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1243d880 is 0 bytes after a block of size 16 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x40785B: tyn_exsorter_sort (tyn_exsorter.c:106)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x407D20: tyn_iS16_encode32 (tyn_coder.c:109)
==2441==    by 0x409621: tyn_p4d_encode32 (tyn_coder.c:725)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1243d880 is 0 bytes after a block of size 16 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x40785B: tyn_exsorter_sort (tyn_exsorter.c:106)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Invalid read of size 4
==2441==    at 0x407D37: tyn_iS16_encode32 (tyn_coder.c:108)
==2441==    by 0x409621: tyn_p4d_encode32 (tyn_coder.c:725)
==2441==    by 0x4045F0: nodes_term32_flush (tyn_indexer.c:132)
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==  Address 0x1243d8a8 is not stack'd, malloc'd or (recently) free'd
==2441==

==2441== Use of uninitialised value of size 8
==2441==    at 0x3AE9C4726B: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4071EF: tyn_build_index (tyn_indexer.c:888)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Conditional jump or move depends on uninitialised value(s)
==2441==    at 0x3AE9C47275: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4071EF: tyn_build_index (tyn_indexer.c:888)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
document id: 0
==2441== Conditional jump or move depends on uninitialised value(s)
==2441==    at 0x3AE9C4774E: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4073AD: tyn_build_index (tyn_indexer.c:900)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Use of uninitialised value of size 8
==2441==    at 0x3AE9C4726B: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4073AD: tyn_build_index (tyn_indexer.c:900)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
==2441== Conditional jump or move depends on uninitialised value(s)
==2441==    at 0x3AE9C47275: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4073AD: tyn_build_index (tyn_indexer.c:900)
==2441==    by 0x40384F: main (tyn_indexer.c:943)
==2441==
int32_t category: 0
bytes_collected: 972467429
196220 bytes/s
tyn_config->indexer->dl_directory: /dragon/tyan
tyn_config->indexer->memory_limit: 10240000
==2441==
==2441== HEAP SUMMARY:
==2441==     in use at exit: 214,695,668 bytes in 399,272 blocks
==2441==   total heap usage: 401,095 allocs, 1,823 frees, 219,225,806 bytes allocated
==2441==
==2441== LEAK SUMMARY:
==2441==    definitely lost: 9,442,580 bytes in 27 blocks
==2441==    indirectly lost: 34,682,771 bytes in 81 blocks
==2441==      possibly lost: 170,557,809 bytes in 399,140 blocks
==2441==    still reachable: 12,508 bytes in 24 blocks
==2441==         suppressed: 0 bytes in 0 blocks
==2441== Rerun with --leak-check=full to see details of leaked memory
==2441==
==2441== For counts of detected and suppressed errors, rerun with: -v
==2441== Use --track-origins=yes to see where uninitialised values come from
==2441== ERROR SUMMARY: 32680 errors from 23 contexts (suppressed: 2 from 2)

Report

Leave an answer
Cancel reply

You must login to add an answer.

Need An Account,

1 Answer

Editorial Team · Answer 1 · 2026-05-31T21:50:38+00:00

That’s a lot of output you have here. You should have condensed it more to the areas you are interested. I’ll try to give a few general pointers, though.

==2441== Invalid write of size 4

opens a “block” that tells you details about the allocated memory and the stack backtrace of its allocation and the error site. The ==2441== is the PID of the process, which is useful in cases where multiple processes run in parallel.

Error site (comments added with #). The stack backtrace is always in reverse order, meaning the crash (or what would have become a crash normally) site is at the top of the listing and the calls that led there are listed in reverse order below:

# This points to a function nodes_term32_flush() in file tyn_indexer.c on line 227
==2441==    at 0x404893: nodes_term32_flush (tyn_indexer.c:227)
# the format is the same, at this line tyn_exsorter_sort() calls nodes_term32_flush()
==2441==    by 0x407B77: tyn_exsorter_sort (tyn_exsorter.c:131)
# ... and so on
==2441==    by 0x406DDE: tyn_build_index (tyn_indexer.c:731)
# ... leading up to the process "entry point"
==2441==    by 0x40384F: main (tyn_indexer.c:943)

The memory block that was affected. The first line tells us that a block of exactly one MiB was allocated and that (together with the output above) you were reading the first 4 bytes (likely a 32bit value) after the last allocated byte of that block. The rest of the format follows what you know from the stack backtrace above.

==2441==  Address 0x1233c080 is 0 bytes after a block of size 1,048,576 alloc'd
==2441==    at 0x4A074CD: malloc (vg_replace_malloc.c:236)
==2441==    by 0x406BEB: tyn_build_index (tyn_indexer.c:663)
==2441==    by 0x40384F: main (tyn_indexer.c:943)

All the repetitions afterwards are – from experience – most often a consequence of the first error. So always start fixing the first listed issue in a sequence of reported issues.

Now for the other error class that appears in your output:

==2441== Use of uninitialised value of size 8
# All library functions here ...
==2441==    at 0x3AE9C4726B: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
# ... but this one should be yours. Check out this file/line to see what can be the problem with the printf() call
==2441==    by 0x4071EF: tyn_build_index (tyn_indexer.c:888)
==2441==    by 0x40384F: main (tyn_indexer.c:943)

The following is likely again just a consequence of the output above:

==2441== Conditional jump or move depends on uninitialised value(s)
==2441==    at 0x3AE9C47275: _itoa_word (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C49852: vfprintf (in /lib64/libc-2.14.90.so)
==2441==    by 0x3AE9C51FE8: printf (in /lib64/libc-2.14.90.so)
==2441==    by 0x4071EF: tyn_build_index (tyn_indexer.c:888)
==2441==    by 0x40384F: main (tyn_indexer.c:943)

I strongly suggest two things: read the Valgrind manual (I know this sounds patronizing, but it’s worth it) and make use of its many options. I created a few functions that I am using on my development machines:

# vim: set autoindent smartindent tabstop=2 shiftwidth=2 expandtab filetype=sh:

function vgrun
{
  local COMMAND="$1"
  local NAME="$2"
  [[ -n "$COMMAND" ]] || { echo "Syntax: vgrun <command> <name>"; return; }
  [[ -n "$NAME" ]] || { echo "Syntax vgrun <command> <name>"; return; }
  valgrind \
        --leak-check=full --error-limit=no --track-origins=yes \
        --undef-value-errors=yes --log-file=valgrind-${NAME}.log \
        --read-var-info=yes \
        $COMMAND | tee valgrind-${NAME}-output.log 2>&1
}

function vgtrace
{
  local COMMAND="$1"
  local NAME="$2"
  [[ -n "$COMMAND" ]] || { echo "Syntax: vgtrace <command> <name>"; return; }
  [[ -n "$NAME" ]] || { echo "Syntax vgtrace <command> <name>"; return; }
  valgrind \
        --leak-check=full --error-limit=no --track-origins=yes \
        --undef-value-errors=yes --log-file=valgrind-${NAME}.log \
        --read-var-info=yes --trace-children=yes \
        $COMMAND | tee valgrind-${NAME}-output.log 2>&1
}

function vgdbg
{
  [[ -n "$*" ]] || { echo "Syntax: vgrun <command>"; return; }
  valgrind \
        --leak-check=full --error-limit=no --track-origins=yes \
        --undef-value-errors=yes --read-var-info=yes --db-attach=yes \
        "$@"
}

vgrun simply runs a command with Valgrind. Note that the command needs to be in quotes with all its parameters for this one to work. vgtrace is merely a variation of vgrun, adding --trace-children=yes to the Valgrind command line. The most useful, by far, is vgdbg, which will ask you to attach GDB to the running program and thus lets you interactively debug the issue at hand, including properly inspecting stack frames, values etc – if you speak GDB, that is.

Sign Up

Sign In

Forgot Password

The Archive Base Latest Questions

I wrote a C program on linux to process large amount of data, it

Leave an answerCancel reply

1 Answer

Leave an answer
Cancel reply