Question

Hi I’m trying to tokenize a string by loading an entire file into a char[] using fread.
For some strange reason it is not always working, and valgrind complains in this very small sample program.

Given an input like test.txt

first
second

And the following program

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/stat.h>


//returns the filesize in bytes
size_t fsize(const char* fname){
  struct stat st ;
  stat(fname,&st);
  return st.st_size;
}

int main(int argc, char *argv[]){
  FILE *fp = NULL;
  if(NULL==(fp=fopen(argv[1],"r"))){
    fprintf(stderr,"\t-> Error reading file:%s\n",argv[1]);
    return 0;
  }
  char buffer[fsize(argv[1])];
  fread(buffer,sizeof(char),fsize(argv[1]),fp);
  char *str = strtok(buffer," \t\n");

  while(NULL!=str){
    fprintf(stderr,"token is:%s with strlen:%lu\n",str,strlen(str));
    str = strtok(NULL," \t\n");
  }
  return 0;
}

compiling like

gcc test.c -std=c99 -ggdb

running like

./a.out test.txt

thanks

Answer 1

Your buffer size should be filesize + 1. The +1 is for the null char.

filesize = fsize(argv[1]);
char buffer[filesize + 1];

Also fread does not put a \0 at the end of the string. So you’ll have to do it yourself as:

fread(buffer,sizeof(char),filesize,fp);
buffer[filesize] = 0;