Question

Using C, is there a way to read only the last line of a file without looping it’s entire content?

Thing is that file contains millions of lines, each of them holding an integer (long long int). The file itself can be quite large, I presume even up to 1000mb. I know for sure that the last line won’t be longer than 55 digits, but could be 2 only digits as well. It’s out of options to use any kind of database… I’ve considered it already.

Maybe its a silly question, but coming from PHP background I find it hard to answer. I looked everywhere but found nothing clean.

Currently I’m using:

if ((fd = fopen(filename, "r")) != NULL) // open file
{
    fseek(fd, 0, SEEK_SET); // make sure start from 0
    while(!feof(fd))
    {
        memset(buff, 0x00, buff_len); // clean buffer
        fscanf(fd, "%[^\n]\n", buff); // read file *prefer using fscanf
    }
    printf("Last Line :: %d\n", atoi(buff)); // for testing I'm using small integers
}

This way I’m looping file’s content and as soon as file gets bigger than ~500k lines things slow down pretty bad….

Thank you in advance.
maxim

Answer 1

If there is a maximum line length, seek to that distance before the end.
Read up to the end, and find the last end-of-line in your buffer.

If there is no maximum line length, guess a reasonable value, read that much at the end, and if there is no end-of-line, double your guess and try again.

In your case:

/* max length including newline */
static const long max_len = 55 + 1;
/* space for all of that plus a nul terminator */
char buf[max_len + 1];

/* now read that many bytes from the end of the file */
fseek(fd, -max_len, SEEK_END);
ssize_t len = read(fd, buf, max_len);

/* don't forget the nul terminator */
buf[len] = '\0';

/* and find the last newline character (there must be one, right?) */
char *last_newline = strrchr(buf, '\n');
char *last_line = last_newline+1;