Question

I am getting start to socket use in C programming language. I am trying to make an simple http request and store the buffer obtained from read() in my buffer. For this, I use pointers/realloc(), the C programs works fine, compile no errors, but it is reading only a part of the http response.

For example, if I try to get the binary of google’s logo: http://www.google.com/images/srpr/logo3w.png the Content-Length say 7007 bytes, but strlen(buffer) say 5146 for me.I belive that the mistake out here is my buf_size and realloc() why bytesreaded is 7337 the 330 bytes I belive that is of headers.

Here is my code:

char *
httpget(const char * domain, const int port, const char * headers)
{
    int sockfd; /* Socket file descrption */
    int buf_size = MAX_BUFFER_SIZE;

    struct sockaddr_in  sock_addr; 
    struct hostent  *   host;

    char * buffer;
    char * newbuf;
    char * tbuf;

    sockfd = socket(AF_INET, /* Uses IPV4 Internet protocols */
                    SOCK_STREAM, /* Uses the TCP (Transfer Communication Protocol) */
                    0  /* "0" for socket () function choose the correct protocol based on the socket type. */
                    );

    if( sockfd == -1 )
    {
        return NULL;
    }

    host = gethostbyname(domain);

    if( NULL == host )
    {
        close(sockfd);
        return NULL;
    }

    memset(&sock_addr, '\0', sizeof(sock_addr));
    sock_addr.sin_family = AF_INET;
    memcpy( &sock_addr.sin_addr.s_addr,
            host -> h_addr,
            host -> h_length );

    sock_addr.sin_port = htons(port);

    if( connect(sockfd, (struct sockaddr *) &sock_addr, sizeof(sock_addr)) == -1)
    {
        close(sockfd);
        return NULL;
    }

    if( write(sockfd, headers, strlen(headers) + 1) == -1)
    {
        close(sockfd);
        return NULL;
    }

    buffer = malloc( MAX_BUFFER_SIZE );
    tbuf = malloc( MAX_BUFFER_SIZE );

    if(buffer == NULL || tbuf == NULL)
    {
        return NULL;
    }

    int bytesloaded = 0;
    int readed;

    while( (readed = read(sockfd, tbuf, MAX_BUFFER_SIZE)) > 0 )
    {   

        if(bytesloaded + readed >= buf_size)
        {
            buf_size = buf_size + MAX_BUFFER_SIZE;  
            newbuf = realloc(buffer, buf_size);

            if(newbuf != NULL)
               buffer = newbuf; 
            else 
              return NULL;
        }
          memcpy(buffer + bytesloaded, tbuf, readed);
      bytesloaded += readed;
    }

    //printf("bytesreaded = %d and buffer len is %d\n", bytesloaded, strlen(buffer));
    free(tbuf);
    close(sockfd);

    return buffer;
}

then:

char * domain = "www.google.com\0";
char * sheaders = "GET /images/srpr/logo3w.png HTTP/1.1\r\nHost:www.google.com\r\nConnection:close\r\n\r\n\n\0";
int port = 80;
char * response = httpget(domain, port, sheaders);

Answer 1

Don’t use str* functions on arbitrary data. These are made to operate on C strings, which are zero-terminated. Binary data (most image formats) can contain zeros in the middle.

You should be using memcpy/memmove, and you have to rely on the return value of read to know how much data you actually got. strlen on binary data is meaningless.

Try replacing this part:

bytesloaded += readed;
strcat(buffer, tbuf);

With something like:

if (bytesloaded+readed >= buf_size) {
  // do the realloc now
}
memcpy(buffer+bytesloaded, tbuf, readed);
bytesloded += readed;

buffer + x (with x an integer type whose value is less than the allocated buffer size) is a pointer to the xth char in buffer. (This is pointer arithmetic. The type of buffer matters. In this case, it is invalid if x is negative.)
You need to perform the re-allocation before you attempt the memcpy otherwise you risk writing past the end of the buffer.
memcpy is safe here because you know that buffer and tbuf don’t overlap.