Question

Before you decide its a tl:dr (too long, didnt read) post try to read at least some, since Its a question broken down in a lot of small pieces. Some of which you can probably answer and help me.

Please try to help me as much as you can. These types of problems are very common on the internet and I think you will help me and much more people after me.

I am currently researching HTTP services and the protocol itself so that I can discover if it is useful to me.
I have some basic questions as well as some code that needs to be discussed.

First I would like to know how does the communication start? I have discovered that the client sends a message in which it requests a resource (is this correct?). Then what happens? I (as a server) have to reply with what?
Do I need to append a carriage return and a line feed after every response? Somewhere it says there even need to be two (\r\n\r\n).
How can an asynchronous writing be established? (I hope this question is understandable) My primary goal is to achieve a connection between a client and a server and then a continuous data stream from server to the client. Does the client need to reply for every message it gets?
I hope I made my questions clear, since I’m not an expert in these things (yet, I am very interested in it).

And for the programming part of my problem.
I have managed to put together a simple program in Qt in C++ (server side) and a simple client in Objective C (iOS). The client connects and I can read the request header. It is like this:

Data available, incoming:  "GET / HTTP/1.1
Host: localhost:9990
Connection: close
User-Agent: CFStream%20test/1.0 CFNetwork/609 Darwin/12.2.0

Should I reply to this header manually? And if so, what?

The client side code looks like this (i know its not pseudo but i think its pretty self-explanatory):

- (void)setupStream
{
    NSURL *url = [NSURL URLWithString:@"http://localhost:9990"];

    CFHTTPMessageRef message = CFHTTPMessageCreateRequest(NULL, (CFStringRef)@"GET", (CFURLRef)url, kCFHTTPVersion1_1);

    stream = CFReadStreamCreateForHTTPRequest(NULL, message);
    CFRelease(message);

    if (!CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPShouldAutoredirect, kCFBooleanTrue))
    {
        NSLog(@"Some error.");
    }

    CFDictionaryRef proxySettings = CFNetworkCopySystemProxySettings();
    CFReadStreamSetProperty(stream, kCFStreamPropertyHTTPProxy, proxySettings);
    CFRelease(proxySettings);

    if (!CFReadStreamOpen(stream))
    {

        CFRelease(stream);
        NSLog(@"Error opening stream.");
    }

    CFStreamClientContext context = {0, self, NULL, NULL, NULL};
    CFReadStreamSetClient(stream, kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred, readStreamCallback, &context);
    CFReadStreamScheduleWithRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
    NSLog(@"Done");
}

This is the setup stream method. The stream variable is a class variable of type CFReadStreamRef.

The callback looks like this:

static void readStreamCallback(CFReadStreamRef aStream, CFStreamEventType event, void *client)
{
    ViewController *controller = (ViewController*)client;
    [controller handleEvent:event forStream:aStream];
}

And the handle event like this:

- (void)handleEvent:(CFStreamEventType)event forStream:(CFReadStreamRef)aStream
{
    if (aStream != stream)
    {
        return;
    }

    NSLog(@"Handle event callback");

    switch (event)
    {
        case kCFStreamEventHasBytesAvailable:
            NSLog(@"Work log");
            UInt8 bytes[11];
            CFIndex length;
            length = CFReadStreamRead(stream, bytes, 11); //I know 11 bytes is hard coded, its in testing stage now. Feel free to suggest me how to do it better.

            if (length == -1)
            {
                NSLog(@"Error, data length = -1");
                return;
            }

            NSLog(@"Len: %li, data: %s", length, bytes);
        break;
        default:
            NSLog(@"Other event");
        break;
    }
}

And thats practically all the client code that is worth mentioning. The Qt Server part (I will only post the important parts) is done like this: (this is a subclassed QTcpServer class). First the startServer(); is called:

bool Server::startServer()
{
    if (!this->listen(QHostAddress::Any, 9990))
        return false;

    return true;
}

When there is a connection incoming the incomingConnection is fired off with the socket descriptor as a parameter:

void Server::incomingConnection(int handle)
{
    qDebug("New client connected");
    ServerClient *client = new ServerClient(handle, this); //The constructor takes in the socket descriptor needed to set up the socket and the parent (this)
    client->setVectorLocation(clients.count()); //This is a int from a Qvector in which i append the clients, its not important for understanding right now.
    connect(client, SIGNAL(clientDisconnected(int)), this, SLOT(clientDisconnected(int)), Qt::QueuedConnection); //When the client socket emits a disconnected signal the ServerClient class emits a client disconnected signal which the server uses to delete that client from the vector (thats why I use "setVectorLocation(int)") - not important right now
    clients.push_back(client); //And then I append the client to the QVector - not important right now
}

The ClientServer class constructor just creates a new socket and connects the required methods:

ServerClient::ServerClient(int handle, QObject *parent) :
QObject(parent)
{
    socket = new QTcpSocket(this); //Socket is a class variable

    connect(socket, SIGNAL(disconnected()), this, SLOT(disconnected()));
    connect(socket, SIGNAL(readyRead()), this, SLOT(readyRead()));

    socket->setSocketDescriptor(handle);
}

Ready read just writes me the data incoming (it wont be much user later i think):

void ServerClient::readyRead()
{
    qDebug() << "Data available, incoming: " << socket->readAll();
}

And finally the write data:

void ServerClient::writeData(QByteArray *data)
{
    data->append("\r\n\r\n"); //I have read this must be appended to all outgoing data from a HTTP server
    socket->write(*data);
    socket->flush();
    qDebug() << "Written data to client: " << *data;
}

This code however does not always work. Sometimes when I write message like “Message” the client recieves all the data and some things that shouldnt be there (the new line and a wierd symbol – can NSLog cause this?). Sometimes when I send "Hellow" the client only gets "Hel" and some other funky stuff.

What are the problems? What should I pay more attention about? Anything that will help me will be MUCH appreciated. And please dont paste in some links that contain a book with a few hundred pages, Im sure this can be solved just by explaining things to me.

THANKS A LOT!
Jan.

Answer 1

You asked many questions … and that’s a perfectly legitimate thing to do 🙂

I confess – it was too long, I didn’t read 🙁

BUT …

1) Yes, the HTTP protocol does expect na “CRLF” (“\r\n”). Many servers and many clients are “forgiving”, but strictly speaking – yes, you need them.

REFERENCE: RFC 2616

2) Wanting to understand HTTP “internals” is also perfectly legitimate – I applaud you.

One good way is to read the RFC(s).

Another is to use a “telnet” client: http://blog.tonycode.com/tech-stuff/http-notes/making-http-requests-via-telnet

Yet another is to study requests and responses in FF Firebug

3) Socket programming is another issue – which explains why sometimes you might read “hello world”, and other times you might just get “hel”.

Strong recommendation: Beej’s Guide to Network Programming

4) Finally, no way would I actually write a server in Qt with C++ (except maybe as a toy “science experiment”, or for some really off-the-wall requirement)

I would definitely write server code in C# (for Windows servers), Java (for everything else) or a scripting language I felt comfortable with (Perl, Ruby/RoR, Python and Lua all come to mind).

IMHO .. and hope that helps!