Question

My question has two parts regarding haproxy.

Question 1:

The service that I am working with deals with uploading massive XML documents, and downloading results that are also pretty large. The uploaded XML documents are used by workers to generate responses, which can later be downloaded form the workers. Here is a rough setup:

–request–> http://www.domain.com –> worker(1 or 2).domain.com

http://www.domain.com is a static content server, and is also where haproxy would reside. All requests will initially come here. The static content (html, images, etc) should be served by nginx on http://www.domain.com.

However requests with specific urls such as http://www.domain.com/%5Bupload%5D/%5Bid1%5D, or http://www.domain.com/%5Bdownload%5D/%5Bid1%5D need to be served by the workers themselves. An additional catch here is that the forwarding needs to be sticky based on the url.

For example:

Suppose an XML file was uploaded to http://www.domain.com/upload/123. haproxy will see that the url contains upload and will route the request to either worker1.domain.com or worker2.domain.com. Later, when a GET request to http://www.domain.com/download/123 is made, I would need haproxy to route it to the worker that that originally served upload request with the same id (123). Basically, I cannot have the upload request for a given id go to worker1 and then download request for the same id go to worker2. Would something like this be possible using haproxy?

Question 2:

Given that I am uploading a file to http://www.domain.com/upload, and haproxy routes it to worker1.domain.com, will the entire file pass through haproxy? That is to say, while worker1.domain.com will receive the file, will haproxy also take the same bandwidth hit since it is doing the forwarding?

Thank you very much!

Answer 1

1. This solution would require one backend per worker plus the load balancing backend:
   • haproxy sends all regular requests to the general backend
   • the workers set a cookie on the client
   • subsequent requests are sent to the worker-specific backends by inspecting the cookie (with hdr_sub(Cookie)), something like “use_backend worker1 if { hdr_sub(Cookie) SRV=1 }
2. Yes, the entire file passes through haproxy. You could let the workers redirect the clients to the worker directly, in that case the client will continue using the worker. However if the worker goes down the client won’t have a way to find a different worker. Depending on your use case, having haproxy proxy the whole file isn’t a problem though, it’s very performant.

You might be able to do the backend selection with a table but I’m not sure how. You can store things in tables in the backend and read them in the frontend, though.