Question

We have an API call where a large amount of data(16 MB on average) is been sent in the request. The request data is different every time so caching is now good. As our clients receiving the data may use other technologies e.g. PHP, there is a default size which is different across these technologies.

As the data size exceeds the default amount(PHP 8MB), we were thinking of using compression if possible.

Can this be done?

From initial research(may not be correct) compression is used for web site content with caching been used by IIS, so not sure if our problem can be solved using compression.

Answer 1

You can use dynamic content compression.

Compression is not related to caching on IIS.

While this will reduce the size, depending on your data(images or video maybe), it may not compress it to below 8MB. You should consider allowing your client to ask for data in chunks.

Enable chunk transfer encoding on IIS.

Your php clients can receive this data using a client that supports chunk encoding like guzzle

The entity body of requests and responses is inherently a PHP stream
in Guzzle. The body of the request can be either a string or a PHP
stream which are converted into a Guzzle\Http\EntityBody object using
its factory method. When using a string, the entity body is stored in
a temp PHP stream. The use of temp PHP streams helps to protect your
application from running out of memory when sending or receiving large
entity bodies in your messages