Question

Recently I came across one nice problem, which turned up as simple to understand as hard to find any way to solve. The problem is:

Write a program, that reads a text from input and prints some other
program on output. If we compile and run the printed program, it must
output the original text.

The input text is supposed to be rather large (more than 10000 characters).

The only (and very strong) requirement is that the size of the archive (i.e. the program printed) must be strictly less than the size of the original text. This makes impossible obvious solutions like

std::string s;
/* read the text into s */
std::cout << "#include<iostream> int main () { std::cout<<\"" << s << "\"; }";

I believe some archiving techniques are to be used here.

Answer 1

Unfortunately, such a program does not exist.

To see why this is so, we need to do a bit of math. First, let’s count up how many binary strings there are of length n. Each of the bits can be either a 0 or 1, which gives us one of two choices for each of those bits. Since there are two choices per bit and n bits, there are thus a total of 2ⁿ binary strings of length n.

Now, let’s suppose that we want to build a compression algorithm that always compresses a bitstring of length n into a bitstring of length less than n. In order for this to work, we need to count up how many different strings of length less than n there are. Well, this is given by the number of bitstrings of length 0, plus the number of bitstrings of length 1, plus the number of bitstrings of length 2, etc., all the way up to n – 1. This total is

2⁰ + 2¹ + 2² + … + 2^{n – 1}

Using a bit of math, we can get that this number is equal to 2ⁿ – 1. In other words, the total number of bitstrings of length less than n is one smaller than the number of bitstrings of length n.

But this is a problem. In order for us to have a lossless compression algorithm that always maps a string of length n to a string of length at most n – 1, we would have to have some way of associating every bitstring of length n with some shorter bitstring such that no two bitstrings of length n are associated with the same shorter bitstream. This way, we can compress the string by just mapping it to the associated shorter string, and we can decompress it by reversing the mapping. The restriction that no two bitstrings of length n map to the same shorter string is what makes this lossless – if two length-n bitstrings were to map to the same shorter bitstring, then when it came time to decompress the string, there wouldn’t be a way to know which of the two original bitstrings we had compressed.

This is where we reach a problem. Since there are 2ⁿ different bitstrings of length n and only 2ⁿ-1 shorter bitstrings, there is no possible way we can pair up each bitstring of length n with some shorter bitstring without assigning at least two length-n bitstrings to the same shorter string. This means that no matter how hard we try, no matter how clever we are, and no matter how creative we get with our compression algorithm, there is a hard mathematical limit that says that we can’t always make the text shorter.

So how does this map to your original problem? Well, if we get a string of text of length at least 10000 and need to output a shorter program that prints it, then we would have to have some way of mapping each of the 2¹⁰⁰⁰⁰ strings of length 10000 onto the 2¹⁰⁰⁰⁰ – 1 strings of length less than 10000. That mapping has some other properties, namely that we always have to produce a valid program, but that’s irrelevant here – there simply aren’t enough shorter strings to go around. As a result, the problem you want to solve is impossible.

That said, we might be able to get a program that can compress all but one of the strings of length 10000 to a shorter string. In fact, we might find a compression algorithm that does this, meaning that with probability 1 – 2¹⁰⁰⁰⁰ any string of length 10000 could be compressed. This is such a high probability that if we kept picking strings for the lifetime of the universe, we’d almost certainly never guess the One Bad String.

---

For further reading, there is a concept from information theory called Kolmogorov complexity, which is the length of the smallest program necessary to produce a given string. Some strings are easily compressed (for example, abababababababab), while others are not (for example, sdkjhdbvljkhwqe0235089). There exist strings that are called incompressible strings, for which the string cannot possibly be compressed into any smaller space. This means that any program that would print that string would have to be at least as long as the given string. For a good introduction to Kolmogorov Complexity, you may want to look at Chapter 6 of “Introduction to the Theory of Computation, Second Edition” by Michael Sipser, which has an excellent overview of some of the cooler results. For a more rigorous and in-depth look, consider reading “Elements of Information Theory,” chapter 14.

Hope this helps!