Question

I wrote an application that encrypts text in this way:

1. Get the input text

2. Reverse the text

3. Convert to hexadecimal

4. XOR with a key

5. Base64 encode

Now, I didn’t do a lot of encryption/encoding myself, so my question might sound stupid, but, say I get a file which has a content from the above algorithm and I didn’t know about this algorithm. How would one start “breaking” the text, are there any guidelines, principals, rules to follow?

My question is not tied to those 5 steps, that was a pure example.

As a different example, take the text: A751CD9E1F99. How would I start investigating what this might mean?

Answer 1

In order to break a cipher, cryptanalysts use all the information they can gather. Attacks fall into a number of categories, depending on what is known. Some of the main attacks, from hardest to easiest, are

• ciphertext-only: this is the hardest attack. The analyst tries to collect as many encrypted messages as he can, and analyzes them to look for patterns or biases in the frequency of symbols. However, with good, modern ciphers, there are no patterns. This is infeasible with a good cipher, properly used.
• known-plaintext: having the plaintext corresponding some ciphertext is a big step toward recovering unknown plaintext from new ciphertexts. This is where “reverse-engineering” really begins, because he can test his hypotheses about the algorithm against known input and output. In World War II, cryptanalysts worked hard to build extensive lists of “cribs”—words that were likely to appear in the enemy’s messages—to exploit known-plaintext attacks. For example, weather conditions on a particular day, or place names of battles, etc. were likely to be reported to headquarters in encrypted messages.
• chosen-plaintext: even better is when the cryptanalyst can trick his enemy into encrypting a message created by the cryptanalyst. In wartime, sometimes fake information would be leaked to the enemy, hoping that it would be encrypted and help the cryptanalyst break the code.
• adaptive chosen-plaintext: this is sort of an iterative approach to known plaintext. The cryptanalyst can repeatedly have his chosen-plaintext encrypted by the system, and looks at the results to adjust his next attempt.

Nowadays, likely ways to break a code are through flaws in the system. For example, poor key management might be used, allowing the key to be stolen or guessed. In other cases, a “side-channel” attack might be used. For example, by carefully measuring the time it takes for certain cryptographic operations, an attack might be able to guess that certain bits or bytes of a key are zero, causing a fast path through some algorithm.

Up near the “tinfoil hat” end of the spectrum are methods to intercept radio emissions from computing equipment. This allows a remote agent to “see” what is displayed on a monitor. There are even specially designed fonts to try and disrupt this sort of eavesdropping.