Question

I have to encrypt some file (jpg) using vigenere cipher. I wrote some code, but after encryption and decryption my file is corrupted. The first 1/4 of image displays okay, but the rest of it is corrupted. Here is my code:

@Override
public byte[] encryptFile(byte[] file, String key) {
    char[] keyChars = key.toCharArray();
    byte[] bytes = file;
    for (int i = 0; i < file.length; i++) {
        int keyNR = keyChars[i % keyChars.length] - 32;
        int c = bytes[i] & 255;
        if ((c >= 32) && (c <= 127)) {
            int x = c - 32;
            x = (x + keyNR) % 96;
            bytes[i] = (byte) (x + 32);
        }
    }
    return bytes;
}


@Override
public byte[] decryptFile(byte[] file, String key) {
    char[] keyChars = key.toCharArray();
    byte[] bytes = file;
    for (int i = 0; i < file.length; i++) {
        int keyNR = keyChars[i % keyChars.length] - 32;
        int c = bytes[i] & 255;
        if ((c >= 32) && (c <= 127)) {
            int x = c - 32;
            x = (x - keyNR + 96) % 96;
            bytes[i] = (byte) (x + 32);
        }
    }
    return bytes;
}

What did I do wrong?

EDIT:

reading and writing to file:

public void sendFile(String selectedFile, ICipher cipher, String key) {
    try {
        DataOutputStream outStream = new DataOutputStream(client
                .getOutputStream());
        outStream.flush();
        File file = new File(selectedFile);
        FileInputStream fileStream = new FileInputStream(file);
        long fileSize = file.length();
        long completed = 0;
        long bytesLeft = fileSize - completed;
        String msg = "SENDING_FILE:" + file.getName() + ":" + fileSize;
        outStream.writeUTF(cipher.encryptMsg(msg, key));
        while (completed < fileSize) {
            int step = (int) (bytesLeft > 150000 ? 150000 : bytesLeft);
            byte[] buffer = new byte[step];
            fileStream.read(buffer);
            buffer = cipher.encryptFile(buffer, key);
            outStream.write(buffer);
            completed += step;
            bytesLeft = fileSize - completed;
        }
        outStream.writeUTF(cipher.encryptMsg("SEND_COMPLETE", key));
        fileStream.close();
    } catch (IOException e) {
        e.printStackTrace();
    }

}

    private void downloadFile(String fileName, int fileSize,DataInputStream input,ICipher cipher, String key) {
    try {
        FileOutputStream outStream = new FileOutputStream("C:\\" + fileName);
        int bytesRead = 0, counter = 0;

        while (counter < fileSize) {
            int step = (int) (fileSize > 150000 ? 150000 : fileSize);
            byte[] buffer = new byte[step];
            bytesRead = input.read(buffer);
            if (bytesRead >= 0) {
                buffer = cipher.decryptFile(buffer, key);
                outStream.write(buffer, 0, bytesRead);
                counter += bytesRead;
            }
            if (bytesRead < 1024) {
                outStream.flush();
                break;
            }
        }

        Display.getDefault().syncExec(new Runnable() {
            @Override
            public void run() {
                window.handleMessage("Download sucessfully");
            }
        });
        outStream.close();

    } catch (Exception e) {
        Display.getDefault().syncExec(new Runnable() {
            @Override
            public void run() {
                window.handleMessage("Error on downloading file!");
            }
        });
    }
}

Answer 1

You encode the file in whatever chunks come from the disk I/O:

        int step = (int) (bytesLeft > 150000 ? 150000 : bytesLeft);
        byte[] buffer = new byte[step];
        fileStream.read(buffer);
        buffer = cipher.encryptFile(buffer, key);

But you decode the file in whatever chunks come from the network I/O:

        bytesRead = input.read(buffer);
        if (bytesRead >= 0) {
            buffer = cipher.decryptFile(buffer, key);
            outStream.write(buffer, 0, bytesRead);
            counter += bytesRead;
        }

These chunks are likely to disagree. The disk I/O may always give you full chunks (lucky for you), but the network I/O will likely give you packet-sized chunks (1500 bytes minus header).

The cipher should get an offset into the already encoded/decoded data (or encode/decode everything at once), and use that to shift the key appropriately, or this may happen:

original: ...LOREM IPSUM...
key     : ...abCde abCde...
encoded : ...MQUIR JRVYR...
key     : ...abCde Cdeab... <<note the key got shifted
decoded : ...LOREM GNQXP... <<output wrong after the first chunk.

Since the packet data size is (for Ethernet-sized TCP/IP packets) aligned at four bytes, a key of length four is likely to be always aligned.

---

another issue is that you are ignoring the number of bytes read from disk when uploading the file. While disk I/O is likely to always give you full-sized chunks (the file’s likely to be memory-mapped or the underlying native API does provide this guarantee), nothing’s taken for granted. Always use the amount of bytes actually read: bytesRead = fileStream.read(buffer);