Question

For a system I’m designing, I want to be able to verify whether a specific string is ‘valid’ or not, but I want to keep my database of ‘valid’ strings private.

I want to provide clients with a database of all valid strings, but this database is (one-way) encrypted. I’m reluctant to distributing a key to the client, since there are always ways to obtain that specific key from the assembly code of my program (I presume).

Clients must be able to enter strings into my program and it would return a boolean based on the presence of the string in my encrypted file.

More importantly, I want my program to easily check whether the string is in the file, but I want to prevent other programs from easily using (and/or reconstructing) the database.

I have formulated this a bit abstract, because I don’t really know how my system will be looking yet, but I want to know whether something like this would be possible.

Answer 1

What you’re looking for are cryptographic hash functions! MD5 and SHA1 are well known examples, but if you’re building new code without strict performance constraints, SHA256 would be the one to chose, and if you’re looking to make it very difficult to recover the original words, you might want to consider scrypt or bcrypt (although they aren’t as popular and unlikely that your language will include them in the standard library).

Then your database can be as simple as an unordered set of hashes, something like (in Python):

>>> words = ["secret", "hidden"]
>>> db = set([hashlib.sha256(word).hexdigest() for word in words])
>>> db
set(['2bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25b', 'e564b4081d7a9ea4b00dada53bdae70c99b87b6fce869f0c3dd4d2bfa1e53e1c'])
>>> def word_exists(word):
...     return hashlib.sha256(word).hexdigest() in db
...
>>> word_exists("hello")
False
>>> word_exists("hidden")
True