This is so not my area, so I apologize if this is not in scope for this stack.
I am cleaning up (for personal entertainment and making visualization to share with others) survey data (download, 9MB) that went through some manipulations to be anonymized before getting released to the public.
One of the questions was about hourly payment rate and allowed free form text answer. Some of those answers got badly broken characters, two most common cases shown in image below:
I would hate to discard those answers, but I am at loss how to revert them to meaningful state.
-
Ask for better data dump – poked related people about it, but not too hopeful.
-
Try to determine which characters ended up this way. Dealing with encodings is always troublesome and these don’t look like any broken characters I ever seen before so I have no idea where to start and if there are tools available to help with this. That might not even be a valid characters or currency symbols at all.
-
Try to match broken characters to valid currency characters. I strongly suspect one of the two might be € character and other might be £ given that survey was slanted towards English-speaking countries. However will I be able to reliably back up such guess by relative quantity of character to other answers? Unfortunately geo data was not provided, so I can’t match answers to countries.
It was confirmed that this was caused by export bug in survey software and characters do correspond to euro and pound.