Question

I used SQLite3 to implement small application to read from or write to a database. Some records that need to be added to the database are Arabic texts and when they are stored to the database they converted to non-readable and non-understood texts. I use these APIs for write & read:

• sqlite3_open
• sqlite3_prepare
• sqlite3_bind_text
• sqlite3_step

What can I do to solve the problem ?

Answer 1

It is most likely that your text is in non-ASCII encoding. For example, in unicode.

This is because ASCII table has only characters represented by integer numbers from 0 to 127. So there is nothing that can be used to represent Arabic letters. For example, Unicode is using five different ranges to represent Arabic language:

• Arabic (0600—06FF, 224 characters)
• Arabic Supplement (0750—077F, 48 characters)
• Arabic Presentation Forms-A (FB50—FDFF, 608 characters)
• Arabic Presentation Forms-B (FE70—FEFF, 140 characters)
• Rumi Numeral Symbols (10E60—10E7F, 31 characters)

And since there could be more letters/characters that a 8-bit value (char type, which has a length of 1 byte) would allow, wide character is used to represent some (or even all) of those letters.

As a result, the length of the string in characters will be different from length of the string in bytes. My assumption is that when you use sqlite3_bind_text function, you pass a number of characters as a fourth parameter, whereas it should be a number of bytes. Or you could misinterpret this length when reading the string back from the database. The sqlite3_bind_text documentation is saying this about the fourth parameter:

In those routines that have a fourth argument, its value is the number
of bytes in the parameter. To be clear: the value is the number of
bytes in the value, not the number of characters. If the fourth
parameter is negative, the length of the string is the number of bytes
up to the first zero terminator.

Make sure you do the right thing there.

See also:

• Wide characters
• Unicode
• Arabic characters in Unicode

Good luck!