Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
It seems that Sqlite FTS don’t support searching Japanese characters according to my experiments , and discussion here.
#select * from tblEvent_shortdes where short_des MATCH 'ＢＳジャパンの見どころ'
#return nothing
select * from tblEvent_shortdes where short_des MATCH 'パンの見'
Customize tokenizer in FTS seems to be the way to accomplish this but I did not found any promising open sourced tokenizer for Japanese. Will ICU tokenizer do?
You might take a look at ChaSen and MeCab. It has been several years since I used either – and it looks as though neither has been updated recently – but both proved adequate at Japanese tokenization.