Question

Following conversion

SELECT to_tsvector('english', 'Google.com');

returns this:

'google.com':1

Why does TSearch2 engine didn’t return something like this?

'google':2, 'com':1

Or how can i make the engine to return the exploded string as i wrote above?
I just need “Google.com” to be foundable by “google”.

Answer 1

Unfortunately, there is no quick and easy solution.

Denis is correct in that the parser is recognizing it as a hostname, which is why it doesn’t break it up.

There are 3 other things you can do, off the top of my head.

1. You can disable the host parsing in the database. See postgres documentation for details. E.g. something like ALTER TEXT SEARCH CONFIGURATION your_parser_config
DROP MAPPING FOR url, url_path

2. You can write your own custom dictionary.

3. You can pre-parse your data before it’s inserted into the database in some manner (maybe splitting all domains before going into the database).

---

I had a similar issue to you last year and opted for solution (2), above.

My solution was to write a custom dictionary that splits words up on non-word characters. A custom dictionary is a lot easier & quicker to write than a new parser. You still have to write C tho 🙂

The dictionary I wrote would return something like 'www.facebook.com':4, 'com':3, 'facebook':2, 'www':1' for the ‘www.facebook.com’ domain (we had a unique-ish scenario, hence the 4 results instead of 3).

The trouble with a custom dictionary is that you will no longer get stemming (ie: http://www.books.com will come out as www, books and com). I believe there is some work (which may have been completed) to allow chaining of dictionaries which would solve this problem.