Question

Surfing the net I ran into Aquabrowser (no need to click, I’ll post a pic of the relevant part).

It has a nice way of presenting search results and discovering semantically linked entities.

Here is a screenshot taken from one of the demos.

On the left side you have they word you typed and related words.
Clicking them refines your results.

Now as an example project I have a data set of film entities and subjects (like wolrd-war-2 or prison-escape) and their relations.

Now I imagine several use cases, first where a user starts with a keyword.
For example “world war 2”.

Then i would somehow like to calculate related keywords and rank them.

I think about some sql query like this:

Lets assume “world war 2” has id 3.

select keywordId, count(keywordId) as total from keywordRelations 
WHERE movieId IN (select movieId from keywordRelations 
                  join movies using (movieId)      
                  where keywordId=3) 
 group by keywordId order by total desc

which basically should select all movies which also have the keyword world-war-2 and then looks up the keywords which theese films have as well and selects those which occour the most.

I think with theese keywords I can select movies which match best and have a nice tag cloud containing similar movies and related keywords.

I think this should work but its very, very, very inefficient.

And its also only one level or relation.

There must be a better way to do this, but how??

I basically have an collection of entities. They could be different entities (movies, actors, subjects, plot-keywords) etc.

I also have relations between them.

It must somehow be possible to efficiently calculate “semantic distance” for entities.

I also would like to implement more levels of relation.

But I am totally stuck. Well I have tried different approaches but everything ends up in some algorithms that take ages to calculate and the runtime grows exponentially.

Are there any database systems available optimized for that?

Can someone point me in the right direction?

Answer 1

You probably want an RDF triplestore. Redland is a pretty commonly used one, but it really depends on your needs. Queries are done in SPARQL, not SQL. Also… you have to drink the semantic web koolaid.