Question

I am trying to query a database. I already have a file that includes some primary keys out of this whole database. Now I want to filter these primary keys and get only those primary keys which also agree to “other condition”. My primary keys are associated to abstracts in the database. The abstracts are full-text indexed. Now I want to consider the abstracts using the given primary keys, look for my “other condition(terms)” in those abstracts and if its present I want to pull their primary keys out(which is going to be same from the file). My “other condition” is another file with a list of terms. I want to get the abstracts that contain those terms within the given primary keys.

My full-text search is something like this:

while(<FILE1>){
$PK = $_;
foreach $foo(@foo){
my $sth = $dbh->prepare(qq{
   SELECT value
     FROM value_table
    WHERE MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
}) AND primary_key=$PK;

$sth->execute(qq{+"$foo"});
}
}

where $PK is coming from the list of primary keys I already have.
$foo will be the list of the terms (condition 2) I am looking for.

Normally, I can run this query number of $PK times number of $foo. But I learned something about optimization by sub querying where I won’t be running my query # $PK times # $foo. That will get rid of inner loop but will still form combination of every $PK with every term in file 2 that is @foo. Something like as follows:

while(<FILE1>){
$PK = $_;
   my $sth = $dbh->prepare(qq{
   SELECT value
     FROM value_table
    WHERE MATCH (column_text_indexed) AGAINST (**SUB QUERYING HERE**)
}) AND primary_key=$PK;

$sth->execute(qq{+"$foo"});

}

Just I don’t know how to do it. I may be wrong with the syntax. I want to know how to write code for full-text search as well as a subquery. I hope this will be efficient than querying directly the combinations. Any help is greatly appreciated.

Answer 1

I don’t think you need to use a subquery. But you can still get rid of the inner loop by combining the match strings.

my $against = join ' ', map {qq/"$_"/} @foo;
while (my $PK = <FILE1>) {
    chomp $PK;

    my $sth = $dbh->prepare(qq{
       SELECT value
         FROM value_table
        WHERE primary_key = ?
          # no '+' so it can match against at least one of the words in the list
          AND MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
    });
    $sth->execute($PK, $against);

Update

I revised it and have completely removed the query from the loops.

my @primary_keys;
while (my $PK = <FILE1>) {
    chomp $PK;
    push @primary_keys, $PK;
}

my $PK_list = join ',', map {qq/'$_'/} @primary_keys;
my $against = join ' ', map {qq/"$_"/} @foo;

my $sth = $dbh->prepare(qq{
   SELECT value
     FROM value_table
    # placeholders can only represent single scalar values so $PK_list can't be bound
    WHERE primary_key IN ($PK_list)
      # no '+' so it can match against at least one of the words in the list
      AND MATCH (column_text_indexed) AGAINST (? IN BOOLEAN MODE)
});
$sth->execute($against);

# continue with fetching the rows
...;