Question

I have a hash with about 130,000 elements, and I am trying to check all combinations within that hash for something (130,000 x 130,000 combinations). My code looks like this:

 foreach $key1 (keys %CNV)
 {

  foreach $key2 (keys %CNV)
  {
         if (blablabla){do something that doesn't take as long}
  }

 }

As you might expect, this takes ages to run. Does anyone know a quicker way to do this? Many thanks in advance!!

-Abdel

Edit: Update on the blablabla.

Hey guys, thanks for all the feedback! Really appreciate it. I changed the foreach statement to:

for ($j=1;$j<=24;++$j)
 {
  foreach $key1 (keys %{$CNV{$j}})
  {

   foreach $key2 (keys %{$CNV{$j}})
   {
                        if (blablabla){do something}
                        }
                }
        }

The hash is now multidimensional:

$CNV{chromosome}{$start,$end}

I’ll elaborate on what I’m exactly trying to do, as requested.

The blablabla is the following:

if  ( (($CNVstart{$j}{$key1} >= $CNVstart{$j}{$key2}) && ($CNVstart{$j}{$key1} <= $CNVend{$j}{$key2})) ||
   (($CNVend{$j}{$key1} >= $CNVstart{$j}{$key2}) && ($CNVend{$j}{$key1} <= $CNVend{$j}{$key2})) ||
   (($CNVstart{$j}{$key2} >= $CNVstart{$j}{$key1}) && ($CNVstart{$j}{$key2} <= $CNVend{$j}{$key1})) ||
   (($CNVend{$j}{$key2} >= $CNVstart{$j}{$key1}) && ($CNVend{$j}{$key2} <= $CNVend{$j}{$key1})) 
  )    

In short: The hash elements represent a specific part of the DNA (a so called “CNV”, think of it like a gene for now), with a start and an end (which are integers representing their position on that particular chromosome, stored in hashes with the same keys: %CNVstart & %CNVend). I’m trying to check for every combination of CNVs whether they overlap. If there are two elements that overlap within a family (I mean a family of persons whose DNA I have and read in; there is also a for-statement inside the foreach-statement that let’s the program check this for every family, which makes it last even longer), I check whether they also have the same “copy number” (which is stored in another hash with the same keys) and print out the result.

Thank you guys for your time!

Answer 1

I think I found the answer 🙂
Couldn’t have done it without you guys though. I found a way to skip most of the comparisons I make:

for ($j=1;$j<=24;++$j)
 {
            foreach $key1 (sort keys %{$CNV{$j}})
            {


                foreach $key2 (sort keys %{$CNV{$j}})
                {

                    if (($CNVstart{$j}{$key2} < $CNVstart{$j}{$key1}) && ($CNVend{$j}{$key2} < $CNVstart{$j}{$key1}))
                    {
                    next;
                    }


                    if (($CNVstart{$j}{$key2} > $CNVend{$j}{$key1}) && ($CNVend{$j}{$key2} > $CNVend{$j}{$key1}))
                    {
                    last;
                    }

        if  ( (($CNVstart{$j}{$key1} >= $CNVstart{$j}{$key2}) && ($CNVstart{$j}{$key1} <= $CNVend{$j}{$key2})) ||
           (($CNVend{$j}{$key1} >= $CNVstart{$j}{$key2}) && ($CNVend{$j}{$key1} <= $CNVend{$j}{$key2})) ||
           (($CNVstart{$j}{$key2} >= $CNVstart{$j}{$key1}) && ($CNVstart{$j}{$key2} <= $CNVend{$j}{$key1})) ||
           (($CNVend{$j}{$key2} >= $CNVstart{$j}{$key1}) && ($CNVend{$j}{$key2} <= $CNVend{$j}{$key1})) 
          )    {print some stuff out}

    }
    }
}

What I did is:

• sort the keys of the hash for each foreach loop
• do “next” if the CNVs with $key2 still haven’t reached the CNV with $key1 (i.e. start2 and end2 are both smaller than start1)
• and probably the most time-saving: end the foreach loop if the CNV with $key2 has overtaken the CNV with $key1 (i.e. start2 and end2 are both larger than end1)

Thanks a lot for your time and feedback guys!