Question

What is the DRYest way to benchmark against several matrices using the ruby Benchmark stdlib or an equivalent gem? For instance, if I was benchmarking some SQL, I might want to try all combinations of sets of table size, various queries, number of queries, etc.

Answer 1

require 'benchmark'

table_sizes = [100, 1000]
queries = ["QUERY1", "QUERY2"]
num_queries = [1, 2, 3]
combis = table_sizes.product(queries, num_queries)

Benchmark.bm(20) do |x|
  combis.each do |combi| # or |size,query,num| and use those
    x.report( combi.inspect ){ sleep rand } #do something with combi
  end
end

Output:

                           user     system      total        real
[100, "QUERY1", 1]     0.000000   0.000000   0.000000 (  0.524222)
[100, "QUERY1", 2]     0.000000   0.000000   0.000000 (  0.075334)
[100, "QUERY1", 3]     0.000000   0.000000   0.000000 (  0.041683)
[100, "QUERY2", 1]     0.000000   0.000000   0.000000 (  0.227865)
[100, "QUERY2", 2]     0.000000   0.000000   0.000000 (  0.558131)
[100, "QUERY2", 3]     0.000000   0.000000   0.000000 (  0.284121)
[1000, "QUERY1", 1]    0.000000   0.000000   0.000000 (  0.496799)
[1000, "QUERY1", 2]    0.000000   0.000000   0.000000 (  0.304552)
[1000, "QUERY1", 3]    0.000000   0.000000   0.000000 (  0.918314)
[1000, "QUERY2", 1]    0.000000   0.000000   0.000000 (  0.332485)
[1000, "QUERY2", 2]    0.000000   0.000000   0.000000 (  0.379680)
[1000, "QUERY2", 3]    0.000000   0.000000   0.000000 (  0.804835)