Question

While the documentation specifies how one can substitute a variable, it does not seam to support lists.

I have a query that needs to run for each day of a month, so I want to do something like this:

for day in days:
  ALTER TABLE foo ADD PARTITION(${day=hiveconf:day});

Is there a way to do this without! using some kind of scripting language like python/bash.

I have a python implementation, but it need to start a new hive process for each query. Besides being ugly, this has bad performance for small queries because of the hive start-up-time.

Answer 1

There’s not really a great way to do this, and I’m pretty certain there’s no way at all to do it without a secondary scripting language. But it’s not too painful in something like bash:

hive_script=foo.hql
if [ -e "$hive_script" ]
then
    rm $hive_script
fi
days="1 2 3 4 5"
for day in $days
do
    echo "ALTER TABLE foo ADD PARTITION(day=$day);" >> $hive_script
done
hive -f $hive_script
rm $hive_script

Same strategy could be used in python: have the script generate all of the commands you want to execute, writing them to the same file. Then just execute that file.