Question

#!/bin/bash
export HIVE_OPTS="$HIVE_OPTS -hiveconf mapred.job.queue.name=hdmi-technology"
export HIVE_AUX_JARS_PATH=/home/hadoop/lib/HiveUDF.jar
hive -S -e 'set mapred.job.queue.name=hdmi-technology'
hive -S -e 'SELECT count(*) from testingtable2' > attachment.txt

Whenever I try to run the above shell script(count.sh) like below, I always get errors, I have no idea what wrong I am doing as I am new to shell script and I am not sure how I can add environment variables in shell script.

bash-3.00$ sh count.sh
count.sh: HIVE_OPTS= -hiveconf mapred.job.queue.name=hdmi-technology^M: is not an
identifier

Is there anything wrong I am doing in my shell script, by the way I am adding environment variables in first two lines? Any help will be appreciated.

After all the changes I did as per the below comments,

mv count.sh count, 
chmod +x count, 
./count

whenever I try to do this in my prompt directly export HIVE_AUX_JARS_PATH=/home/hadoop/lib/HiveUDF.jar it works fine, but whenever I try to add this in my shell script as mentioned in my question, I always get ‘java.io.FileNotFoundException(File file:/home/hadoop/lib/HiveUDF.jar does not exist. Why is it so?

Answer 1

You edited your shell script on a DOS or Windows machine, which uses the CRLF pair (\r\n) instead of the Unix-style LF (‘\n’) line endings.

The ^M is the tell-tale — it is a \r carriage return character.

This should fix it:

tr -d '\r' < count.sh > count.sh.fixed
mv count.sh.fixed count.sh

Another option:

sed -i 's/\r//g' count.sh