Question

Is there a programmatic way to validate HiveQL statements for errors like basic syntax mistakes? I’d like to check statements before sending them off to Elastic Map Reduce in order to save debugging time.

Answer 1

Yes there is!

It’s pretty easy actually.

Steps:

1. Get a hive thrift client in your language.

I’m in ruby so I use this wrapper – https://github.com/forward/rbhive (gem install rbhive)

If you’re not in ruby, you can download the hive source and run thrift on the included thrift configuration files to generate client code in most languages.

2. Connect to hive on port 10001 and run a describe query

In ruby this looks like this:

RBHive.connect(host, port) do |connection|
    connection.fetch("describe select * from categories limit 10")
end

If the query is invalid the client will throw an exception with details of why the syntax is invalid. Describe will return you a query tree if the syntax IS valid (which you can ignore in this case)

Hope that helps.