Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
As the title says, does Hive support cursors, or something like them? I have a large query that I’d like to fetch in chunks, but I can’t seem to find a Hive friendly solution.
Thanks,
Kyle
As far as I know, there is no such thing in Hive. You can not really have pagination using Hive. Only thing you can do is to execute your hive query and get the ResultSet, then iterate through it.
The interesting part is how you want to handle the large results. You don’t normally want to load all the results in memory, instead, you can stream back your query results. For example, if you are write the results into csv, instead of having a big object containing all the query results before you start writing to csv which potentially can use up your memory, you can handle them iteratively on the wire and write in chunk to your csv file.