Question

I have an server setup that consists of 8 machines on which I run a Hadoop job to download certain assets. A client side agent uses JobConf to start the Hadoop jobs.

The agent gets a response code from the server based on which it can determine how the download proceeded. In the below code, invoke() creates a connection to the REST API exposed by the server. ResponseData is a custom class that can read the downloaded data off the connection associated with the response as below:

ResponseData res = invoke(downloadUrl, contentType);
downloadedAssetStream = New ObjectInputStream(res.connection.getInputStream);

if(res.code != 200)
{
    //Stop hadoop job
}

For stopping the job, what do I need? I’ve heard somewhere that throwing an IOException would do, but I haven’t been able to verify it yet.

Answer 1

Job#killJob with the new MR API or RunningJob#killJob with the old MR API will kill the Job.