Question

I’m trying to run the distributed shell example (using a SVN checkout of Hadoop, which is why the version is set to 3.0.0-SNAPSHOT):

 yarn jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \
 -jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \
 org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command whoami

However it does not work:

12/09/03 13:44:37 FATAL distributedshell.Client: Error running CLient
java.lang.reflect.UndeclaredThrowableException
 at org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:128)
 at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:123)
 at org.hadoop.yarn.client.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:163)
 at org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:316)
 at org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:164)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:601)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.getProtocolImpl(ProtobufRpcEngine.java:398)
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:456)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1732)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1728)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1726)

 at org.apache.hadoop.ipc.Client.call(Client.java:1164)
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
 at $Proxy7.getClusterMetrics(Unknown Source)
 at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:121)
 ... 8 more

The essential problem seems to be in the second trace:

Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB

Does anyone know how protocol registration for Hadoops ProtoBufRPC works? Any idea on how to debug?

Edit: With Hadoop version 2.0.1-alpha, it works slightly better.

12/09/03 18:43:14 INFO distributedshell.Client: Application did not finish. YarnState=FAILED, DSFinalStatus=FAILED. Breaking monitoring loop
12/09/03 18:43:14 ERROR distributedshell.Client: Application failed to complete successfully

So maybe my build did not work right. Any ideas of what is causing the problem above (I’d really like to use HEAD, as I’m planning to do some low level experiments, beyond MapReduce)? Or is HEAD partially broken, does distributed shell on HEAD work for you?

My own (not yet working …) client still fails with the same error:

Caused by: java.io.IOException: Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB

Answer 1

It turned out that the main problem with my own code was that I naively instantiated the Configuration class, instead of instantiating YarnConfiguration. This way, the yarn config files were not read, and it tried to contact the servers on their default ports – which don’t agree with my settings.

The same bug seems to be present in the distributedshell example.