Home/ Questions/Q 6983309
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-27T18:24:08+00:00

I am setting up a test environment using apache accumulo ver 1.4.0, hadoop ver

  • 0

I am setting up a test environment using apache accumulo ver 1.4.0, hadoop ver 0.20.2 and zookeeper ver 3.3.3. Please see below for the issue.

Hadoop and Zookeeper work great together, but when I start accumulo using the procedures on apache incubator I get the following zookeeper stream of info and a warn:

2011-12-08 20:13:56,601 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/zookeeper-3.3.3/bin/../conf/zoo.cfg
2011-12-08 20:13:56,603 - WARN  [main:QuorumPeerMain@105] - Either no config or no quorum defined in config, running  in standalone mode
2011-12-08 20:13:56,616 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/zookeeper-3.3.3/bin/../conf/zoo.cfg
2011-12-08 20:13:56,617 - INFO  [main:ZooKeeperServerMain@94] - Starting server
2011-12-08 20:13:56,626 - INFO  [main:Environment@97] - Server environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT
2011-12-08 20:13:56,627 - INFO  [main:Environment@97] - Server environment:host.name.paz
2011-12-08 20:13:56,627 - INFO  [main:Environment@97] - Server environment:java.version=1.6.0_26
2011-12-08 20:13:56,628 - INFO  [main:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
2011-12-08 20:13:56,629 - INFO  [main:Environment@97] - Server environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.26/jre
2011-12-08 20:13:56,629 - INFO  [main:Environment@97] - Server environment:java.class.path=/home/hadoop/zookeeper-3.3.3/bin/../build/classes:/home/hadoop/zookeeper-3.3.
3/bin/../build/lib/*.jar:/home/hadoop/zookeeper-3.3.3/bin/../zookeeper-3.3.3.jar:/home/hadoop/zookeeper-3.3.3/bin/../lib/log4j-1.2.15.jar:/home/hadoop/zookeeper-3.3.3/b
in/../lib/jline-0.9.94.jar:/home/hadoop/zookeeper-3.3.3/bin/../src/java/lib/*.jar:/home/hadoop/zookeeper-3.3.3/bin/../conf:
2011-12-08 20:13:56,630 - INFO  [main:Environment@97] - Server environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/i386/server:/usr/lib/jvm/java-6-su
n-1.6.0.26/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.26/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2011-12-08 20:13:56,630 - INFO  [main:Environment@97] - Server environment:java.io.tmpdir=/tmp
2011-12-08 20:13:56,631 - INFO  [main:Environment@97] - Server environment:java.compiler=<NA>
2011-12-08 20:13:56,631 - INFO  [main:Environment@97] - Server environment:os.name=Linux
2011-12-08 20:13:56,632 - INFO  [main:Environment@97] - Server environment:os.arch=i386
2011-12-08 20:13:56,633 - INFO  [main:Environment@97] - Server environment:os.version=3.0.0-13-generic
2011-12-08 20:13:56,633 - INFO  [main:Environment@97] - Server environment:user.name=hadoop
2011-12-08 20:13:56,634 - INFO  [main:Environment@97] - Server environment:user.home=/home/hadoop
2011-12-08 20:13:56,634 - INFO  [main:Environment@97] - Server environment:user.dir=/home/hadoop
2011-12-08 20:13:56,641 - INFO  [main:ZooKeeperServer@663] - tickTime set to 2000
2011-12-08 20:13:56,641 - INFO  [main:ZooKeeperServer@672] - minSessionTimeout set to -1
2011-12-08 20:13:56,642 - INFO  [main:ZooKeeperServer@681] - maxSessionTimeout set to -1
2011-12-08 20:13:56,661 - INFO  [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
2011-12-08 20:13:56,691 - INFO  [main:FileSnap@82] - Reading snapshot /home/hadoop/zoo/dataDir/version-2/snapshot.0
2011-12-08 20:13:56,708 - INFO  [main:FileTxnSnapLog@208] - Snapshotting: 4e
2011-12-08 20:14:52,147 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:40694
2011-12-08 20:14:52,153 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /0:0:0:0:0:0:0:1:40694
2011-12-08 20:14:52,154 - INFO  [SyncThread:0:FileTxnLog@197] - Creating new log file: log.4f
2011-12-08 20:14:52,410 - INFO  [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13420623ee70000 with negotiated timeout 30000 for client /0:0:0:0:0:0:0:1:4069
4
2011-12-08 20:14:52,959 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:38446
2011-12-08 20:14:52,962 - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@777] - Client attempting to establish new session at /127.0.0.1:38446
2011-12-08 20:14:53,007 - INFO  [SyncThread:0:NIOServerCnxn@1580] - Established session 0x13420623ee70001 with negotiated timeout 30000 for client /127.0.0.1:38446
2011-12-08 20:14:59,932 - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@634] - EndOfStreamException: Unable to read additional data from client session
id 0x13420623ee70000, likely client has closed socket

and when I start the accumulo shell I get the following errors:

18 12:44:38,746 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:38,846 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:38,947 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,048 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,148 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,249 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,350 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
18 12:44:39,450 [impl.ServerClient] WARN : Failed to find an available server in the list of servers: []
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Corrected tserver memory settings to not exceed waht is allowed by the JVM. Tserver does not crash and error resolved.
    Answer came from accumulo-incubator user list and is reposted at the bottom.
    Basically, when going in and modifying memory settings to run in pseudo-distributed mode on a laptop I made incorrect modifications to the accumulo-site.xml and accumulo-env.sh files concerning the tablet server memory usage. The clue to the error was found in the /home/hadoop/accumulo/logs/tserver*.log file:

    20 18:20:00,951 [tabletserver.NativeMap] ERROR: Failed to load native
map library
/home/hadoop/accumulo/lib/native/map/libNativeMap-Linux-i386-32.so
java.lang.UnsatisfiedLinkError: Can't load library:
/home/hadoop/accumulo/lib/native/map/libNativeMap-Linux-i386-32.so
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1706)
at java.lang.Runtime.load0(Runtime.java:770)
at java.lang.System.load(System.java:1003)
at
org.apache.accumulo.server.tabletserver.NativeMap.loadNativeLib(NativeMap.java:144)
at
org.apache.accumulo.server.tabletserver.NativeMap.<clinit>(NativeMap.java:156)
at
org.apache.accumulo.server.tabletserver.TabletServerResourceManager.<init>(TabletServerResourceManager.java:123)
at
org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2959)
at
org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3085)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.accumulo.start.Main$1.run(Main.java:89)
at java.lang.Thread.run(Thread.java:662)
20 18:20:00,999 [tabletserver.TabletServer] ERROR: Uncaught exception in
TabletServer.main, exiting
java.lang.IllegalArgumentException: Maximum tablet server map memory
134,217,728 and block cache sizes 186,646,528 is too large for this JVM
configuration 132,579,328
at
org.apache.accumulo.server.tabletserver.TabletServerResourceManager.<init>(TabletServerResourceManager.java:134)
at
org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2959)
at
org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3085)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.accumulo.start.Main$1.run(Main.java:89)
at java.lang.Thread.run(Thread.java:662)

    Specific help text:
    “The native libraries are not loading, which is shifting in-memory
    map into the java workspace. Add to that your block cache size, and your
    specifications for memory use are higher than the JVM will be allowed to
    allocate. The tablet servers complain, and exit. You should see these
    complaints on the accumulo monitor web pages.

    You may find a benefit to rebuilding the native map library, which will
    move the allocation of the in-memory map to outside the JVM. This is
    not required.

    The size of any memory dedicated to cache must be smaller than the size
    of the JVM, which must include substantial working space for RPC calls
    and garbage collection over time.”

    • 0