Question

I have installed ElasticSearch on Mac OS X using Homebrew. It works. The cluster started off with “green” health. However, right after adding data, it has gone to “yellow”.

The cluster health is status is: green, yellow or red. On the shard level, a red status indicates that the specific shard is not allocated in the cluster, yellow means that the primary shard is allocated but replicas are not, and green means that all shards are allocated. The index level status is controlled by the worst shard status. The cluster status is controlled by the worst index status.

So, my replica shards are not allocated. How do I allocate them? (I’m thinking out loud.)

According to Shay on “I keep getting cluster health status of Yellow”: “the shard allocation mechanism does not allocate a shard and its replica on the same node, though it does allocate different shards on the same node. So, you will need two nodes to get cluster state of green.”

So, I need to start up a second node. I did this by:

cd ~/Library/LaunchAgents/
cp homebrew.mxcl.elasticsearch.plist homebrew.mxcl.elasticsearch-2.plist
# change line 8 to: homebrew.mxcl.elasticsearch-2
launchctl load -wF ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch-2.plist

Now I have “Korvus” on http://localhost:9200/ and “Iron Monger” on http://localhost:9201/. Woot. But, I don’t see any indications that they know about each other. How do I connect / introduce them to each other?

Note: I read Zen Discovery, but do not feel enlightened yet.

Update 2012-08-13 11:30 PM EST:

Here are my two nodes:

curl "http://localhost:9200/_cluster/health?pretty=true"
{
  "cluster_name" : "elasticsearch_david",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

curl "http://localhost:9201/_cluster/health?pretty=true"
{
  "cluster_name" : "elasticsearch_david",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

Update 2012-08-13 11:35 PM EST:

To clarify, my question is not how to “ignore” the problem by setting index.number_of_replicas: 0. I want multiple nodes, connected.

Update 2012-08-13 11:48 PM EST:

I just posted a double gist that includes elasticsearch.yml and elasticsearch_david.log. It looks to me like both nodes are calling themselves ‘master’. Is that what I should expect?

Update 2012-08-14 12:36 AM EST:

And the novel continues! 🙂 If I disconnect my Mac from all external networks, and then restart the nodes, then they find each other. Double Woot. This makes me think that the problem is with my network/multicast configuration. Currently I have this in my config: network.host: 127.0.0.1. Perhaps this is not correct?

Answer 1

Solved. Do not use network.host: 127.0.0.1. Leave that line commented out to let it be automatically derived.

The default elasticsearch.yml was correct. A configuration tweak by the Homebrew installer specifies the 127.0.0.1 loopback interface:

# Set up ElasticSearch for local development:
inreplace "#{prefix}/config/elasticsearch.yml" do |s|
  # ...
  # 3. Bind to loopback IP for laptops roaming different networks
  s.gsub! /#\s*network\.host\: [^\n]+/, "network.host: 127.0.0.1"
end

I filed an issue on the Homebrew issue tracker.