Java Riak Connection Problems

Why does Riak log inavailable channel closed message to ERROR

Looks like the developers noticed the same thing, because this bug was filed in June of last year, and and the message was reduced to INFO level in this PR a month later.

If you're still seeing them as errors, update your java-riak-client to v2.0.7 or later.

java.io.EOFException while retrieving data from Riak

Thank you so much for all your help.

I have found the problem!

My port # was 8087 instead of 8098

We need to configure that in /etc/riak/app.conf (Under API settings)

I have reconfigured that and It fixed it.

Thank you!

Riak simple query doesn't return data - Riak Java Client

Since you've already got Riak up and running, you just need to get search going:

First make sure you have search enabled in your app.config file on every node in your cluster:

{riak_search, [
               {enabled, true}
              ]},

If you changed that you will need to restart Riak for it to take effect.

Then from the command line, install the search hook on the bucket you want indexed:

# search-cmd install testbucket
 :: Installing Riak Search <--> KV hook on bucket 'testbucket'.

At this point if there is already data in the bucket, it will not be indexed. You will need to re-put any pre-existing data that you want indexed.

For a quick demonstration, I created 3 keys, creatively named 1,2, and 3; each containing a simple json object:

curl localhost:8098/buckets/testbucket/keys/1 -H "content-type: application/json" -XPUT \
 -d '{"firstName":"Tom", "color":"red"}'
curl localhost:8098/buckets/testbucket/keys/2 -H "content-type: application/json" -XPUT \
 -d '{"firstName":"Dick", "color":"green"}'
curl localhost:8098/buckets/testbucket/keys/3 -H "content-type: application/json" -XPUT \
 -d '{"firstName":"Harry", "color":"blue"}'

I can then query search to find the keys:

# curl http://localhost:8098/solr/testbucket/select\?q=firstName:Harry         
<?xml version="1.0" encoding="UTF-8"?>
<response>
  <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">1</int>
    <lst name="params">
      <str name="indent">on</str>
      <str name="start">0</str>
      <str name="q">firstName:Harry</str>
      <str name="q.op">or</str>
      <str name="filter"></str>
      <str name="df">value</str>
      <str name="wt">standard</str>
      <str name="version">1.1</str>
      <str name="rows">1</str>
    </lst>
  </lst>
  <result name="response" numFound="1" start="0" maxScore="0.353553">
    <doc>
      <str name="id">3
      </str>
      <str name="color">blue
      </str>
      <str name="firstName">Harry
      </str>
    </doc>
  </result>
</response>

# curl http://localhost:8098/solr/testbucket/select\?q=color:red%20or%20firstName:Harry
<?xml version="1.0" encoding="UTF-8"?>
<response>
  <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">2</int>
    <lst name="params">
      <str name="indent">on</str>
      <str name="start">0</str>
      <str name="q">color:red or firstName:Harry</str>
      <str name="q.op">or</str>
      <str name="filter"></str>
      <str name="df">value</str>
      <str name="wt">standard</str>
      <str name="version">1.1</str>
      <str name="rows">2</str>
    </lst>
  </lst>
  <result name="response" numFound="2" start="0" maxScore="0.143844">
    <doc>
      <str name="id">1
      </str>
      <str name="color">red
      </str>
      <str name="firstName">Tom
      </str>
    </doc>
    <doc>
      <str name="id">3
      </str>
      <str name="color">blue
      </str>
      <str name="firstName">Harry
      </str>
    </doc>
  </result>
</response>

I don't have a Scala install handy to whip up an example, but this should get you going in the right direction.

In case you haven't already seen them, the search docs are here:

http://docs.basho.com/riak/latest/dev/using/search/

Riak Java HTTPClientAdapter TCP CLOSE_WAIT

So it's been almost a year, so I thought I would go ahead and post how I solved this problem.

The solution was to change the client implementation we were using to the HTTPClientAdapter provided by the java client, passing in the configuration to implement pools and max connections. Here's some code example of how to do it.

First, we are on an older version of RIAK, so here's the amven dependency:

<dependency>
    <groupId>com.basho.riak</groupId>
    <artifactId>riak-client</artifactId>
    <version>1.1.4</version>
</dependency>

And here's the example:

public RawClient riakClient(){

    RiakConfig config = new RiakConfig(riakUrl);
    //httpConnectionsTimetolive is in seconds, but timeout is in milliseconds
    config.setTimeout(30000);
    config.setUrl("http://myriakurl/);
    config.setMaxConnections(100);//Or whatever value you need

    RiakClient client = new RiakClient(riakConfig);

    return new HTTPClientAdapter(client);
}

I actually broke that up a bit in my implementation and used Spring to inject values; I just wanted to show a simplified example for it.

By setting the timeout to something less than the standard five minutes, the system will not hang to the connections for too long (so, 5 minutes + whatever you set the timeout to) which causes the connectiosn to enter the close_wait status sooner.

And of course setting the max connections in the pool prevents the application from opening up 10's of thousands of connections.

How to understand this Riak stacktrace?

The stack trace is telling us that there was a case clause exception at line 180 of the fie riak_kv_wm_mapred.erl

The clause at that line is handling the responses for the pipe processing the map phase, which appears to be returning the error preflist_exhausted, which is not explicitly handled by the case statement.

That error usually indicates that one or more vnodes were overloaded or otherwise unavailable, and fallbacks had not yet started to take over their workload.

The affected partition was 913438523331814323877303020447676887284957839360, the console.log and error.log may have further details about what happened.

---

Related Topics