A redis backend for riak.

Awesome + Awesome = AWESOMER

Riak is AWESOME.

redis is AWESOME.

Why not marry them and keep the babies ?

The riak_redis_backend is exactly the point. Data stuffed into riak is stored on a redis running on localhost on the standard port. (Having non-local data would kind of miss the point, right ?)
An added trick is that the redis database index is different for each riak nodes — if you happen to run all your Riak nodes on the same machine.

Warnings :

Implementation sucks big time, and though it passes the riak_test_util:standard_backend_test, I made it timeout on some MapRed jobs. And MapRed tasks also runs slower than dets ones.

[Update] performance has been improved by using a better encoding (meaning I removed that base64 horror … fprof told me that). My test code went down from 25 seconds to 15 seconds.

Can it be useful — one day ?

Not sure. Given that redis cannot currently have a dataset bigger than memory (but plans to have virtual memory), it kind of limit the data size.

Performance is currently worse than riak_dets_backend, but given that Redis is *fast*, one can hope for enhancements. I think that most of the improvement should come from the erldis library which could need some tweaking for better handling binaries.

Where is it ?

It’s over there !

How can I use it

Install the erldis (binary branch) application in your lib/ Riak node directory.
Compile and hurl in the riak/ebin directory. Restart riak node, make sure redis is running, and off you go.

3 comments so far

  1. Jacob on

    Have you run riak_redis_backend thru fprof? Great tool for identifying performance bottlenecks. Let me know if there’s anything I can do to speed up erldis.

  2. cstar on

    >>Have you run riak_redis_backend thru fprof?

    No, but I will be doing it right away😉

    >> Let me know if there’s anything I can do to speed up erldis.

    I *think* moving to binaries could be worth it, I’ll see if I can find some time tonight
    Anyway I think most of the issues lie in:
    – riak_redis_backend.erl
    – my understanding of Riak and how it hits storage.

  3. Hermelinda on

    Κeep this going ρlease, great job!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: