Thursday, 28 January 2010

Distributed caching with protobuf-net

No, this isn't my server farm.

I’ve been playing with distributed caches lately, and of course this is a really good use-case for something like protobuf-net – if there is anywhere you want to minimise your memory costs, network IO costs and CPU costs, it is your cache (it gets kinda busy).

As a pure example of “which system can I get installed and running the quickest”, I looked at memcached, using the windows service from here and the enyim.com client. Not only were they a breeze to get working, but this setup seems pretty popular, and also covers memcached providers, which uses enyim under the hood.

Fortunately, the enyim client has an extension-point that allows you to nominate a serializer through your config file (actually it calls it a transcoder, but I’m not sure it fits the definition). This allows us to swap BinaryFormatter for protobuf-net, which (for suitable types) has huge savings – in my crude tests I was seeing only 20% of the original network traffic, and roughly 75% of the original CPU costs.

Enough bluster…

OK, so in r282 I’ve added just such a transcoder (I still don’t like the name…) – and the good thing it is so effortless to configure it:

Before:

<enyim.com>
<memcached>
<servers>
<add address="127.0.0.1" port="11211" />

After:

<enyim.com>
<memcached transcoder="ProtoBuf.Caching.Enyim.NetTranscoder, protobuf-net.Extensions">
<servers>
<add address="127.0.0.1" port="11211" />

Wasn't that was painless? and some huge benefits (although obviously you’d want to run some integration/regression tests if applying this to an existing system). Oh, and of course you need to include protobuf-net and protobuf-net.Extensions.dll in your project, in addition to the enyim dll.

My only niggle is that because there is no way of knowing the type up-front, I had to include the outermost type information on the wire (hence the “Net” in the transcoder’s name, since it won’t be truly portable). I could get around this by having the caller indicate the type, but the enyim dll doesn’t play that game.

What next?

If you find it good/bad/pointless/buggy please let me know.

If you’ve got another cache that might benefit from some protobuf-net love, please let me know. Velocity, perhaps? Azure?

18 comments:

Mythz said...

Hey Marc, good job on the client.

Just want to know if its possible in hooking up the ProtoBuf formatter in code rather than .config so I can take out 1 less moving part?

Marc Gravell said...

@Mythz - are you using the "enyim" client directly? Or via the "providers" code? If done directly there is an interface you can implement and pass to the client-constructor (the interface provides the transcoder). If "providers", I'd need to check how it creates the client.

mgmtdocs said...

Hi Marc,

Distributed caching really impresses me a lot because it is the best way to overcome the performance and scalability issues. We are using a third party distributed .NET caching product called NCacheand we really getting benefits form it. I’ll recommend this caching solution because it has a very rich range of topologies to offer. It also provides appliance for Memcached as well which makes it the ultimate choice as far as distributed caching is concerned.

Jader Dias said...

I recompiled protobuf-net.Extensions because it references Enyim.Caching 1.2.0.0 and only 1.2.0.8 is available in the Enyim's page.

Then I tried to use your transcoder but the data that went to memcached was still encoded with the default encoder:

OMyCompany.MyNameSpace, Version=1.0.0.0, Culture=neutral, PublicKeyToken=nullMyCompany.MyNameSpace.MyClassk__BackingFieldk__8¥$RĂ‚?gFieldk__BackingFieldk__BackingField

Jader Dias said...

Or maybe it did work indeed, but the text part of the output is very similar to the default encoder.

Jader Dias said...

There is a library called ServiceStack that when using Redis (another memcached-like cache) it inserts the type information in a separate variable so it doesn't have to insert it in every variable.
So you could steal this idea if it would improve your library performance.

Marc Gravell said...

Yes, I'm aware of ServiceStack - I've spoken to the dev etc. I have some plans to look at this after protobuf-net "v2" finishes.

Mythz said...

I'm sure @Marc knows exactly what needs to be done in order to get max perf in protobuf-net!

Well looks like there is a lot of people wanting to get there hands on protobuf-net v2 when its finally ready :) Hopefully not too long to go now, when should we expect to get the first RC?

Marc Gravell said...

I had actually hoped to get it done by now, but I've had some... distractions. Changed jobs (twice, actually) in the last few weeks. I should be able to announce my new awesome job next week.

But I am still progressing this. Honest.

Mythz said...

Yep, heard about the change in your first job (which was a while ago) and save on the commute (which hopefully would've given you enough free time to get v2 ready :).

If you've actually got a new job after that, then judging by the excitement, I can only guess that you're joining your Stack Overflow buddy Jon Skeet over at Google? If that's the case it sounds like bitter sweet, i.e. Google == Good && Java == Bad :)

Jader Dias said...

From the sample data that I posted in my first comment, what serializer do you think I am using? The default one, or maybe it's yours?

Marc Gravell said...

That doesn't scream protobuf-net to me. Is your take marked as a contract? (proto-contract/data-contract/xml-type)?

And no, not Google. Nor Microsoft, Apple, Sun, Novell.

Jader Dias said...

Thank you! I marked it as Serializable, I'll change that.

Tom said...

Any ideas if the extensions dll works with v2? I'll give it a shot if there's a chance they'll work together.

Marc Gravell said...

@Tom there is no fundamental reason it *shouldn't*, although I don't have a memcached setup at the moment

Anonymous said...

Thank you for this. Had to do a bit of tweaking to get it to work on azure - but yeah - awesome.

Spikyz said...

Humm I install via NuGet and don't see a "ProtoBuf.Caching" namespace. Did I mss it? Thanks!

Gaurav Gat said...

Hey Marc, I am trying to hook up protobuf for Azure Caching(preview). Any Idea how this can be achieved?