Enigmastation.com

I was playing around with a speed test of Gigaspaces XAP, and got some pretty good numbers out of it – but I couldn’t escape the thought that I was missing something. Then I had it: a comparison point!

If GigaSpaces is X, then… it’s X. But if we compare it to something else – a Y, if you will – then we get a better sense of whether it’s actually fast or not.

Timings capture only the time spent in the DAO, as captured by a dynamic proxy. Therefore, a given test might take four minutes, but yield only 60 seconds’ worth of measurements, because none of the non-DAO stuff was measured. The idea was to isolate any testing code away from the actual focus of the speed test, which was: “How fast do read/write/delete operations run under simple transactions, when connecting to external servers?”

The tests are serial – one operation at a time, over and over again. This is way not real-world. I know it, you should know it, if you take these benchmarks as anything other than a “hmm, that’s interesting” data point, a yeti will hunt you down and eat your toes.

So I created a simple 1:M mapping, an Owner to OwnerAttribute model, and created mappings for Hibernate and GigaSpaces. (Same object, just annotated for both GigaSpaces and Hibernate.) The test ran over four basic operations in bulk, using transactions on every operation, based around a common DAO interface.

When I first ran the tests, I was horrified: Hibernate was so freakin’ bleedin’ farglin’ bleepin’ slow SLOW SLOW! Amazingly slow. Horrifyingly slow. I isolated the cause to two reasons: session acquisition and object model. The session acquisition surprised me: I was acquiring one session and using it for all operations, which I thought would be faster.

After all, it takes time to acquire a session, does it not? Boy, was I wrong.

This is why you test your assumptions, people. Having the DAO get a new session from the factory every call sped things up dramatically. Score one for Hibernate, one point lost for me.

The object model was a stickier problem. The test creates a number of attributes for the Owner object and stores them as well.

When I had OwnerAttribute stored as an external record (@OneToMany, etc.) Hibernate crawled.

Crawled bad.

Crawled “Are you reusing the session, Joe?” bad.

So then I thought about the object model. In the real-world situation this models, I don’t ever search for owners by attributes; the actual attributes are more like function call parameters. So there was no reason to have them stored externally, in either the Hibernate or GigaSpaces model.

Therefore I stored them as a Serializable in the Owner class. This simplified the object model by a lot (the DAO for GigaSpaces would have had to compensate for an Owner being an aggregate) and also, well, sped things up for Hibernate.

The tradeoffs: I couldn’t search by attribute (again, not part of the real application), Hibernate sped up by leaps and bounds, and the GigaSpaces DAO implementation was far simpler than it would have been.

What of those is valuable to you is, well, up to you. Personally, I find the DAO implementation more expensive up front, and the speed benefits are worth it.

The tests themselves: 100000 writes, 100000 serial reads (in order by PK), 100000 random reads (Collections.shuffle() on the PKs), 100000 serial deletes (i.e., destructive reads), 100000 random deletes.

Hibernate did way better than I thought it would. Here’s one view of the data:

Now, Hibernate was talking to an external Derby instance; GigaSpaces was talking to an external GigaSpaces container. Some outliers on Hibernate slowed it down a little; the actual perceptive numbers are probably better than you see here.

But this is still really good. I’d expected Hibernate to be in the 6-8 millisecond range, and it was way faster than I thought.

I’m including the source code to the test, if you’re interested. Again, this data isn’t worth making a real conclusion about – you need to test your own object models – but it’s interesting.

Related posts:

1. Commentary on the datastore benchmark
2. Rocket Surgery: What is Inversion of Control?
3. Java schools of thought?
4. New Article: “Considering Datastores.”
5. Modules aren’t expensive enough to avoid them.