Mark's Blog

For a project of mine I have been looking today on key/value stores. Redis, Tokyo Cabinet and Memcache are the popular ones, but I have been interested in benchmarking the distributed species of its kind (“DKVS”). Those are needed in high scalable systems as replacement for traditional databases, which happen not to scale well.

These are the criteria which the DKVS must meet:

1. distribute data amongst one to four nodes (and the more the better)
2. nodes must be able to be added on-the-fly,
3. … and removed on-the-fly
4. data has to be held redundantly, even after removal of a node
5. data access has to be transparent, i.e. every node shall be equal
6. scale with additional nodes – for every additional node there should be an increase of GET and SET operations per second up to the maximum dictated by the hardware and network

Read more »

Tags: distributed system, key/value store, nosql, systems architecture

Amazon and Rackspace both provide virtualized servers billed per hour. That makes deploying scalable applications, especially with distributed key/value stores, very handy: You could launch new servers just as load, customer count or requests increase.

Today I’ve run the Redis benchmark on the SheevaPlug, Amazon EC2 and Rackspace Cloudservers to see where a small Redis database will perform the best. Read more »

Tags: Amazon EC2, benchmark, cloud, Rackspace CloudServers, Redis, SheevaPlug

Unlike Apache httpd server Nginx has no ebuilds which let you select which modules you want to have installed with it. Yet.

On last Saturday I have written an eclass to simplify maintaining Nginx ebuilds and added a new Portage variable NGINX_MODULES to USE_EXPAND. In other words, from that on you can select Nginx modules like for Apache. As an example, Read more »

Tags: ebuild, Gentoo, know-how, nginx, ossdl overlay

The nice thing about Gentoo is that you can extend its port-like “portage” system by third-party trees. They are called ‘overlays‘ to the official main tree. I have started mine about four years ago after having used Gentoo for more than three years, then. A lot has changed, but something is still missing – and that’s my OSSDL overlay meant for.

The overlay has ebuilds for Read more »

Tags: ebuild, Gentoo, GNU/Linux, ossdl overlay, portage