[vox-tech] Clusters, renderfarms and MySQL, LAN projects using more than one Linux at a time

[Bill Broadley]

> Processing times can be improved with data structures. I'm not sure
> storing MySQL tables across
> multiple machines will speed this up. The data travels across your
> motherboard's bus
> much faster than it will travel across the network and then your
> motherboard's bus.
> But as you said, it's a excuse to try clustering.

Last I checked a mysql cluster has to fit in ram, but that may have
changed.  For better scaling I'd suggest NoSQL key/value stores if that
fits your need.  Lugod has a talk on RIAK not too long ago.  MongoDB is
also common.

> I've never implemented a cluster, however, because running MySQL on a
> single machine has always
> fulfilled my needs.

Certainly I'd recommend tuning mysql for a single machine before trying
to scale it larger.

> The only tweak I've needed is to store a 300 million
> record table
> on another machine through NFS. And short of a complete backup,

It's inherently unsafe to use mysql to access tables over NFS.  I guess
it could be ok if it's read only.

As for building your own cluster I'd suggest starting with any of the
popular cluster distributions, or if you want to do it yourself start
with your favorite linux distribution and then add cobbler and puppet.

I suggest slurm, if you want a batch queue to manage resources for you.