[vox-tech] Performance tuning for http file serving

[Bill Broadley]

On 04/14/2010 11:23 PM, Brian Lavender wrote:
>> What gets ugly is if you have 2 or more clients accessing 2 or more
>> files.  Suddenly it becomes very very important to intelligently handle
>> your I/O.  Say you have 4 clients, reading 4 ISO files, and a relatively
>> stupid/straight forward I/O system.  Say you read 4KB from each file (in
>> user space), then do a send.
>
> That's like operating a disk in PIO mode. I would think something that
> can leverage the DMA controller better would avoid thrashing.

DMA controllers make it easier on the kernel, but it doesn't change the 
nature of random I/O limiting performance.

> Perhaps
> there is a way to tell apache to use memory cache and dump readaheads
> into memory. Linux should be able to handle this fine.

I didn't see anything.

> I was looking to see if there is a config on this, but I can't find one
> at the moment. I was looking at this cache page.
>
> http://httpd.apache.org/docs/2.2/mod/mod_mem_cache.html

Yeah, I looked around, it seemed to be rather focussed on having objects 
smaller than cache.

If you want to quantify the problem I'd just download a 4GB file and 
measure the bandwidth.  Then access enough files so that they sum to 
twice as big as ram.  Say 4 4GB files if you have 8GB ram, or 8 4GB 
files if you have 16.  Launch a client each, then when they get half way 
done fire another download for each file.

If you are saturating the network, or at least within 75% of so it 
sounds like linux is prefetching and/or using a large enough transfer 
size so that random I/O isn't killing performance.

If you are at 5% or less then I'd continue looking for some tuning.

> I imagine someone else has encountered this problem before.

I seem to recall some GigE connected freebsd box that was one of the 
busiest ftp servers in the world working on this kind of thing.  I saw a 
few published articles on the software, hardware, and kernel tuning.  I 
look around for apache tuning for large file transfers and didn't really 
find much.  Maybe it's not a problem.


>
> brian