Best Filesystem for 2TB RAID-5 storing lots of small files?

[feenster99]

Hi,
What filesystem would you recommend for a Server that is storing hundreds of thousands of small files (from 64K to 1MB), nested in 100's of folders, on a 2TB RAID-6 array? The O/S is Ubuntu Server 9.04, and is run from it's own hard drive.

I need to be able to expand the filesystem size without too much downtime, and run filesystem checks etc quickly.

What would you recommend?

Cheers,
Matt

 

[whitecrook]

depends on lots of things.... and you are always best to test.

Also, significant performance tuning occurs in how you lay down the file system rather than the actual filesystem especially since you have a RAID6 (or raid 5 which is it?) array.

You will probably want to adjust your RAID block size, and your stripe size, stride width, block size, and for things like ext3 adjust the inodes correctly.

I rate XFS highly, but there are SERIOUS problems with it in certain situations, notably with vmware and dbs used in some way together.

JFS is rated very highly and both XFS and JFS can expand a live filesystem. Also they can provide snapshotting I think.

EXT3 is well known realiable and trusted but doens't perform as fast and and doesn't have dynamic inode allocation which may cause problems when the file system fills up unless you plan for it.

And 64K to 1MB isn't really what's considered small files. Small files are generaly less than a few K so if you reading guides beware.

Is the RAID array SW raid? hardware raid? hows it configured?

oh and you can optimise for read access, write access, through put, etc etc. It all depends what you are doing.

 

[feenster99]

depends on lots of things.... and you are always best to test.

Also, significant performance tuning occurs in how you lay down the file system rather than the actual filesystem especially since you have a RAID6 (or raid 5 which is it?) array.

I'll probably go for RAID-5 with a hot spare (effectively RAID-6 I guess, but it's marketed slightly differently it seems)

You will probably want to adjust your RAID block size, and your stripe size, stride width, block size, and for things like ext3 adjust the inodes correctly.

How would I go about finding out what I should be setting these values to?

I rate XFS highly, but there are SERIOUS problems with it in certain situations, notably with vmware and dbs used in some way together.

JFS is rated very highly and both XFS and JFS can expand a live filesystem. Also they can provide snapshotting I think.

EXT3 is well known realiable and trusted but doens't perform as fast and and doesn't have dynamic inode allocation which may cause problems when the file system fills up unless you plan for it.

And 64K to 1MB isn't really what's considered small files. Small files are generaly less than a few K so if you reading guides beware.

Thanks for the info - i'll bear that in mind.

Is the RAID array SW raid? hardware raid? hows it configured?

oh and you can optimise for read access, write access, through put, etc etc. It all depends what you are doing.

It will be powered from an Adaptec 2820SA PCI-X Hardware RAID card (8 port SATA). It will be running in an IBM x235 Server (4GB RAM, 3.0GHz Xeon, Ubuntu 9.04 etc). The function of the Server is to run as an off-site backup server for multiple users - their data is backed up to it over the 'net, and then replicated to another Server. Therefore, read/write is fairly balanced. By no means am I after ultra fast - it currently runs off EXT3 and does OK. Just want to make sure I squeeze as much out of it as possible.

Cheers,
Matt

 

[yashiro]

9.04 on a server? Brave man.

 

[whitecrook]

I'll probably go for RAID-5 with a hot spare (effectively RAID-6 I guess, but it's marketed slightly differently it seems)

No, RAID-5 and a hotspare is not RAID 6 no matter what way you look at it.

How would I go about finding out what I should be setting these values to?
Read the relevent documentation for your harddisk, your RAID card, your filesystem etc etc.

e.g. have a look at man mk2ef for a taster of the options just for EXT3

http://linux.die.net/man/8/mke2fs

Have a read of how to set up your RAID card precisely. I've been playing with softraid and dell MD arrays recently, and there are PDFS and wikis and web forums and official whitepapers detailing every single thing you'd like to know about the system. Find the equiv for you system.


It will be powered from an Adaptec 2820SA PCI-X Hardware RAID card (8 port SATA). It will be running in an IBM x235 Server (4GB RAM, 3.0GHz Xeon, Ubuntu 9.04 etc). The function of the Server is to run as an off-site backup server for multiple users - their data is backed up to it over the 'net, and then replicated to another Server. Therefore, read/write is fairly balanced. By no means am I after ultra fast - it currently runs off EXT3 and does OK. Just want to make sure I squeeze as much out of it as possible.




Cheers,
Matt

Hmmm "I squeeze as much out of it as possible" why?

It's 'just' a back up machine unless your net connection transfers 50-100MB a second you really have no worries.... Is time to take data off onto the other system an issue/likely to be an issue?

And 9.04 on a production server....

Also, at the end of the day what does it matter - just get is working stably and solidly. It's not worth risking your ass being handed to you when it goes down for the sake of speed or whatever... 9 times out of 10 the defaults are perfectly fine.

 

[feenster99]

Fair enough, will have a further read up but will go with the defaults. And, I've been running 9.04 on two other servers since it's release and have had no issues whatsoever, so i'm happy to go with it.