Hi! (my first post :x)
I haven't been using Linux for very long (I'm doing an IT administration course, a large component of the Linux subject is using the command-line) and I'm wondering what would be the best filesystem to store all the images I download off the image boards.
At the moment my Linux box is using Ubuntu (Breezy) on a 40GB HDD with the ext3 filesystem.
I have a samba share to my windows box (which I use more often, Linux is primarily a learning environment + file server) which I've recently been saving the images I find on these image boards + 4chan.
I read that ReiserFS is superior over ext3 with smaller files which is why it's used on fileservers, is this true with images around ~100kb?
I'm thinking of having a 5GB partition for images and the rest for downloading bigger files (movies, demos etc) which would use ext3.
Any suggestions?
ReiserFS isn't a bad choice... but it's not nearly as reliable as EXT3. Especially on non-x86 architectures (although I'm guessing this is a moot point).
XFS is another option -- designed for high performance on particularly large arrays (>100GB).
Well I was thinking of building a fileserver in the near future (3 x300gb SATA). What would be good setup for that, if I wanted around 50gb for pictures and the rest for movies/big stuff?
Or could I have software RAID with XFS for a really big single logical drive that would store everything?
I'm not too imaginative with how RAID + mount points and Linux-based partitioning can work compared to Windows :[
XFS works best with large files. ~100KB won't really take advantage of XFS's strengths, unless something has changed the past year.
As far as I know, there haven't been any Resier3 problems for a few years now. Apparently it's quite good at massive directories, which might come in handy...
i store 2.25mil images... on ext3...
and i don't seriously believe the filesystem in use makes any difference (for pretty much anything)
the most important part is to ensure a sane number of files/directory for efficient access
It's worth noting that EXT3's access times can be greatly improved if you specify -O dir_index when running mke2fs to create the partition.
This tells mke2fs to create a hashed B-tree index for directories, and boosts it to almost the performance of ReiserFS. (which uses B*-trees)
Few tips from another random guy interested in medium scale image archives:
First of all, if you have only 5gB of files, it doesn't really matter what fs you use, and you really shouldn't notice any significant difference. (unless it's 5 gigs of 5kB thumbs on 4kB/block filesystem).
Creating a small separate partition could avoid some fragmentation problems, but there are also some drawbacks. Do you have any idea on how fast will your storage grow? Because once it hits 5gB you'll have to resize that partition or copy everything to a new one. Heck, if you really want few files apart on some "partition" just to avoid fragmentation, create a 5gB file containing desired filesystem and loop-mount it. But note that *nix filesystems are not as retarded as NTFS, and as long as there's enough space on your drive they don't get fragmented much. (on a side note, I use the loop-mount method for gentoo's portage, that is 110993 files containing 109055kB of data and taking 469248kB of disk space if used with 4kB block size or exactly the half of it on 1kB or 2kB block size, wich is rather an extreme case)
As for your software raid idea: raid it cool, makes your penis bigger and attract girls, but:
So, if you feel like having one really big partition or want some "easy resizing", just take look on LVM. But unless you have 10+ drives, I don't think you'll need something like that. It's way better to keep your drives separate.
as for >>5 , I'm pretty sure my access times on reiser partitions are very comparable to his on ext3 (we use the same storage system, the only difference is that I don't have 2mil files :p)