SUMMARY: advise wanted for multi-terabytes filesystems

From: Rob De Langhe <>
Date: Tue Apr 27 2004 - 02:46:02 EDT
Needless to say that there were again numerous valuable replies ...
Thx to Grzegorz Bakalarski, Octave Orgeron, Steve Starer, Alex Madden,
Przemol, and specially Dik Casper who mentioned the most important aspect :
at creation-time of a filesystem (with Solaris' "newfs") that is intended to
grow past the 1TB limit, use the "-T" option to allow it to grow as such.

Opinions were generally ok with using Solaris-9's UFS type of filesystem,
creating it with large blocks-per-inode ("newfs"-option "-i", set to
multi-megabyte size), and using the logging feature (mount-option "logging")
to accelerate/avoid any FSCK.

Altough maybe technically possible, I guess we will follow anyhow some
people's advice to keep not everything on a single filesystem but store the
Oracle data-files over multiple approx-1TB filesystems.

Several people suggested using Veritas' VxFS, but to us that's no added
value compared to the features already present in Solaris-9, and above all
it's quite expensive as well. If anyone can still point me to an objective
performance-comparison between VxFS and (Solaris-)UFS, that would be very
Someone also mentioned the possibility to bypass the filesystem-layer, and
let Oracle use the raw disk access method available in Oracle-9. Again, if
someone can present objective performance-comparisons, would be very nice.

Performance-wise we have the LUNs set up as RAID-5 sets over 10 disks for
each LUN (giving good striped performance while reading, sure we know it's
slower during writes bcos of the RAID-5 operations), and the LUNs are
balanced over the two controllers in the EMC array, each controller being
connected to a separate HBA in the server using FCAL. At this moment, that
renders service-times of 1-4 msec for 200-800 KB read/sec plus 300-2000 KB
written/sec, the CPU (SF480R, 2x900MHz) meanwhile busy for 20% of its time
blocked on I/O.
Is this good, or could it be (much) better ?


-----Original Message-----
From: Rob De Langhe 
Sent: Friday, April 23, 2004 9:17 AM
To: ''
Subject: advise wanted for multi-terabytes filesystems

we have a pair of (Solaris-9) Oracle-servers connected via FCAL to EMC
storage, with RAID-5 LUNs from that storage array managed by Veritas Volume
Manager on the hosts.
On these LUNs, we have create filesystems to hold the Oracle files. One of
them, that contains the actual data-files, is mounted as "/oraAdb" with some
subdirectories for each database instance :
and so on
This filesystem is created using plain Solaris-command ("newfs -i 65536 -f
8192 /dev/vx/rdsk/..."), so no Veritas-Filesystem is used. The option
"logging" is used in "/etc/vfstab" to have it journalling its changes.
Since our database will grow from (currently) 1 TB to approx 14 TB, I was
wondering how I should keep this stable : is it recommended/safe practice to
keep all the Oracle-DBF files on a single, therefor multi-terabyte
filesystem (mount on "/oraAdb") ? What if the server crashes suddenly : is
it sufficient/safe to have the "logging" option so that a FSCK will not take
days to complete ? Are there any Solaris limits we will hit when growing
this filesystem to is max capacity ?
Or should we split the Oracle-DBF files over multiple -say 1TB capacity-
filesystems, and just add more of these filesystems ?
TIA for any suggestions and/or experiences.

Visit us at the Telecom cITy Fair - The largest IT Fair in Belgium! 25, 26,
27 May - Brussels Expo

Get your free tickets here! _______________________________________________
sunmanagers mailing list

Visit us at the Telecom cITy Fair - The largest IT Fair in Belgium!
25, 26, 27 May - Brussels Expo

Get your free tickets here!
sunmanagers mailing list
Received on Tue Apr 27 02:45:58 2004

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:43:33 EST