SUMMARY(ish) newfs problems with -b -f on sun4u to avoid fragment ation iss ues

From: McCabe, Keith (London) (mccabkei@MLE.CO.UK)
Date: Tue Nov 24 1998 - 04:50:30 CST


Firstly, many thanks to:

nobroin@sced.esoc.esa.de
Casper Dik [casper@holland.Sun.COM]
Toens Bueker [toens@highway.bertelsmann.de]
Craig Bevins [craigb@bitcom.net.au]
Dennis Martens [MARTENSD@health.qld.gov.au]

I'll start by providing the answer to the question I asked which came from
Casper. Which was why won't newfs accept '-f 512' as an option.

> The minimum blocksize is related to the pagesize on the Ultras.
> (The code to load two blocks for one page isn't there)

> The bit masks used in UFS don't allow for a difference of more
> than a factor 8 between blocksize and fragment size.

So minimum block size on the Ultra is 8k and therefore minimum fragment size
is 1k. This is clearly not what the newfs man page says but hey - since when
did RTFM ever do anyone any good.

Some people thought my problem was a shortage of inodes but this is not the
case. df -k and df -o i show free disk space and free inodes and yet when
the filesystem is fsck'd it shows up that there is 50% fragmentation and 0
blocks free.

The workaround came from Tons:

> We got the same problem with our web-caches. Since then we
> create the filesystems with:

> newfs -v -c 16 -C 7 -m 0 -o space /dev/rdsk/blafasel

I must say I was sceptical about this because these values appear to be the
defaults (except for space and minfree). Except that when I ran newfs -Nv on
my filesystem I could see that the defaults according to the man page are
not the defaults that mkfs seems to use. These default values seem to differ
between 2.5.1, 2.6 (fully patched) and the man page.

                         2.5.1 mkfs 2.6 mkfs man page
maxcontig (-C) -1 128 7
cgize (-c) 16 96 16

Anyway - this fixed the problem. Can anyone explain why these values vary
and why they have this side effect?

Finally - if you ever find yourself needing to restore this kind of data
(thousands of small files) make sure you pick up fastfs from Peter Grey/
Casper Dik for a 500% increase in filesystem speed. It basically turns of
synchronous metadata writes in the filesystem for the duration of the
restore (then you should turn it back on).
ftp.wins.uva.nl:/pub/solaris/fastfs.c.gz

Cheers
Keith McCabe

--- original mail ----

Hi all,

I need to restore a tar archive onto a 4Gb UFS filesystem which contains
thousands of small files. The problem is that this leads to excessive
fragmentation and I cannot complete the restore.

I'd like to newfs the filesystem with a smaller block and fragment size. ie

newfs -b 4096 -f 512 /dev/rdsk/blah but I know that for some reason Sun
still don't support block sizes other than 8192 on sun4u systems (according
to man page).

They say nothing about not supporting a fragment size of 512 and yet when I
try to newfs the filesystem I get

"fragment size 512 is too small, minimum with block size 8192 is 1024".
Anyone know of a workaround? Patch? Better idea?

thanks in advance

Cheers
Keith McCabe
Merrill Lynch Europe



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:12:53 CDT