summary: filesystem grows out of control

Hi,

Thanks for the replies from:

Darren Dunham
Hichael Morton
Stephen Cawley
Alex Ugolini
Alan Pae

Finally, I have to bring the machine to single user mode with basic filesystem mounted. There 
is one new, big file under one automount point. I still have no idea how such file created 
since the consensus is it is impossible to create file under mounted filesystem.

thanks & have a good long weekend,

carl

:> Message: 10
:> Date: Wed, 25 Jun 2003 17:52:46 -0400 (EDT)
:> From: Carl Ma <carl_ma@scotiacapital.com>
:> Subject: filesystem grows out of control
:> To: sunmanagers@sunmanagers.org
:> Message-ID: <200306252155.h5PLt4q05582@sbtormlr2.ScotiaCapital.com>
:> Content-Type: TEXT/plain; charset=us-ascii
:>
:> Hello all,
:>
:> We have one production server - E4500 with T3+ diskarray running
:> solaris 8 with
:> latest patch level. It used to be very stable till today.
:>
:> %df -k -F ufs
:> Filesystem            kbytes    used   avail capacity  Mounted on
:> /dev/md/dsk/d0    6191949 3293302 2836728    54%    /
:> /proc                      0       0       0     0%    /proc
:> fd                         0       0       0     0%    /dev/fd
:> mnttab                     0       0       0     0%    /etc/mnttab
:> swap                  670536       0  670536     0%    /var/run
:> swap                  671664    1128  670536     1%    /tmp
:> /dev/md/dsk/d1    400089861 200071534  200018327   50%    /space
:> /dev/md/dsk/d2    200149000   20987800  179161200	10%	/space/data1
:> /dev/md/dsk/d3	...	...	...	...	/space/data2
:> ...
:> /dev/md/dsk/dN	..	...	...     ...	/space/dataN
:>
:> We can see there are lots of local filesystems  mounted under /space.Also
:> hundreds of files,directories and automount/nfs mount points are
:> also under
:> /space directory.
:>
:> Today, the /space usage grows continuously,slowly but constantly.
:> It grows about
:> 10% since noon,which is very abnormal.
:>
:> I did the following check:
:>
:> 1."du -d -sk /space" every 5 seconds,which shows the usage is growing
:> 2."du -d -sk /space/*" every 5 seconds,which gives me all the directories
:> capacity information including nfs.(-d doesn't work??)
:> 3."find /space -xdev -type f -mtime -1 -ls | sort -n -k7", it
:> does give me the
:> list of changed files on /space. There are a lot but none has big size.
:> 4. failed to use tnfdump to analyze since this filesystem has
:> lots of I/O. I am
:> lost at certain stage. :-)
:>
:> My questions are:
:> 1.may I just display the size of every directories under
:> /space(d1),excluding
:> NFS and directories crossing filesytem boundaries(du -dsk
:> /space/* doesn't
:> work)?
:> 2.is it possible that someone hides data under mount points? how
:> can I find it?
:>
:> Right now, I am working on a dirty script and would like to hear
:> from others. I
:> will summarize if solution is available.
:>
:> thanks a lot,
:>
:> carl
:>
:
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers