Clear winner was ufsdump | ufsrestore
Apparently ufsdump only sends instructions to ufsrestore over the pipe,
not the entire file, so 2Gb limitations are not a big problem, even on
large file systems.
rdist, rsync, tar and cpio were also mentioned.
Casper Dik also suggested ufsdump | ufsrestore, but I lost his e-mail.
He made a mysterious reference to something that I've heard him make
mysterious references to in the past: fastfs. This can apparently
dramatically improve the performance of ufsdump | ufsrestore. I mention
this because a couple of people mentioned that ufsdump | ufsrestore did
the best job, but was not known for speed...
In any case, my transition went well. By just replacing drives in my
Baydel RAID units, I have increased online capacity from 46Gb to 260Gb
and didn't modify a single permission, file modification time, etc.
But it did take about 3 times as long as I scheduled! :-) I need to find
out more about fastfs, and I also need a faster server! Of course I'm not
going to work on my scheduling, just spend more money on hardware! :-)
Many thanks to all that responded:
Thomas Lester <tlester@methos.iakom.com>
Ronald Loftin <reloftin@syr.edu>
Paul Pescitelli <pbp@mindspring.com>
Tim Evans <tkevans@tkevans.com>
Dave Haut <dhaut@level1.com>
John Douglass <john.douglass@anl.gov>
Chan Cao <ctcao@fastech.com>
Matthew Stier <Matthew.Stier@fnc.fujitsu.com>
Sebastian Benoit <benoit@Mathematik.Uni-Marburg.de>
Seth Rothenberg <SROTHENB@montefiore.org>
Salehi, Michael E <Mike.Salehi@usa.xerox.com>
----------
Kevin Sheehan {Consulting Poster Child} <kevin@joltin.com>
You will probably have a problem on 2.5.1 - one way to avoid it is to run
the tars on subdirectories so that the total tar size is less than 2GB.
Might even want to run multiple tars at the same time...
----------
Janis Lykakis <Janis.Lykakis@ehv.ce.philips.com>
You have two systems, each with a Raid unit and you want to copy
the data FROM raid1 on machine1 TO raid2 on machine2, is that correct?
If so, then I found the fastest way to be:
On machine2, make a directory /RAID1
on machine2 mount machine1:/raid1 on /RAID1
cd /RAID1
find . -depth -print | cpio -pdm /raid2
Because NFS reads are faster than NFS writes, you should not
do this
the other way around, so do not mount the empty raid2 on machine1
and copy raid1 to raid2 via NFS. Always mount the full raid1 via
NFS on the machine to wich you want to copy the data
This find/cpio pipe worked much faster than dump/restore. Big
filesystems are no problem.
Copy speed highly depends on the number and size of the files.
----------
From: Thomas Lester <tlester@methos.iakom.com>
Nope... that's how I'd do it. "ufsdump 0cufl - /filesystem | ufsdump xvf
----------
From: Ronald Loftin <reloftin@syr.edu>
IMHO ( or not-so-humble opinion ) you should have no great difficulty.
The
2GB limit under Solaris 2.5.1 applies to individual files, not to the
filesystem. I have done this under 2.5.1 with an Oracle database living
in
multiple 2GB files in a single filesystem and had no problem.
----------
From: Paul Pescitelli <pbp@mindspring.com>
I would take a look at rsync http://rsync.samba.org I am not saying that
this is the best option, but a definite contender.
UFSDUMP should work just fine also..
----------
From: Tim Evans <tkevans@tkevans.com>
Why not use rdist?
----------
From: Dave Haut <dhaut@level1.com>
I've never done 30 GB, but I have done 8-10 GB and I didn't have any
problems
with using tar | tar.
If time is not an issue, I would use ufsdump | ufsrestore. tar | tar is
quicker, but I would recomment that you use gnutar instead because tar has
a
character limit on filenames. If you have really long filenames on the
filesystem that you are trying to copy over, tar will not copy them ( gtar
will ).
----------
From: John Douglass <john.douglass@anl.gov>
Never had to move 30Gb but have moved upwards of 20 using a ufsdump
piped to the ufsrestore without problem. A couple notes about the tar:
1) it tends to fill out zero spaced files, so if you have a core file
that is 10Mb but takes up 3 disk blocks, and then tar and untar it, the
core file will take up 10Mb. Use star or gnu's tar to avoid this (gnu's
tar has a -S option for preserving sparse files)
2) tar has a maximum length for file names (including path) so if you
have some directory structures that are really deep, it may fail --
again use star or gnu's tar to avoid this since their limit is
significantly higher.
I've never really used cpio for this purpose but it would certainly
work. My suggestion is a simply ufsdump | ufsrestore:
ufsdump 0cf - /filesystem | (cd /whereever; ufsrestore rx -)
I'm not at a solaris box right now so I can check my options
appropriately. Also, if the new RAID is hung off a different box you
can stick in an rsh after the pipe:
ufsdump ... | rsh newbox '(cd xxx; ufsrestore ...)'
----------
From: Chan Cao <ctcao@fastech.com>
Don't know if I'm of any help but what I'm about to suggest is along the
line of 1-file-at-a-time copy.
find . -exec cp {} /newdir \;
I depend a lot on the "tar" construct as well as "ufsdump and
ufsrestore"...
You seem to hint at some problems. I hope you include a line describing
the problem for those of us uninformed.
Also, what about shutting down the machine and perform a dd with some
big blocksize?
----------
From: Matthew Stier <Matthew.Stier@fnc.fujitsu.com>
I'd recommend either ufsdump/ufsrestore, or cpio. Tar has a poorly
documented problem of support filenames up to 100 characters in length.
My preferece is ufsdump/ufsrestore. It is designed for this task, and
will
preserve file times better.
----------
From: Sebastian Benoit <benoit@Mathematik.Uni-Marburg.de>
Use 'rsync' from http://samba.anu.edu.au/rsync/
You can use it to copy the filesystem off hours, stop it when
your system is busy and then restart it (it will find out which files
need to be copied (again) and which not - much like rdist, but with a
more intelligent algorithm.
----------
From: Seth Rothenberg <SROTHENB@montefiore.org>
Tom,
I guess this is after the fact, but I thought I would
answer you anyway....
We recently did an upgrade, and the ufsdump | ufsrestore method,
as found in the "examples" section of the ufsrestore man page,
was a vital part of our procedure, and it worked as documented.
In the past, I have even used it across a network. I don't know
if that is guaranteed.
----------
From: "Salehi, Michael E" <Mike.Salehi@usa.xerox.com>
If you find a way to speed it up, let us know but ufsdump 0uf - /
|
(cd /mnt; restore rf - )
worked for me.
-- -- Tom Mornini -- InfoMania Printing and Prepress
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:13:35 CDT