SUMMARY: moving few million small files over NFS

From: NetComrade <>
Date: Tue Nov 19 2002 - 12:00:10 EST
The general concensus is that tar over ssh is better.

One of you suggested using dd instead of tar

One of you suggested mounting with 'noatime' "That way it will have one
less update to do on each file on each filesystem. Also, if you don't need
to keep the perms and ownership (you probably do) then use a method that
doesn't retain them like plain 'cp' because on really small files the
perms and ownership take longer than the actual data."

One of you suggested taring, gzipping, scp/ftp and ungzipping and

Thanks to 

Casper Dik
Kerekes, Ed
Kevin Korb
Jon Godfrey
Steve Maher

for replies
On Tue, 19 Nov 2002, NetComrade wrote:

> would you recommend 
> 	* cp and then rm
> 	* mv 
> 	* ssh -C nfshost "cd /nfs/dir ;tar cf - *"| (tar xBpf -)
> 	* any other method
> i would think 3 is faster, b/c it doesn't have to communicate on a
> file-by-file basis between hosts.
sunmanagers mailing list
Received on Tue Nov 19 12:01:24 2002

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:42:58 EST