Many Thanks to Crist Clark, Francisco, Andrew Hay, Matthew Stier, Brad Morrison, James Abendschan, Richard Basch, Dean Ross-Smith, and Ceri Davies. You don't want the backquotes around the gzip commands in your script... By enclosing your gzip commands in backquotes (`), you are asking the gzip command be run in a subshell and then the output be executed... For instance... `echo /bin/ls` will actually run ls As far as performance enhancements - NFS mounting and SCP were suggested options and utilizing more CPU power by running the commands in parallel. Running the compressions in parallel in csh is trivial, /usr/local/bin/gzip $dumpfile1 & /usr/local/bin/gzip $dumpfile2 & /usr/local/bin/gzip $masterfile & wait The three gzips are run in the background, and the wait command waits for them to finish before the script continues on to do the rsync. You could scp it to your other server and it would be quicker and less processor intensive. if you have an rsa or dsa public key set up, then you don't worry about password issues. dumping a 4GB db to disk is a little over 10minutes on a 100mbit _____ From: Hallman, John Sent: Thursday, February 01, 2007 12:01 PM To: 'sunmanagers@sunmanagers.org' Subject: gzip and rsync return /usr/sbin/: Permission denied I've got a v440 4xCPU 16GB RAM - sol9 that every weeknight runs a cronjob as root which creates two dumpfiles and a master file from sybase, compresses them with gzip then moves via rsync to another server for safe keeping. load averages: 0.27, 0.32, 0.48 09:46:13 96 processes: 95 sleeping, 1 on cpu CPU states: 91.6% idle, 3.2% user, 4.5% kernel, 0.6% iowait, 0.0% swap Memory: 16G real, 4000M free, 11G swap in use, 9421M swap free The dumpfiles are around 10gb, compressed about 3gb (default gzip) The weird part is in the script it's giving me a permission denied even though the compression is done and the rsync works. Any ideas? Also any thoughts on improving this process (The Dump takes about 8 minutes, the first compression takes 31 minutes, the second compression 35 minutes, and the rsync takes 11 minutes - both the single gzip and rsync use 25% of the CPUs, There is only a 29GB partition to do this in - My DBA is almost done with changing the way the dump is done - so it is already compressed and should be ready in the next few days) During the first gzip load averages: 1.00, 0.77, 0.45 00:17:07 100 processes: 98 sleeping, 2 on cpu CPU states: 72.9% idle, 23.7% user, 3.4% kernel, 0.0% iowait, 0.0% swap Memory: 16G real, 4758M free, 11G swap in use, 9401M swap free PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND 4504 root 1 0 0 1344K 1024K cpu/3 3:27 24.93% gzip During the first rsync load averages: 1.03, 1.02, 1.02 01:23:49 101 processes: 99 sleeping, 2 on cpu CPU states: 74.2% idle, 23.0% user, 2.7% kernel, 0.1% iowait, 0.0% swap Memory: 16G real, 4996M free, 11G swap in use, 9404M swap free PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND 8224 root 1 0 0 3232K 2352K cpu/3 0:12 11.27% rsync Script: #!/bin/csh -f .......... .......... ls -ltr $dumpdir/* echo "-- `date` ---" echo "Compressing Dumpfile #1" echo " " `/usr/local/bin/gzip $dumpfile1` echo "-- `date` ---" echo "Compressing Dumpfile #2" echo " " `/usr/local/bin/gzip $dumpfile2` echo "-- `date` ---" echo "Compressing Masterfile" echo " " `/usr/local/bin/gzip $masterfile` ls -ltr $dumpdir/* Script output -rw-r--r-- 1 sybase sybase 10941902848 Feb 1 00:13 /dump/prod7-2-02012007 -rw-r--r-- 1 sybase sybase 10946639872 Feb 1 00:13 /dump/prod7-1-02012007 -rw-r--r-- 1 sybase sybase 6281216 Feb 1 00:13 /dump/master-02012007 -- Thu Feb 1 00:13:39 PST 2007 --- Compressing Dumpfile #1 /usr/sbin/: Permission denied 2010.0u 78.0s 0:00 20880000% 0+0k 0+0io 0pf+0w -- Thu Feb 1 00:48:37 PST 2007 --- Compressing Dumpfile #2 /usr/sbin/: Permission denied 2011.0u 77.0s 0:00 20880000% 0+0k 0+0io 0pf+0w -- Thu Feb 1 01:23:35 PST 2007 --- Compressing Masterfile /usr/sbin/: Permission denied -rw-r--r-- 1 sybase sybase 3167122140 Feb 1 00:13 /dump/prod7-2-02012007.gz -rw-r--r-- 1 sybase sybase 3166857336 Feb 1 00:13 /dump/prod7-1-02012007.gz -rw-r--r-- 1 sybase sybase 1050394 Feb 1 00:13 /dump/master-02012007.gz bash-2.05$ ls -l /usr/local/bin/gzip -rwxr-xr-x 3 bin bin 303324 Oct 16 2002 /usr/local/bin/gzip gzip 1.3.5 bash-2.05$ ls -l /usr/local/bin/rsync -rwxr-xr-x 1 bin bin 2410784 Oct 7 2005 /usr/local/bin/rsync rsync version 2.6.6 protocol version 29 _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagersReceived on Thu Feb 1 19:07:12 2007
This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:44:04 EST