[Summary] saving disksuite info while rebuilding server

From: Chris Hoogendyk <choogend_at_library.umass.edu>
Date: Tue Feb 10 2004 - 14:57:18 EST
Finally done. Got a number of replies, some of which were useful. None 
fully covered the bases. I put in a support call to Sun and fleshed out 
the details (got to speak with a Network Storage Services Technical 
Support Engineer). Original message first. Then a summary of how I 
accomplished what I needed to do.


Chris Hoogendyk

  O__  ---- Network Specialist & Unix Systems Administrator
 c/ /'_ --- Library Information Systems & Technology Services
(*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst 



-------- Original Message --------
Subject: saving disksuite info while rebuilding server
Date: Fri, 30 Jan 2004 13:24:57 -0500
From: Chris Hoogendyk <choogend@library.umass.edu>
To: Sun Managers <sunmanagers@sunmanagers.org>

I'll try this once again.

I got one reply that had some useful information, but I need more guidance.

I have an E250 with an external D1000 all set up with DiskSuite,
including mirrored root drive and raid 10 on the D1000.

It is running Solaris 7.

I need to rebuild it (and repartitian the root drive) by installing
Solaris 8 without losing the state database information or any of the
data on the external D1000.

Can anyone give me assurrances, guidance, steps ....

Is it possible to install Solaris 8, repartitian the root drive, and
keep the rest of the system intact?


Chris Hoogendyk

   O__  ---- Network Specialist & Unix Systems Administrator
  c/ /'_ --- Library Information Systems & Technology Services
 (*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst 



sunmanagers mailing list


I found
<http://docs.sun.com/db/doc/806-3206/6jccd0hcb?a=view> which
goes through the updgrade in some detail. However, I have a corrupted
system (not the metadb or anything like that). My Solaris 7 system was
hacked and I can't trust it. I need to wipe it, not upgrade it. The
<http://docs.sun.com/db/doc/806-3206/6jccd0hca?a=view> warns
"Caution - If a full install of Solaris is performed, all DiskSuite
configuration information will be lost. Losing the DiskSuite
configuration information can result in data loss or complete system

I really wanted to re partitian the root drive and start fresh.

The critical data is on external arrays. Those are the metadevices that
need to be brought up without loss after the rebuild. The documentation
doesn't tell how to do this, it seems to imply that you shouldn't. However,
Sun support helped me through it.

Step 1: make sure you have tape backups of everything.

I had an old workgroup storage array with six 18G drives formatted RAID5 
that only had about 11G of data at the moment. This is the one I was 
concerned about getting back up. So it was easy to do a tape of it just 
in case.

My D1000 had 170G of data and until I could do the rebuild, my only 
means of backing it up was DDS/3 tapes which would take several days and 
upwards of a dozen tapes (the data is mostly tiff so it is already 
compressed). Fortunately, I had configured the D1000 as RAID 10, so I 
basically had two full copies on two separate controllers. Inefficient 
storage utilization, but very fast and secure. That meant that I could 
try to bring up one controller and then add the other as a mirror if it 

My root stuff was going to get blown away anyway with a repartitian and 
full install of Solaris 8. So, have at it. I did have tape backups 
anyway so that I could pull individual files if I needed them.

Step 2: specific information needed for the DiskSuite rebuilding process.

# metastat -p > ~chris/backup/md.orig.tab
# metadb > ~chris/backup/metadb.orig
# df -k > ~chris/backup/df.orig
# cp /etc/vfstab ~chris/backup/vfstab.orig
# cp /etc/system ~chris/backup/system.orig

Find the directory that has md.cf and/or md.conf (could be 
/etc/opt/SUNWmd) and copy this whole directory to ~chris/backup

Copy all these files to another server where you will have access to 
them during the install and configuration. If necessary do it using a 
tape and ufsdump/ufsrestore.

I had an advantage that the internal drives were all 9G, the old 
workgroup array had all 18G drives, and the D1000 had all 36G drives. So 
after the install, I could diagnose which was which using prtvtoc, just 
in case they somehow got assigned differently during the full install. 
Didn't happen. But, if this is a concern, you might want to also grab 
copies of the /dev/dsk, /devices and /etc/path_to_inst for reference.

Step 3: boot off install CD and do a full install.

Scary. Just what the documentation I found warned against. Even if I had 
a mirrored root drive, it wouldn't matter, because I am going to rebuild 
all that. Booted off a CD, it won't see it that way anyway.

So, now I can specify custom partitianing, use a 9G for root, a 9G for 
/usr/local, a 9G for /export/home, and a 9G left over, not touching the 
external array drives. Specified /var as a separate partitian of / just 
so I could have it big enough but also not over-run everything else if 
it got out of hand.

Be sure to select DiskSuite 4.2.1 as an install option.

Take care of securing the system. Shut down un-needed ports in 
inetd.conf, pull down latest recommended & security patches, do 
tcp_wrappers, etc., etc., ....

Step 4: OK, 3 all done, rebooted, now configure the arrays.

First, there are no state databases. The first one needs to be forced. I 
created them exactly as they were according to my metadb.orig. Turns out 
this was a problem, not because the strategy was necessarily wrong, but 
because the configuration of my old workgroup array was wrong originally 
when it was set up almost 5 years ago before I had any training. It had 
been set up with the state databases and the actual data on the same 
partitian. The drives come from Sun with s0 and s1 small, s2 everything, 
and s6 what's left after s0 and s1. This was set up with an s7 also 
everything and the state database and data both on s7.

Anyway, based the metadb.orig

# metadb -a -f c1t1d0s7 c1t2d0s7 c1t3d0s7 c1t4d0s7

This actually ended up blowing the data on that array. Sort of expected. 
I later reconfigured it the way it should have been and recovered it 
from tape.

Then, based on the md.orig.tab, and doublechecking with prtvtoc to see 
if it was the right drives:

# prtvtoc /dev/rdsk/c3t0d0s2

    <output snipped>

# metainit d21 1 6 c3t0d0s6 c3t1d0s6 c3t2d0s6 c3t3d0s6 c3t4d0s6 c3t5d0s6 
-i 64b

# metainit d20 -m d21

# metainit d22 1 6 c2t0d0s6 c2t1d0s6 c2t2d0s6 c2t3d0s6 c2t4d0s6 c2t5d0s6 
-i 64b

# metattach d20 d22

Then, based on the vfstab.orig, I edited /etc/vfstab and added the two 
lines for the external arrays. Then create the mount point corresponding 
to these and mount the D1000.

# mkdir /array1

# mkdir /array2

# mount /array2

Then "cd /array2" and check it out. Everything was there! Cool. That was 
the important one.

Then I had to rebuild the old workgroup array from scratch. But first, I 
needed to move the state databases off of it. later I could put them 
back on. Need to maintain some at all times (not delete them all and 
then add some somewhere else).

# metadb -a c3t0d0s1 c3t1d0s1 c3t2d0s1 c2t0d0s1 c2t1d0s1 c2t2d0s1

Note that the D1000 had been set up with the data on s6 and the state 
databases on s1, so they are separate partitians and utilize the default 
Sun formatting that they came with.


# metadb -d c1t1d0s7 c1t2d0s7 c1t3d0s7 c1t4d0s7

And I'm ready to rebuild this array from scratch.

# metainit d10 -r c1t6d0s6 c1t5d0s6 c1t4d0s6 c1t3d0s6 c1t2d0s6 c1t1d0s6 
-i 32b

Wait a long time.

# newfs /dev/md/rdsk/d10

Wait a long time.

# fsck /dev/md/rdsk/d10

# mount /array1

# cd /array1

# ufsrestore -if /dev/rmt/0n

Recover all the data. (Wait a long time.) Check to see that it is there 
and alright.


# metadb -a c1t1d0s1 c1t2d0s1 c1t3d0s1

This means I have 3 state databases on each external controller (the 
D1000 has two), and should still have a majority if one goes down. If 
the D1000 went down entirely, I would have only 3 of 9, but I could 
force it from there if need be. However, at that point I would be more 
concerned with getting the D1000 back up. If the old array went down, 
the D1000 would still be up and would have a majority.

Then check everything with metadb, metastat, etc.

Finally, make sure user accounts are all set, users have access, Samba 
is set up properly, etc.

It was a lot of trauma, but in the end it was not really that difficult. 
If I have to do it again, I could probably do it in a day or so. Also, 
since I have the old workgroup array configured properly (data on s6 and 
state database on s1), I would be able to bring it back up without the 
time consuming initialization, newfs and tape recovery. I would be able 
to do

# metainit d10 -r c1t6d0s6 c1t5d0s6 c1t4d0s6 c1t3d0s6 c1t2d0s6 c1t1d0s6 
-k -i 32b

Note the -k near the end. For a raid device, that tells metainit that 
the device has already been raided and just needs to have its 
configuration set up and confirmed.

I think that is everything. Now I can proceed with getting Netbackup 
installed, configured and running.


Chris Hoogendyk

   O__  ---- Network Specialist & Unix Systems Administrator
  c/ /'_ --- Library Information Systems & Technology Services
 (*) \(*) -- W.E.B. Du Bois Library
~~~~~~~~~~ - University of Massachusetts, Amherst 


sunmanagers mailing list
Received on Tue Feb 10 15:04:20 2004

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:43:26 EST