SUMMARY: Enterprise 3500 FC-AL Drives Not Visible

From: Tom Kinter <tmk_at_mayo.edu>
Date: Fri Jun 09 2006 - 10:03:50 EDT
Sun Managers,

As it has been said many times before, this group is fantastic! Thanks eveyone
for all of your help.
I'm somewhat embarassed by the situation that's probably causing my problem. I
need to connect
the upper frame!

The original question:

> I'm having a problem seeing the drives I have installed in the upper bank of
4
> slots on our Enterprise 3500. We are running Solaris 8.
> I have four drives in the lower bank (slots 0-3) and two in the upper
> (slots4,5)  When I run the format command it lists only the drives in the
> lower bank. I looked for an existing article and I found one, "SUMMARY
E3500
> not seeing some of its internal drives", but I couldn't exactly understand
it
> and the links it gave no longer exist. Could someone please point me in a
> direction to begin diagnosing the problem? I appreciate your time.

The replies:

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

have you connected the upper frame? you need 2 Fcal interface for each
 frame, lower and upper...
check sunsolve.sun.com in the handbook section

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Did you run devfsadm after installing the drives?  Are there any messages in
the messages file?
Did you check to make sure that there is a board in the upper bank?  I know
these sound like
 basic questions but I thought I'd put it out there.

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Insert the new disks and then reboot.
I performed almost this very task a few months ago in the hope that
something direct (like, say, running `devfsadm`) would allow me to
access the new drives, but no dice -- I had to do a reconfiguration
reboot. It was Solaris 2/2004, I think, on a two-board E3500, and I put
a new drive into the far righthand slots of each row.
If you find a "real" solution, I would be interested in heraring
about it. :7)

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

I bet you neeed another gbic and fiber cable

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Check the fiber cabling on the back of the machine. When looking at the
back, on the left side is a fibre board with slots labeled 'UB', 'UA',
'LB', 'LA'. The upper slots are connected to UA and UB. The lower slots
are connected to LA and LB. If there is not fibre connection to UA or UB,
these drives won't be visible.

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Did this ever work in the past?  If so, then you have a fiber
cable or GBIC failure, most likely.

If it never worked before, then perhaps you simply have not
fiber- cabled the top row -- they are two separate banks of four
drives, each on it's own fiber channel link.

In the back of the E3500, you should see a very slender system
board off to the left, right next to the power module which is on
the far left in the rear view.  That board should have at least
space for up to four GBIC modules, and therefore up to four fiber
channel cables.  This is the FC-AL Interface Board, and connects
external fiber channel cables directly to each of the two
internal drive bay FC chains.

You only need two connections to make both top and bottom banks
work -- the other two are for redundancy.

A cable should be connected from one of the top two GBIC slots on
the FC-AL Interface Board to an IO board GBIC slot (probably, but
not necessarily, on the far right in the rear view).   And you'll
need at least one connection from one of the bottom two GBIC
slots on the FC-AL Interface board again over to a GBIC slot on
an IO board.

If you only have one such connection then that's the problem.

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Couple of things to check.
* Make sure the External Fiber cables from the left rear to the right
  rear of the 3500 are in place.  It talks to those internal shelves via
  two external sets of fiber connectors (go figure).  If you only have
  one set of connectors hooked up, you'll only see one shelf.
* I had a nightmare here with FC-AL disks.  Sun support had me download
  the current Sun Disk FC-AL microcode, and reflash all the drives to the
  same Microcode level.  This cured all the bad stuff, but its made me
  really gun shy about adding disks to the 3500 without taking it down
  first, because mixing disk microcode levels caused channel hangs and some
  data loss before support pointed me to that disk microcode upgrade.

Here's some old notes I had from that fun time back in 2002
 There's apparently a random timing bug in enterprise 3500s
 that results in second FCAL controller vanishing during a
 warm start.

 The requirements are (in case you care):
  * One or more Seagate model ST118273F 18GB disks on
    the second controller.
  * One of them getting whigged out during a reset (which happens
    across a reboot) and latching up the entire controller.

  Sun Patch 109962 upgrades the drive level firmware to 9954, and
  that seems to have fixed it.

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

If probe-scsi-all or probe-fcal at the OBP> doesn't see it, get a new
I/O board, gbics, or fiber cables.

Try yanking 1 disk at a time. A single fiber disk performing a
"broadcast storm" can knock out the entire loop.

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

I know this sounds like a dumb question, but, have you got the top bank
cabled in?

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Check the fiber cables at the back. Next to the power supply tou can see
the internal disks fiber ports. The two port at the below connects to
0-3 disks, the two port at the top connects to 4-7.(or reverse i donnot
remmember now) You have to connect upper two as the two below...

The upper bank needs another fiber lead from the IO board.
Your current set-up is probable:-
IO board Fiber 0(port A GBIC 0) to Interface board GBIC LA or GBIC LB

You need to add a fiber:-
IO board Fiber 1(port B GBIC 1) to Interface board GBIC UA or GBIC UB

Then at the OK prompt run:-
probe-fcal-all

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

With fcal drives you might need to use luxadm to add additional drives after
the install.  I know for sure you do with the A5000, but I don't know if
luxadm works with E3500 internal drives (It does with A5x00 and SSA arrays).

-----------------------------------------------------------------------------
---------------------------------------
-----------------------------------------------------------------------------
---------------------------------------

Many thanks to:

Luc S
Scott J
William E
Scott K
Thomas C
KR
Ric A
JV
Andrew W
Mahmut S
Richard S
Marco G

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Tom Kinter   tmk@mayo.edu   507-284-4981
             Ultrasound Research Lab
      Mayo Foundation Rochester MN 55905 USA
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
Received on Fri Jun 9 10:05:10 2006

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:43:58 EST