The problem turned out to be that the Qlogic cards being used by Solstice disksuite were being hi-jacked by mpxio. Here's the appropriate entries from the various files involved (all in /kernel/drv). In this case we're attaching an HP SAN, presenting multiple LUNs up to 6. There's two Qlogic fibre cards controlling the internal drives, and they display on separate controllers. If you're using the optional wiring/configuration where they all appear on one, you only need one entry in qlc.conf. #################################fp.conf################################# name="fp" class="fibre-channel" port=0; name="fp" class="fibre-channel" port=1; name="fp" class="fibre-channel" port=2; name="fp" class="fibre-channel" port=3; name="fp" class="fibre-channel" port=4; name="fp" class="fibre-channel" port=5; # # List of ULP modules for loading during port driver attach time # load-ulp-list="1","fcp"; scsi-binding-set="fcp"; #################################end fp.conf################################# #################################bottom of qlc.conf################################# # the following lines disable multiplexing on the internal FC-AL drives, otherwise # half of them disappear on reboot name="qlc" parent="/pci@8,600000" unit-address="2" mpxio-disable="yes"; name="qlc" parent="/pci@8,700000" unit-address="4" mpxio-disable="yes"; #################################end qlc.conf################################# #################################scsi_vhci.conf############################# #### name="scsi_vhci" class="root"; mpxio-disable="no"; load-balance="round-robin"; auto-failback="enable"; device-type-scsi-options-list = "HP HSV200", "symmetric-option"; symmetric-option = 0x1000000; #################################end scsi_vhci.conf################################# For some reason the last four lines have to be pretty much verbatim. I had a comment for the second last line and it failed. For some reason the blank line needs to be there. With this files in place, do a boot -r. If you see a whole bunch of disks in 'format' instead of what' you've presented from the SAN (usually four times what you'd expect to see) mpxio didn't start correctly, usually because of a configuration problem. If there's SAN resources missing, its generally because they're on an unrecognized LUN, hence the mods to fp.conf. If some of your local hard drives disappear, it's a problem with Qlogic configuration. -----Original Message----- From: sunmanagers-bounces@sunmanagers.org [mailto:sunmanagers-bounces@sunmanagers.org] On Behalf Of Bruce Shaw Sent: 13 December 2006 01:23 To: 'sunmanagers@sunmanagers.org' Subject: configuring mpxio for emulex HBA's on a V880 I'm attempting to install Emulex HBA's on a V880 under Solaris 8 using SUNWsan. Boot disks etc. are mirrored using Solstice disksuite. If I configure /kernel/drv/scsi_vhci.conf with: ################################################################ mpxio-disable="no"; load-balance="round-robin"; auto-failback="enable"; device-type-scsi-options-list = "HP HSV200", "symmetric-option"; symmetric-option = 0x1000000; ################################################################ ...half my local hardware disks disappear from format, Solstice disksuite fails and files get corrupted and cannot be removed and generate I/O errors. I'm presuming I have to configure scsi_vhci.conf with mpxio-disable="yes" and individually configure multiplex devices within /kernel/drv/fp.conf. Format reports: 0. c1t0d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e801,0 1. c1t1d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e737,0 2. c1t2d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf279fb8,0 3. c1t3d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e0b8,0 4. c1t4d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e928,0 5. c1t5d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68df99,0 6. c1t8d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf6824ed,0 7. c1t9d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf682648,0 8. c1t10d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf2795e3,0 9. c1t11d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf681059,0 10. c1t12d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf681ce0,0 11. c1t13d0 <SUN36G cyl 24620 alt 2 hd 27 sec 107> /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf681ea5,0 for internal FC/AL drives. As it stands, fp.conf contains: ################################################# name="fp" class="fibre-channel" port=0; name="fp" class="fibre-channel" port=1; load-ulp-list="1","fcp"; scsi-binding-set="fcp"; ################################################# /usr/platform/sun4u/prtdiag -v reports: ######################################################################## #### ######## ========================= IO Cards ========================= Bus Max IO Port Bus Freq Bus Dev, Brd Type ID Side Slot MHz Freq Func State Name Model ---- ---- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ---------------------- I/O PCI 8 B 1 33 33 4,0 ok pci-pci8086,b154.0/network (netw+ PCI-BRIDGE I/O PCI 8 B 1 33 33 0,0 ok network-pci108e,abba.20 SUNW,pci-ce/pci-bridge I/O PCI 8 B 0 33 33 5,0 ok scsi-pci1000,f.1000.1000.14/disk+ I/O PCI 8 B 0 33 33 5,1 ok scsi-pci1000,f.1000.1000.14/disk+ I/O PCI 9 B 6 33 33 2,0 ok fibre-channel-pci10df,fa00.10df.+ LP10000 I/O PCI 9 B 5 33 33 3,0 ok fibre-channel-pci10df,fa00.10df.+ LP10000 ######################################################################## #### ######## prtpicl -v reports: ######################################################################## #### ######## fibre-channel (obp-device, 19000001ecf) :DeviceID 0x2 :UnitAddress 2 :vendor-id 0x10df :device-id 0xfa00 :revision-id 0x1 :subsystem-vendor-id 0x10df :subsystem-id 0xfa00 :min-grant 0xff :max-latency 0 :cache-line-size 0x10 :latency-timer 0x40 :interrupts 00 00 00 01 :class-code 0xc0400 :devsel-speed 0x1 :fast-back-to-back :66mhz-capable :compatible (19000001eedTBL) | pci10df,fa00.10df.fa00.1 | | pci10df,fa00.10df.fa00 | | pci10df,fa00 | | pci10df,fa00.1 | | pci10df,fa00 | | pciclass,0c0400 | | pciclass,0c04 | :reg 00 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00 10 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 03 00 10 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 01 00 10 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 :assigned-addresses 83 00 10 10 00 00 00 00 00 12 40 00 00 00 00 00 00 00 20 00 83 00 10 18 00 00 00 00 00 12 60 00 00 00 00 00 00 00 20 00 81 00 10 20 00 00 00 00 00 00 03 00 00 00 00 00 00 00 01 00 :num-nodes 512 :pm-support 0 :adisc-support 1 :assign-alpa 0 :cr-count 1 :cr-delay 0 :ack0 0 :link-speed 0 :topology 0 :network-on 1 :ub-bufs 1000 :num-iocbs 1024 :log-errors -1 :log-warnings -1 :log-notices -1 :console-errors 0 :console-warnings 0 :console-notices 0 :model LP10000 :devfs-path /pci@9,700000/fibre-channel@2 :driver-name emlxs :binding-name pci10df,fa00 :bus-addr 2 :instance 0 :_class obp-device :name fibre-channel fp (devctl, 19000001f03) :scsi-binding-set fcp :load-ulp-list (19000001f13TBL) | 1 | | fcp | :class fibre-channel :port 0 :pm-components (19000001f0cTBL) | NAME=FC Port | | 0=Port Down | | 1=Port Up | :devfs-path /pci@9,700000/fibre-channel@2/fp@0,0 :driver-name fp :binding-name fp :bus-addr 0,0 :instance 1 :_class devctl :name fp fibre-channel (obp-device, 19000001f17) :DeviceID 0x3 :UnitAddress 3 :vendor-id 0x10df :device-id 0xfa00 :revision-id 0x1 :subsystem-vendor-id 0x10df :subsystem-id 0xfa00 :min-grant 0xff :max-latency 0 :cache-line-size 0x10 :latency-timer 0x40 :interrupts 00 00 00 01 :class-code 0xc0400 :devsel-speed 0x1 :fast-back-to-back :66mhz-capable :compatible (19000001f35TBL) | pci10df,fa00.10df.fa00.1 | | pci10df,fa00.10df.fa00 | | pci10df,fa00 | | pci10df,fa00.1 | | pci10df,fa00 | | pciclass,0c0400 | | pciclass,0c04 | :reg 00 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00 18 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 03 00 18 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20 00 01 00 18 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 :assigned-addresses 83 00 18 10 00 00 00 00 00 12 80 00 00 00 00 00 00 00 20 00 83 00 18 18 00 00 00 00 00 12 a0 00 00 00 00 00 00 00 20 00 81 00 18 20 00 00 00 00 00 00 04 00 00 00 00 00 00 00 01 00 :num-nodes 512 :pm-support 0 :adisc-support 1 :assign-alpa 0 :cr-count 1 :cr-delay 0 :ack0 0 :link-speed 0 :topology 0 :network-on 1 :ub-bufs 1000 :num-iocbs 1024 :log-errors -1 :log-warnings -1 :log-notices -1 :console-errors 0 :console-warnings 0 :console-notices 0 :model LP10000 :devfs-path /pci@9,700000/fibre-channel@3 :driver-name emlxs :binding-name pci10df,fa00 :bus-addr 3 :instance 1 :_class obp-device :name fibre-channel fp (devctl, 19000001f4b) :scsi-binding-set fcp :load-ulp-list (19000001f5bTBL) | 1 | | fcp | :class fibre-channel :port 0 :pm-components (19000001f54TBL) | NAME=FC Port | | 0=Port Down | | 1=Port Up | :devfs-path /pci@9,700000/fibre-channel@3/fp@0,0 :driver-name fp :binding-name fp :bus-addr 0,0 :instance 2 :_class devctl :name fp ######################################################################## #### ######## So what I presumably need to do in fp.conf is to explicitly disable the onboard fibre controller so it doesn't multiplex the internal disks, but enable the specific entries for the SAN. The documentation example shows: mpxio-disable="yes"; name="fp" parent="/pci@13,2000/pci@2/SUNW,qlc@5" port=0 mpxio-disable="no"; but just to be on the safe side (yes, after trying to back out of the last install I'm paranoid) I'd like to explicitly disable the onboard internal drives eg. /pci@8,600000/SUNW,qlc@2/fp@0,0/ssd@w21000004cf68e801,0 but I don't understand what to use for a port number. The SAN is going to be broadcasting on LUN 1 and 2. Does that translate into "port=1" and "port=2"? Or, in short, what should my fp.conf entries look like to explicity disable multiplexing on the internal hard drives, but enable it on the SAN connections? Finally, is there any way of getting mpxio going without a reconfiguration reboot? It's almost impossible to get downtime on this machine. -- This communication is intended for the use of the recipient to which it is addressed, and may contain confidential, personal and or privileged information. Please contact us immediately if you are not the intended recipient of this communication, and do not copy, distribute, or take action relying on it. Any communication received in error, or subsequent reply, should be deleted or destroyed. _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagers This communication is intended for the use of the recipient to which it is addressed, and may contain confidential, personal and or privileged information. Please contact us immediately if you are not the intended recipient of this communication, and do not copy, distribute, or take action relying on it. Any communication received in error, or subsequent reply, should be deleted or destroyed. _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagersReceived on Thu Dec 28 13:47:27 2006
This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:44:03 EST