SUMMARY: Solaris 9 Sparc iSCSI Initiator - Options ?

Hi all,

I had submitted this query in the "fairly recent" past and had gotten
on answer back which I summarized to the list (basically - Cisco has
an EOL iScsi initiator for Solaris9 but not many other options exist)

Since then I did some testing in a related vein which I have found
interesting enough that I think it should be summarized back to the
list as an "addendum" on this topic, with the hope that this is of
interest to other sunmanagers.

I've tested and deployed a solution based on "AoE"
(ATA-over-Ethernet), using a generic Linux server as the AoE target
(which is "exporting" 6x500gig drives in a raid6 array of approx
1.9Tb) and then using the free AoE initiator for Sparc Solaris package
to access this storage.

The deployment/use/functionality is similar to iSCSI but it is a
better fit for my needs (better performance; software is still under
active development - for starters).  The net result has been
"gig-ether-mediated-direct-attached" storage on my e4800 server of
performance better than expected/desired, with really simple
setup/deployment (and a very low cost :-)

Notes are attached below which illustrate the (trivial) process
involved in doing this.

Certainly I believe kudos are in order for the folks @ Coraid (who are
keen on AoE and are developing AoE software for many platforms,
including solaris).

I hope this is of some interest to someone possibly,

---Tim Chipman



-----------paste----------------
Basic Context:

-AoE target (which exports block device storage) is a generic CentOS
(RHEL) Linux box

-AoE initiator (which imports the block device, and uses the disk
capacity as a "direct attached" storage - is Solaris9 Sparc e4800
server

-connectivity is via regular gig-ether (no jumbo frames currently, nor
trunking etc.)

Steps involved are detailed below:

(1) linux box (centOS RHEL5 64-bit - absolutely minimal server install
but with "kernel-devel" and "gcc" packages added)

(a) Setup AoE tools: userspace tools to help evaluate status of AoE
(probably not essential)
root@linux# make; make install

(b) Setup vblade: userspace binary which allows export of local block
devices via AoE:
root@linux# make ; make install

(c) start vblade and "export" metadevice /dev/md0 (software raid6 1.8 Tb):
(syntax note - 1 1 refers to "shelf and tray" id of the "AoE
Blade";used in addressing the device from AoE clients)
root@linux# vblade 1 1 eth1 /dev/md0 &

(2)  Then setup your AoE Client (Solaris 8/9/10 sparc-intel)

(a) install the binary package pre-compiled AoE software:
root@sun# pkgadd -d ./CORDaoe-1.3.2

(b) Configure the NIC to use for AoE and then start the service:
root@sun# echo "aoechan 0 ether skge0" > /etc/aoe.conf
root@sun# /etc/init.d/aoe start

(c) Probe to see that disk is visible:
root@sun#  aoectl list
0/1/1

(d) Label the disk, a kludge required since >1Tb
root@sun# aoelabinit /dev/rdsk/cad101s0

(e) Now we can see the thing via "Format" correctly:

root@sun # format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
      0. c0t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
         /ssm@0,0/pci@18,700000/pci@1/SUNW,isptwo@4/sd@0,0
...etc....
      9. cad101s0 <Coraid E therDrive vblade  201>   <== this is the AoE "disk"
         /pseudo/aoed@101

Specify disk (enter its number):


(f) Format and mount: Takes approx 3-4 minutes:

root@sun # newfs /dev/rdsk/cad101s0
...output truncated here ...
root@sun # mkdir /data/aoe-disk
root@sun # chmod 777 /data/aoe-disk
root@sun # mount /dev/dsk/cad101s0 /data/aoe-disk

Verify we see it OK:

root@kt-4800 # df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d10      20755753 12571816 7976380    62%    /
/proc                      0       0       0     0%    /proc
....etc.......
/dev/dsk/cad101s0    1929788336   65560 1910424896     1%    /data/aoe-disk

For fun, Do a few tests with Bonnie:


root@sun # cd /data/aoe-disk ; /opt/bin/Bonnie -s 1024 -m aoe-nas
...etc... for each local disk being tested...


             -------Sequential Output-------- ---Sequential Input-- --Random--
             -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  K/sec %CPU  /sec %CPU
=======  ==== ===== ==== ===== ==== ===== ==== ===== ===== ======
===== ======= ====
aoe-nas  1024 12750 31.8  8858  7.7 12113  7.3 42449 100.0 495221 99.7
18089.8 162.8
int-sun  1024  7696 27.0  7331  7.2  7624  5.8 33083 100.0 564553
100.1 29710.6 200.5
nfs-nas  1024  2664  7.4  5370  4.4  3404  0.4 40898 100.0 326999
100.1   742.8  27.9

note the nfs-nas is an older external 3rd party NAS device running on
8x300gig Raid5 via gig-ether connectivity (of known mediocre
performance)

int-sun is a 6250 sun disk array (fibre disks and fcal connectivity,
effectively a t3 variant I think)


Misc Hardware Notes: Linux box is running on very basic hardware: asus
AM2NPV-vm mainboard, Sempron 1800mhz CPU, 1gig ram, 6x500gig drives
(4xSata and 2xATA) - 500gig chosen because they are very affordable at
approx $130 each, although 750gig disks could be used just as easily
to create a larger array of approx 2.8Tb capacity with Raid6 type
config.  Connectivity is via onboard gig-ether which does NOT seem to
support jumbo frames. I plan to add a dge-530t gig-ether NIC which
does support jumbo frames - hopefully for some slight improved
performance and/or reduced load on servers.

The e4800 has a Syskonnect skge 9821 type PCI nic, something I'm fond
of (they tend to work well and are a good price).

Note that for connecting  the AoE you should ideally have either (A) a
spare gig-ether nic in your solaris machine, or (B) a gig-ether nic
which is fairly idle and is on a "trusted" subnet.


------ORIGINAL POSTING asking about iScsi---------------
 Hi all,

I'm looking at the prospect of using iSCSI to provide storage to a
somewhat aged Solaris8 (enterprise 4800) server in our department, but
I am having trouble getting a handle on my options for iSCSI Initiator
software.

Google indicates there is lots of support for iSCSI (both initiator
and target) in Solaris10, but I get the feeling Sun hasn't back-ported
any of this stuff.

Most open-source iSCSI initiator projects seem focused on Linux.

I gather one option I may have is the Cisco iSCSI initiator, which I
gather may be now somewhat out of favour (at least on platforms where
other options exist?)

I am simply curious if anyone else has been using iSCSI initiator
software (free ideally) on production Solaris 9 Sparc platforms, and
if so, any pointers on what was used (and how implementation has
been!?) -- would be greatly appreciated.

Many thanks,

--Tim Chipman
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers