[SUMMARY] Re: sec:u Trying to understand Solaris's /devices ....

From: Wilkinson, Alex <alex.wilkinson_at_dsto.defence.gov.au>
Date: Tue Nov 23 2004 - 07:05:38 EST
Original question:

0n Fri, Nov 12, 2004 at 11:05:35AM +1030, Wilkinson, Alex wrote: 

> Hi all,
> 
> (426)-12.11.2004 FreeBSD-{10:56}-[/dev]->df .
> Filesystem 1K-blocks Used Avail Capacity  Mounted on
> devfs              1    1     0   100%    /dev
> 
> (426)-12.11.2004 solaris-{10:55}-[/dev]->df .
> /                  (/dev/vx/dsk/rootvol):34020298 blocks  3741943 files
> 
> Now I thought solaris implemented DEVFS in the same fashion as FreeBSD ?
> 
> Doesn't solaris have the concept of mounting a 'device filesystem' on /devices ?
> 
> Because, I know we use devfsadm(1m) to dynamically manage /dev and /devices on
> Solaris but neither mount points refer to a device filesystem, but a 'real'fs.
> 
> Can someone give me a quick rundown on the way Solaris manages /devices.
> 
> Thanks
> 
> - aW

Thanks to:

James Noyes
Darren Dunham
Ric Anderson
Matthew Stier
Russell Page

******************************************************************************

Solaris uses two different "device-related" directories, but neither one is
a separate filesystem unto itself.

"/devices" is set up in a parallel fashion to the physical configuration of
the devices and busses.  Software-only devices are in "/devices/pseudo", and
physical devices will be in a directory named after the bus type, such as
"/devices/pci@1f,4000", or "/devices/sbus@1f,0".  Devices with other devices
attached will be a directory as well, such as SCSI controllers, or USB hubs.
The attached devices, such as disks or keyboards will be under those
directories.  The actual block-special and character-special device files are
all set up in this directory tree.  For example:

brw-r-----   1 root     sys       32,  0 Jul 16  2003
/devices/pci@1f,4000/scsi@3/sd@0,0:a
crw-r-----   1 root     sys       32,  0 Dec 30  2003
/devices/pci@1f,4000/scsi@3/sd@0,0:a,raw

The /dev directory is where many other systems (Linux, FreeBSD, and HP-UX all
do as far as I know) put their block-special and character-special devices.
These systems have no analogous concept of the "/devices" directory, and
therefore have no place where their filesystem parallels the physical
connection of the devices.  For example, all of their disks appear in a
single directory with no real indication of which groups of them may be
connected to different controllers, or inside different enclosures.

Solaris uses the /dev directory to provide a matching naming scheme to these
other systems, but the only thing Solaris places in the the /dev directory is
symbolic links to the REAL block-special and character-special devices back in
/devices.  For example:

lrwxrwxrwx   1 root     root          41 Jul 16  2003 /dev/dsk/c0t0d0s0 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:a

lrwxrwxrwx   1 root     root          45 Jul 16  2003 /dev/rdsk/c0t0d0s0 ->
../../devices/pci@1f,4000/scsi@3/sd@0,0:a,raw

lrwxrwxrwx   1 root     other         53 Sep  5  2003 diskette ->
../devices/pci@1f,4000/ebus@1/fdthree@14,3023f0:c

The devfsadm command allows for easy manipulation of the block-special,
character-special and symbolic link files contained in /dev and /devices to
manage the devices, but the device-related files all exist as real files on
the / filesystem.  They do not get a filesystem unto themselves.

An interesting byproduct of this design is that once the symbolic link is
established, since it resides on the / filesystem, and points to a device file
that respresents the actual connectivity and placement of the device, you
don't get devices spontaneously "changing names".

Anecdote: I watched a friend and Linux-lover nearly lose his sanity when a
newly-attached 12-disk enclosure on his Linux system claimed the first 12 disk
IDs /dev/sda through /dev/sdl for itself because it was on a lower bus than
the internal controller, and happily relocated his two internal disks to
/dev/sdm and /dev/sdn.  Solaris will never do that.  Once a /dev/dsk/cXtXdX
assignment is linked to a particular disk, it sticks (for the most part).
New disks will be assigned proper cXtXdX identifiers reflecting the controller
and scsi/atapi ID they're connected with.

It takes a little getting used to, but I've found that it seems to be more
sensible than having to chase down devices that seem to migrate /dev
identifiers on every reboot.ZZ

******************************************************************************

> Hi all,
>
> (426)-12.11.2004 FreeBSD-{10:56}-[/dev]->df .
> Filesystem 1K-blocks Used Avail Capacity  Mounted on
> devfs              1    1     0   100%    /dev
>
> (426)-12.11.2004 solaris-{10:55}-[/dev]->df .
> /                  (/dev/vx/dsk/rootvol):34020298 blocks  3741943 files
>
> Now I thought solaris implemented DEVFS in the same fashion as FreeBSD ?

Not through Solaris 9 it doesn't.  It may do some of that in 10.


> Doesn't solaris have the concept of mounting a 'device filesystem' on
> /devices ?

No, not any production release.

> Because, I know we use devfsadm(1m) to dynamically manage /dev and /devices
on
> Solaris but neither mount points refer to a device filesystem, but a
> 'real'fs.

Yup.

> Can someone give me a quick rundown on the way Solaris manages
> /devices.

Devices are built there either by a package installation, or by the
device driver directly when it polls attached devices (usually based on
prompting by 'devfsadm').


******************************************************************************

OK, here's my attempt :-)
        /devices contains device entries
        /dev contains symlinks to things in /devices
        for the most part, but there are exceptions like
        /dev/fd/xxx
for disks, the links in /dev/dsk point to the physical
device in /devices using things like
        /dev/dsk/c1t4d0s7->/devices/pci@1f,4000/scsi@3,1/sd@4,0:h

/dev/dsk is the device independent way to reference things
in some sense, as /dev/dsk/c0t0d0s0 refers to the disk
on controller 0, target 0, device 0, slice 0, regardless
of whether the disk is an IDE drive at
  /devices/pci@1f,0/ide@d/dad@0,0:a
or a SCSI drive at
  /devices/pci@1f,4000/scsi@3/sd@0,0:a
"Target" refers to the SCSI ID of the device for SCSI disks,
and is an index 0-3 for IDE drives on a dual port IDE controller
such as found in the Ultra 5, 10, and Blade-100 systems.  A
"device" in the above is a SCSI LUN; its zero for non-
SCSI disks.  A slice is the same as a BSD partition.

File system references use /dev/dsk/cXtXdXsX in /etc/vfstab
to specify the mapping between mount points and partitions.
    
"df" by default is gives System V looking output as you show
below.  I long ago stuck in
        alias df='df -k'
in my .kshrc.  Thus, I get
 $ cd /
 $ df .
 Filesystem            kbytes    used   avail capacity  Mounted on
 /dev/dsk/c0t0d0s0    3529071 2272869 1220912    66%    /
which is probably more of what you expect.  You can check Inodes
with
 $ df -o i .
 Filesystem             iused   ifree  %iused  Mounted on
 /dev/dsk/c0t0d0s0      80110  359186    18%   /

You can list local disks with
 $ df -F ufs
 Filesystem            kbytes    used   avail capacity  Mounted on
 /dev/dsk/c0t0d0s0    3529071 2272926 1220855    66%    /
 /dev/dsk/c0t0d0s3    29463453 4764908 24403911    17%    /export/clusmgt
 /dev/dsk/c0t1d0s2    35009161 3126790 31532280    10%    /export/u1
 /dev/dsk/c1t2d0s2    35321458   35057 34933187     1%    /export/u3
 /dev/dsk/c1t1d0s2    35321458 22475632 12492612    65%    /export/u2


Notes and cautions:
* The entries in /devices use the boot rom IO device tree
  nomenclature.  Thus, /devices entries are platform and
  PCI slot specific.  Replace PCI with SBUS on older iron.
  Although you can clone boot disks, it takes some serious
  flapping in /etc and /devices if you want the new disk
  to boot on a different hardware platform than it was built
  on.
* entries in /dev/rdsk are the RAW devices and refer to
  the same hardware locations as their /dev/dsk counterparts.

******************************************************************************

The ''/dev" directory is simply a BSD holdover, since most applications
require it.

The "/devices" directory entries are simply pointers into the kernel.
Choose a "/devices" subdirectory and do a long listing of it.  You will
see two numbers between the gid and datestamp fields.  These numbers are
driver and device numbers.  Each driver loaded by Solaris is assigned a
driver number.

All "/devices" entries that use the same driver, will be assigned the
same driver number.
     
Each device that uses a driver is also assigned it's own device number. 
(Both of these are setup during a reconfiguration reboot.
     
Thus when I decide to write something to /dev/console, the operating
sytems opens /dev/console; realizing it is symbolic link opens the file   
it refers to. Realizing that that file is a character device, reads it's
driver and device numbers, then can make the call to the appropariate
driver, with a reference to the applicable device to be written to.

******************************************************************************

So are we all!

When the POST (Power On Self Test)  runs, it creates an in-memory tree
structure that maps the devices actually found on the system. You can poke
around here at the "ok" prompt. This tree survives the bootstrap and is used
by the kernel to actually access the devices. When you run a "reconfigure"
boot, the script /etc/rcS.d/S50devfsadm builds a file system copy under
/devices. It then creates a tree under /dev that consists almost entirely of
symbolic links pointing back into /devices.
     
Climbs on soapbox. (I've been working with Solaris since it was in beta and
I still think this is all badly broken.)  For true compatibility with
standard UNIX practice at the time, they should have built a normal tree
under /dev and made the /device directory the one full of symlinks. Then at
least "ls -l /dev/null" would have continued to return sensible results.  In
fact, I don't think the /devices directory is of any use to users,
programmers, or administrators. I still don't understand why it is there at
all.
     
Also while I understand their rationale for embedding the controller,
target, lun, and partition in the disk names, it would also have been really
useful if they had given us a set of names that didn't vary across machines.
hd0a, hd1a, etc. Certainly Linux does, and my memory of BSD is that it did
too.

I have been unable to automate installing and configuring Disksuite from
Jumpstart because every Sun model seems to have a different set of names for
its disks, and I can't find a reliable, trustworthy way of finding these
names from a script. Maybe if I wasted several more days poking around I
could.

End of rant.


******************************************************************************

The End.
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
Received on Tue Nov 23 11:14:11 2004

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:43:40 EST