SUMMARY: Tarring onto an Exabyte 8500

From: Guy Freeman (g.freeman@fml.co.uk)
Date: Mon Aug 15 1994 - 05:31:54 CDT


My original question:
> Dear Sun Managers,
>
> I want to tar data onto a remote Exabyte 8500 tape drive. I'll be using
> a command like:
>
> tar cvf - file | rsh remote dd of=/dev/nrst0.
>
> What blocksize should I use to get maximum use out of the tape??
>

I received many replies (included below).

Most said, make sure you use /dev/nrst8 rather than nrst0 to use the high
density mode of the Exabyte.

For blocksize, a lot of people said it doesn't really matter very much,
while others said use 112 or 126. I've tried 112 and this seems to work ok
so I'm gonna stick with this. Command used:

        tar cvfb - 112 file | rsh remote dd of=/dev/nrst0 obs=112b

                        -----------------------------

Many, many thanks to the following Sun Managers:

jpl@allegra.att.com (John P. Linderman)
glenn@uniq.com.au (Glenn Satchell - Uniq Professional Services)
fintan_keeling@horizon.ie
anderson@neon.mitre.org (Mark S. Anderson)
epl@Kodak.COM (Gene Loriot (epl@Kodak.COM.))
brb@ike.safb.af.mil (SrA Bruce R Baier)
Anthony.Worrall@reading.ac.uk (Anthony Worrall)
mel@geol.abdn.ac.uk (Andrew Mellanby)
Gary.Richardson@proteon.com (Gary Richardson)
bismark@alta.jpl.nasa.gov (Bismark Espinoza)

                        -----------------------------

>
> From jpl@allegra.att.com Thu Aug 11 12:52:58 1994
> Date: Thu, 11 Aug 94 07:49:32 EDT
> From: jpl@allegra.att.com (John P. Linderman)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 1168
>
> You need to be careful on many counts. First, be sure to do something like
> obs=10k on the dd. Otherwise, the tar stream will be fragmented going over
> the net, and will NOT be reassembled into full-sized blocks. Use obs=, not
> just bs=. See the dd man page.
>
> On many systems, you'll run into integer overflow problems on the
> seek pointer if you try to write more than about 2 gig as a single file.
> Keep chunks smaller than that.
>
> I believe that file marks are about 40Kb on 8500's. There are no
> inter-record gaps as such. I believe stripes of 10Kb are written
> on each helical swipe. Any multiple of 10Kb should keep you from
> getting padding, and will get you full use of the tape. But I'm not
> a hardware jockey, so get confirmation on this last paragraph before
> you depend on it. Tapes are only $10. Don't be greedy.
>
> From glenn@uniq.com.au Thu Aug 11 14:06:45 1994
> Date: Thu, 11 Aug 1994 23:00:32 --1000
> From: glenn@uniq.com.au (Glenn Satchell - Uniq Professional Services)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> X-Sun-Charset: US-ASCII
> Content-Length: 1134
>
> tar cvf - file | rsh remote dd of=/dev/nrst8 obs=126b
>
> You need rst8 to get high density mode, th e126 is the largest block
> that can be transferred in one scsi command so this will make the
> transfer go as fast as possible.
>
> To untar the tape use
>
> tar xvbf 126 /dev/rst8
>
> regards,
> --
> Glenn Satchell glenn@uniq.com.au | NO /\ PAIN
> Uniq Professional Services Pty Ltd ACN 056 279 335 | / \
> PO Box 70, Paddington, NSW 2021, (Sydney) Australia | \ /
> Phone 02 360 7434 Pager 016 287 000 Fax 02 331 2572 | NO \/ GAIN
>
> From fintan_keeling@horizon.ie Thu Aug 11 15:17:52 1994
> Date: Thu, 11 Aug 1994 15:08:45 +0000
> From: Fintan Keeling <fintan_keeling@horizon.ie>
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> X-Sun-Charset: US-ASCII
> Content-Length: 590
>
>
> Now, there's a good point. And I thought that the block size specifeid on
> dd was only the output buffer size. Presuming you machin ain't too busy,
> I would go for bs=64k or more if you have the memory. If the block size
> afect the tape, please let me know.
>
> Best of Irish,
> Fintan Keeling.
>
>
> ______________________________________________________________
> Fintan Keeling fintan.keeling@horizon.ie
> Customer Support Tel +353 1 6716313
> Horizon Open Systems Fax +353 1 6714061
> 1-3 High Street
> Dublin 8
> Ireland
>
>
> From anderson@neon.mitre.org Thu Aug 11 15:29:50 1994
> Date: Thu, 11 Aug 94 10:29:33 EDT
> From: anderson@neon.mitre.org (Mark S. Anderson)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 1834
>
>
> It doesn't matter what blocksize you use. But, in general, larger
> blocking factors result in better throughput and better medium
> utilization.
>
> If you want the tape to be readable by tar when you extract the files,
> then you had better be sure that the blocksize is compatable with tar.
> The default tar blocking factor is 20 blocks (20*512 bytes). According
> to the manpage, tar automatically determines the blocksize when reading
> tapes -- but I wouldn't count on it until I tested it. The default
> blocksize for dd is 512 bytes. You might want to change your command
> to:
>
>
> tar cvf - file | rsh remote dd of=/dev/nrst0 bs=10240, or
> tar cvbf 126 - file | rsh remote dd of=/dev/nrst0 bs=65512
>
> If you will be reading the tape through a pipe, then you will need to
> use the B option of tar.
>
> I would urge you to get into the habit of using the manpages.
> Everything I have written was taken directly from the manpages for tar
> and dd.
>
>
>
>
> Mark Anderson
> ----------------------------------------------------------
> The MITRE Corporation manderso@mitre.org
> 7525 Colshire Drive, MS W747 voice: (703) 883-6439
> McLean, VA 22102 FAX: (703) 883-1905
>
> From epl@Kodak.COM Thu Aug 11 15:42:51 1994
> Date: Thu, 11 Aug 1994 10:39:48 +0500
> From: epl@Kodak.COM (Gene Loriot (epl@Kodak.COM.))
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> X-Sun-Charset: US-ASCII
> Content-Length: 581
>
>
>
> Sun tell me that the exabyte uses a block size of 112 for optimal read and
> especially writting....BUT you must specify this block size, it is not
> either tar, or ufsdump's default.
> --
> #### ###### Gene Loriot E-mail: epl@Kodak.COM
> ### ######## Eastman Kodak Company Voice: (716) 724-6962
> ## ########## Customer Equipment Services Fax: (716) 724-9860
> # ### KODAK ## 343 State Street
> ## ########## Mail Code: 00708
> ### ######## Rochester, New York 14650-0708
> #### ###### U. S. A. [On a clear DISK you can SEEK forever]
>
> From brb@ike.safb.af.mil Thu Aug 11 15:52:58 1994
> Date: Thu, 11 Aug 94 09:46:09 CDT
> From: brb@ike.safb.af.mil (SrA Bruce R Baier)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 1377
>
> I'm assuming you are using SunOS 4.X on your machines.
> READ these instructions FULLY before you proceed.
>
> What you need to do is a 2-step process. I got this from Exabyte tech
> support: (and am using it for dumps in 5GB mode)
>
> 1. In the file /usr/kvm/sys/scsi/targets/st_conf.c
> look for a section that begins with -
>
> "Exabyte EXB-8500 8mm Helical Scan", 16, "EXABYTE EXB-8500",
>
> and underneath there is a line -
>
> { 0x14, 0x00, 0x8C, 0x8C },
>
> Change this line to:
>
> { 0x15, 0x00, 0x8C, 0x8C },
>
> Then save this file and make a new kernel.
> (This needs to be done on the machine with the tape drive).
>
> 2. You will need to address the drive as /dev/(n)rst8 to
> enable high-density (5GB) mode. Using /dev/(n)rst0 will
> use low-density (2.3GB) mode.
> Therefore, your tar command would look like:
>
> tar cvf - file | rsh remote dd of=/dev/nrst8
>
> Blocksize shouldn't affect the density of the tape.
> Good luck. Let me know how this turns out.
>
> Bruce R Baier, SrA, USAF
> 1500 CSGP/COSWM
> Scott AFB, IL, USA
>
> brb@ike.safb.af.mil
>
>
>
> From Anthony.Worrall@reading.ac.uk Thu Aug 11 16:35:54 1994
> Date: Thu, 11 Aug 1994 16:33:51 +0000
> From: Anthony.Worrall@reading.ac.uk (Anthony Worrall)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> X-Sun-Charset: US-ASCII
> Content-Length: 314
>
>
> tar cvfb - 112 file | rsh remote dd of=/dev/nrts0 obs=112b
>
>
> Anthony.Worrall@Reading.ac.uk
>
>
>
> From @aberdeen.ac.uk:mel@geology.aberdeen.ac.uk Thu Aug 11 16:50:55 1994
> X-Sender: mel@hutton.geol.abdn.ac.uk
> Mime-Version: 1.0
> Content-Type> : > text/plain> ; > charset="us-ascii">
> Date: Thu, 11 Aug 1994 18:35:30 +0100
> To: g.freeman@fml.co.uk (Guy Freeman)
> From: mel@geology.aberdeen.ac.uk (Andrew Mellanby)
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 1043
>
> The block size is 126.
> BUT.. You won't get 5G unless you use the 5G device driver,
> Specify the device as /dev/nrst8. (/dev/nrst0 is the 2.5G device driver).
>
> There is a good page on Answerbook about this.
>
> +------------------------------------------------------------------------+
> | Andrew Mellanby Tel: +44-224-272750 (am) 3462 (pm) |
> | Computing Officer Fax: +44-224-272785 |
> | Department of Geology Email: mel@geol.abdn.ac.uk |
> | and Petroleum Geology |
> | Meston Building |
> | King's College, Aberdeen AB9 2UE |
> +------------------------------------------------------------------------+
>
>
> From Gary.Richardson@proteon.com Thu Aug 11 17:07:55 1994
> Date: Thu, 11 Aug 94 12:06:42 EDT
> From: Gary.Richardson@proteon.com (Gary Richardson)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 1316
>
>
> Hi,
>
> I don't think you'll need to worry about a block size. You should,
> however, use /dev/rst8 and not rst0:
>
> Tape Unit Low(2gbyte) High(5gbyte)
> ---------------------------------------------
> st0 /dev/rst0 /dev/rst8
> st1 /dev/rst1 /dev/rst9
> st2 /dev/rst2 /dev/rst10
>
>
> I think that'll take care of fitting 5GB on the tape. But as far as
> blocksize, when using dump I use a blocksize of 126 for all my backups.
> I would think that would work for your tar'ing as well.
>
> Good luck!
>
> -Gary
>
> From bismark@alta.jpl.nasa.gov Thu Aug 11 19:07:58 1994
> Date: Thu, 11 Aug 94 11:13:48 PDT
> From: bismark@alta.jpl.nasa.gov (Bismark Espinoza)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Cc: bismark@alta.jpl.nasa.gov
> Content-Length: 129
>
>
> This is what I use for an Exabyte 5 GB:
>
> rsh remote_host /usr/etc/rdump 0udbsf 54000 126 13000 local_host:/dev/nrst8 /dev/rsd5a
>
> From allyn@netcom.com Thu Aug 11 20:35:03 1994
> From: allyn@netcom.com (allyn)
> Subject: Re: Tarring onto an Exabyte 8500
> To: g.freeman@fml.co.uk
> Date: Thu, 11 Aug 1994 12:33:51 -0700 (PDT)
> Mime-Version: 1.0
> Content-Type> : > text/plain> ; > charset=US-ASCII>
> Content-Transfer-Encoding: 7bit
> Content-Length: 104
>
> It does not matter what blocksize you use. The larger the blocksize, the
> faster the transfer will be.
>
>
> From rick@mrpeabody.llnl.gov Thu Aug 11 22:14:04 1994
> Mime-Version: 1.0
> Content-Type> : > text/plain> ; > charset="us-ascii">
> Date: Thu, 11 Aug 1994 14:12:04 -0800
> To: g.freeman@fml.co.uk (Guy Freeman)
> From: rick@mrpeabody.llnl.gov (Rick Reineman)
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 780
>
> The man page suggests a block size of 126, the Online: Backup manual says
> the 8mm is optimized at 112 blocks. We use Online: Backup so stick with
> the 112, I think most people will respond with the 126 though. Probably
> doesn't matter much.
>
> Rick Reineman
> System Manager - Lawrence Livermore National Lab
> reineman1@llnl.gov
>
>
>
> From csppw@solomon.technet.sg Fri Aug 12 04:51:13 1994
> Date: Fri, 12 Aug 1994 11:50:09 +0800
> From: Foo Jon Hui <csppw@solomon.technet.sg>
> To: g.freeman@fml.co.uk (Guy Freeman)
> Subject: Re: Tarring onto an Exabyte 8500
> X-Newsreader: TIN [version 1.2 PL0]
> Content-Length: 409
>
> use nrst8 and you will get 5 gig of data on the tape
>
>
> From jpl@allegra.att.com Fri Aug 12 17:59:53 1994
> Date: Fri, 12 Aug 94 12:52:28 EDT
> From: jpl@allegra.att.com (John P. Linderman)
> To: g.freeman@fml.co.uk
> Subject: Re: Tarring onto an Exabyte 8500
> Content-Length: 2677
>
> Thanks a lot for you swift reply to my question. You seem to be pretty
> knowledgable on the subject and I wonder if you could help me. I've had a
> lot of replies now, and most people seem to suggesting using a block size of
> 126 bytes (although there are exceptions) and to make sure I use /dev/nrst8
> rather than /dev/nrst0. What do you think???
>
> Thanks Guy.
>
>
> If I'm your most knowledgable contact, you are in deep trouble. :-)
>
> My (openly imperfect) understanding of helical scan technology is that
> ``stripes'' of 10 1024-byte blocks (including blocking and
> error correction data) are written in each diagonal motion across
> the tape surface. If you write a file of size 5555 bytes,
> you use 6 of these blocks and the remaining 4 (and a good chunk of the
> fifth) are just padding, since a read or write cannot start in mid-block
> or mid-stripe. The drive has a large buffer built in, so that units
> of 10240 bytes can be accumulated and written.
>
> If this is accurate (or close), then there is a certain ``magic''
> about 10K multiples, since there will never be a need to pad the
> data. I see no particular harm in using larger multiples, say
> 20KB or 30KB, but on some very old machines (15-bit architectures),
> you may run a risk of not being able to handle buffer sizes larger
> than 32767. If you are dealing with newer machines, 60KB should be
> fine. Some people like 63K (126b that you keep seeing), but IF
> (that word again) my understanding is right, this really buys you
> nothing, and may end up with some padding. (I don't know how long
> the controller waits before flushing a partially filled buffer.
> If it waits until a close, block size doesn't matter much at all.
> If it does not, 10K multiples would have some advantages. You'd
> need to talk to an Exabyte tech to get the truth.)
>
> The device name may or may not be important. SunOS 4.1.3 and Solaris
> use minor device numbers to select write density. SunOS 4.1.1 did NOT.
> 4.1.1 writes blank tapes and tapes written in 8500 format in the 8500
> format (``5 gig''), and it rewrites tapes originally written in 8200
> format (``2.3 gig'') in 8200 format. I don't know what 4.1.2 did.
> /dev/rmt8 is the ``right name'' to select the 8500 format on 4.1.3,
> assuming that tape appears at boot as rmt0. Ours appears as rmt1,
> so we use rmt9 to select 8500 format. If you have an 8505 on 4.1.3
> or solaris, you can also select compression. Check the mtio manual
> page if you are running 4.1.3. Use blank tapes if you are running
> 4.1.1.
>
> By the way, I wrote files of 100 meg onto a ``5 gig'' tape until
> it filled. It hit the end around 4.4 gig. Don't expect to get a
> full 5 gig.
>
> Good luck.
>



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:09:08 CDT