MORE SUMMARY patching binary - you will love this

From: Broken Hill Proprietary (bhppa@sugar.NeoSoft.COM)
Date: Mon Mar 07 1994 - 11:00:07 CST

I have been asked by several people to give a more detailed summary
(this may be an oxymoron, but none-the-less) so some of you might
find this familar:

I recently sent the following:

> Hardware: Sun 670/41
> OS: SunOS 4.1.3
> Hi,
> I have a program that normally runs interactively but I want it to
> run overnight. This should be a simple matter of sticking it into
> an appropriate shell script and giving it to _at_, which I tried.
> The program crashed (with zero grace, I might add), so I ran it
> under trace, kind of like this, but in a shell script with a lot of
> not relevant environmental setup:
> at now + 1 minute <<END
> #! /bin/sh
> exec > stdout 2> stderr
> trace pgm
             This trace is the way I tracked down the problem. trace
             gives a list of all the system calls made by the program.
             When the program orignially failed, it just said something
             like "open cannot find file", didn't even say what file.
             Using trace told me what file.

> This tells me that pgm is trying to open /dev/tty, which is possible,
> albeit lame, in an xterm window where the pgm normally runs, but
> under _at_, there is no longer the hook to the xterm and /dev/tty
> doesn't exist. I guess the reason pgm opens /dev/tty is that it
> wants to tell me something even though my standard out is redirected,
> both in interactive and batch mode.
> So I call my software vendor (I do not have the source, BTW), and they
> are going to get right on it and I should have a fix REAL SOON NOW, but
> meanwhile I REALLY NEED to run this compute hog program hundreds of
> times by the end of this month and not interfere with the very heavy
> day schedule. So I figure, what the heck.

               About a week later, they still haven't even formally
               acknowledged the problem, let alone given it a
               priority, let alone solved it, let alone gotten me
               a fixed copy.
> I wrote a perl script that goes in and finds /dev/tty and changes it to
> /tmp/dmy.

              This perl script looks like:

        perl -pi.bak s#/dev/tty#/temp/dmy#g pgm

>I tried to run the program again and (shockingly enough) it
> works and even gets past the open and opens my file. BUT...

                Again using trace

> The next thing that happens after the open is an ioctl call, which does
> something termio specific and therefore gives me:
> open ("/tmp/dmy", 02, 0) = 5
> ioctl (5, 0x40125401, 0xafebe4) = -1 ENOTTY (Inappropriate ioctl for device)
> writev (2, 0xeffff1f8, 4) = ioctl: Inappropriate ioctl for device
> ---- sudden program death syndrome ----
        BTW, touching /tmp/dmy before executing the program lets it
        go a little further in that the program gracefully closes its
        files and exits, but it still doesn't do its job. I need to
        fool the program into thinking it got what it wanted.
> which makes me think that the filio kind of thing I am not trying to do
> is getting an inappropriate ioctl call.
> So now I am reading ioctl and trying to figure how in the heck I am going
> to patch it. I can find the request -0x40125401-, but I'm a little unsure
> about the arg -0xafebe4-, which I think is a pointer to a struct anyway.
> I think I could just maybe clobber the ioctl call.
> Does anyone have any suggestions while I wait for my vendor to get me
> the real fix?
> Will Morse
> ---- perl, the Swiss Army Chainsaw of sysadmin tools ----

I got several responses. suggested a program called pty, which is available
from This looks like the best solution except it
does not seem to run on SunOS 4.1.3 judging from the documentation.

     Claus Assmann <> advises me that there is
     a Version 4.0 of pty that will run under SunOS 4.1.3. I plan to
     check this out in the near future suggested expect, which is a fairly complicated
solution for just this problem, but I am pursuing it because it may
be useful in other areas too. It uses Tcl, which we already use.
     I have had several other similar messages since. I found expect
     version 2.56, which seems to have the same problem, opening
     a non-existant /dev/tty, as the original program. There may
     very well be a later version of expect that I don't have that
     can do this, however. and both suggested making the /tmp/dmy
a real or pseduo tty. As I see it, this will throw away part of the
report, but let the job complete. I am looking at this as well.

     This is in fact what I did, and the program is working well, the
     whole business of opening /dev/tty is a historical artifact in
     the program, they don't actually use it for anything.

Other suggestions are still welcome.

I want to say how much I appreciate all the help and suggestions. I don't
know how I would get past some of these problems if it weren't for the
kindness of all the thousands (millions?) of people on the net. Thank
you one and all.

Will Morse

This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:08:56 CDT