Well, I am once again amazed at all the responses that I received in
such a short period of time.
I have included all the responses that I received at the time I am
writing this. Thanks VERY much to the following (esp to Leon Koll who
provided a summary of a similar question he asked a while back):
email@example.com (Glenn Satchell - Uniq Professional Services)
firstname.lastname@example.org (Dan Lorenzini)
Michael Assels <email@example.com>
firstname.lastname@example.org (Eckhard Rueggeberg)
email@example.com (Leon Koll)
firstname.lastname@example.org (Lothar Mueller)
email@example.com (Birger A. Wathne)
Mike Raffety <firstname.lastname@example.org>
stern@sunrise.East.Sun.COM (Hal Stern - NE Area Systems Engineer)
email@example.com (Shelley L. Shostak)
firstname.lastname@example.org (Dances on keyboards)
------------- 8< ------------- cut here ------------- >8 ----------------
Here is my original post:
# We have several Suns with multiple Ethernet interfaces. I know that all
# interfaces are configured with a single ethernet address, and that there
# would be no problem as long as they're each on a separate network.
# My question:
# I know there's a way to change that (using ifconfig perhaps??).
# How is this done??? (Do I have to RTFM? :)
# How do we know what the 'real' ethernet address is on the second
# Will this cause any problems???
# The reason why we need to do is because of the Ethernet switching hub
# that we wish to use, which requires the 'real' Ethernet addresses of all
# Any help/comments/suggestions will greatly be appreciated.
# Btw, all machines are Sparc 10s running SunOS 4.1.3.
# Will summarize.
email@example.com (Birger A. Wathne):
'ifconfig ether 8:0:20:x:y:z' should work.
To be safe, find the ethernet address of some other host that will
never get physically connected to the same segment, and use that
address. Or perhaps the address of some dead/returned/obsolete
equipment. You should try to keep the first 3 octets, as they identify
the interface as belonging to a Sun computer.
You could use 8:0:20:19:3d:ff as that is my client's address.
My Classic should never get physically connected to one of
your subnets, I guess.
firstname.lastname@example.org (Lothar Mueller):
We do it this way:
ifconfig ie2 ether 8:0:20:9:d3:b6 in /etc/rc.boot
where 8:0:20:9:d3:b6 is the original ethernet-adress
of the id-prom + 1. We hope we'll nerver get a device
with this adress.
Lothar M|ller, SUN-Systemadministrator (email@example.com)
GMD-I2, Rheinstr. 75, D-64295 Darmstadt, Germany
Tel.: +49 6151 869 897 +204, Fax: ++49-6151-869224
firstname.lastname@example.org (Leon Koll):
the bottom line is :
#ifconfig le1 ether 48:0:20:x:x:x
I suppose these two summaries will be useful to you.
Glad to help you,
Leon Koll email@example.com
System & Network Administrator
Orbotech Ltd. Voice (972) 8-423-664
Yavne, Israel Fax (972) 8-438-769
----- Begin Included Message -----
>From sun-managers Mon Jul 19 05:16 IDT 1993
Subject: SUMMARY: 2 Ethernet from a SS10 on Kalpana possible?
Yes it is possible, but it is not that simple.
The Sun is a SparcStation10 with a FSBE board.
This give me 2 ethernet ports.
The idea of hooking 2 ethernet ports on the same
network is not a gain because of the limitation
of 10Mb/s of that network.
The Kalpana Ethernet switch is a device that
switch ethernet packets from port to port almost
like a bridge depending on the MAC address.
Now, using two ethernet ports with a Kalpana will
give us about 2 x 10Mb/s network as the
switch take care of the traffic.
The important aspect of this configuration is
to have the server thinks it has 2 subnets
while the rest of the sun believes there is
To get it to work I needed to do the following:
1) Turn off IP_FORWARDING in the config of the kernel,
this was causing a lot a bad routing information
on the networks
options IPFORWARDING="-1" # see /sys/netinet/in_proto.c
2) Assign a different Ethernet address to le1
This is required by the Kalpana. With the
same MAC address on 2 ports, the switch keep
updating it's routing table.
I added the following to rc.boot
if [ "$1" != "xx0" ]; then
+ ifconfig $1 ether "`shcat /etc/ether\.$1`"
ifconfig $1 "`shcat /etc/hostname\.$1`" netmask + -trailers up
Where I put the Sun assigned ether in /etc/ether.le0 and
a different one for /etc/ether.le1
3) Create 'virtual' subnet on the SS10 by using
bogus 22.214.171.124 for le0 on 'pseudo' network 126.96.36.199
bogus1 188.8.131.52 for le1 on 'pseudo' network 138.11.128.00
Note: You need to be able to broadcast 184.108.40.206 so
the higher subnet is the most important. Also, because
the rest of the network is on net 138.11, we might
need to cover the full address range with all the
4) Add the following to /etc/networks
5) Modify /etc/networks to create subnet for the SS10
6) Move the following ifconfig before NIS is started
in /etc/rc.local in order to have the Sun believe
it's /etc/networks & /etc/netmasks.
ifconfig -a netmask + broadcast + > /dev/null
7) Prevented in.routed from starting in /etc/rc.local
You don't want the Sun routing packets, you want
the Kalpana doing it.
8) Add necessary route with 'route add ...'
9) Change half to the clients address to 132.11.129.???
10) Place those clients on subnet 138.11.0 on a different
port of the Kalpana than the clients on net 138.11.128.
11) Modify automount maps so that client on pseudo network
220.127.116.11 mount from 18.104.22.168
Without this modification, outbound NFS traffic for
subnet 138.11.128 was going to le1 but the inbound
traffic was going to le0.
The setup is running without noticable problem for about
15 days now. (There is some strange things like two
ntp connection, but does not cause any problem)
I have no tools to mesure if we effectively get
2 x 10Mb/s but netstat show that the traffic is
split in half between each interface and that the collision
rate is back to normal level (< 1.25%).
7:32pm up 6 days, 6:44, 2 users, load average: 1.28, 0.77, 0.72
inet 22.214.171.124 netmask ffff8000 broadcast 126.96.36.199
inet 188.8.131.52 netmask ffff8000 broadcast 184.108.40.206
Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue
le0 1500 220.127.116.11 bogus 21516762 109 20531289 1 74122 0
le1 1500 18.104.22.168 bogus1 18692667 8 23570747 0 290294 0
lo0 1536 loopback localhost 2700868 0 2700868 0 0 0
Destination Gateway Flags Refcnt Use Interface
localhost localhost UH 6 2651461 lo0
22.214.171.124 bogus1 U 4 15335298 le1
matrox-ether bogus U 60 16521136 le0
matrox-token alf-ether UG 0 0 le0
Thanks to the following peoples and in particular to
Mike Raffety to pointing out the concept of letting
the server and only the server believe that there is subnets.
Subject: 2 Ethernet from a SS10 on Kalpana possible?
X-Mailer: ELM [version 2.3 PL11]
We just received a Kalpana 1500C Etherswitch and I want to use 2 ethernet
adapters (le0, le1) from the same SS10 (SunOS 4.1.3) on two ports of
the Kalpana. This would give me 2x10Mb/sec throughput on the SS10 and
still use the same network (that is no subnet).
>From what I read on the archive, it does not seem to be possible because
SunOS will only use le0 in output.
Kalpana send me a configuration sheet regarding NC400 coprocessors, but
this is not our setup, we have only le0 and le1 interface.
They specify that each ethernet adapter should use a different MAC address.
This is configurable with ifconfig by using
8:0:20:1a:f5:a3 for le0 and
48:0:20:1a:f5:a3 for le1
I read that 802.3 permit to "or" 0x40 with the higher address bit to resolve
Kalpana also say that internal routing should be disable on the SS10,
so I have to turn off IP_FORWARDING.
What about the IP address, can I use the same 126.96.36.199 on both
ethernet adapter? (Probably not) Should I use 188.8.131.52 for le0
and 184.108.40.206 for le1 instead?
Is using 2 NC400 different than using 2 lex interface in regards to routing?
Does somebody already try this?
> Real Page ,
Definently don't use the same IP address on both ehternet adaptors. This will cause duplicate IP address error message.
Is the bottleneck your encountering on your network due to traffic or the ethernet interface of the SS10 is unable to handle the amount of information being directed towards ?
There are 3 solutions.
1. maybe think about a subnet using the ss10 as the router (unless you already have a router)
2. look at using ATM to give you 50 MB/s to 10K Mb/s or FDDI, 100 Mb/s ethernet.
Prudential Life and Pensions
=> We just received a Kalpana 1500C Etherswitch and I want to use 2 ethernet
=> adapters (le0, le1) from the same SS10 (SunOS 4.1.3) on two ports of
=> the Kalpana. This would give me 2x10Mb/sec throughput on the SS10 and
=> still use the same network (that is no subnet).
=> >From what I read on the archive, it does not seem to be possible because
=> SunOS will only use le0 in output.
If you specify routes through le1, then that's where the packets will go.
Note that you'll need to specify routes to your local subnet as static
host routes, which may be a bit of a nuisance if there are many hosts on
=> Kalpana send me a configuration sheet regarding NC400 coprocessors, but
=> this is not our setup, we have only le0 and le1 interface.
Sorry. Can't help here.
=> They specify that each ethernet adapter should use a different MAC address.
=> This is configurable with ifconfig by using
=> 8:0:20:1a:f5:a3 for le0 and
=> 48:0:20:1a:f5:a3 for le1
=> I read that 802.3 permit to "or" 0x40 with the higher address bit to resolve
=> this issue.
That's what we do, and there are no problems. A warning: Make sure that
there is no time when the two interfaces are both up with the same ethernet
address, or you'll get a very confused switch when it sees the same address
on two ports. As far as I can tell, this means doing
ifconfig le0 220.127.116.11 netmask 255.255.255.0 broadcast ...
ifconfig le1 ether 48:0:20:1a:f5:a3
ifconfig le1 18.104.22.168 netmask 255.255.255.0 broadcast ...
in that order (i.e., the first thing you do with le1 is change its ethernet
address). Watch out for
vmunix: duplicate IP address!! sent from ethernet address: 0:0:f4:80:7:83
messages in /var/adm/messages. I believe these come during the very brief
time when le1 has no specified IP address. In any event they are only a
nuisance, not a real problem.
=> Kalpana also say that internal routing should be disable on the SS10,
=> so I have to turn off IP_FORWARDING.
This won't hurt, but it shouldn't even be necessary if only one subnet is
connected to the switch. (We have 4 subnets on the switch, so we do turn it
=> What about the IP address, can I use the same 22.214.171.124 on both
=> ethernet adapter? (Probably not) Should I use 126.96.36.199 for le0
=> and 188.8.131.52 for le1 instead?
You'll definitely want two IP addresses. I'm not sure exactly what would
happen with only one, but at very least you would would get multiple
responses to arp, which cause some confusion.
-- +-------------------------------+----------------------------------+ | Michael Assels, Analyst | Everything is what it is and not | | Comp. Sci., Concordia Univ. | another thing. | | 1455 de Maisonneuve O | - Aristotle (or someone else) | | Montreal, Quebec, H3G 1M8 +----------------------------------+ | Voice: (514) 848-3035 | firstname.lastname@example.org | +-------------------------------+----------------------------------+
I have heard from Sun that indeed, you cannot do this because the Sun will only use ethernet, even if two are connected. We solved this problem by having multiple servers talking to multiple kalpanas, but only one connection between any two boxes. --Peter
------------------------------------------ -------------------------------- Peter Baer Galvin (401) 863-7623 Systems Manager, Brown U. CS Department email@example.com Box 1910 (115 Waterman Street) uunet!brunix!pbg Providence, RI 02912 (02906) firstname.lastname@example.org
The only thing that has been left out in your description is the problem relating to the IP addresses on the two interfaces. The two interfaces on the Sun MUST think they are on different >networks<. You will need to set special subnet masks on this host or contrive a setup with a different IP net or subnet address.
This need for different networks on interfaces produces a secondary problem: routing. You will need to tell the Sun which route to use for specific traffic. Adding static host routes is probably the best path. In general remember that traffic for a specific destination will need to always use a specific interface. Dynamic routing is likely to be impossible.
If what you were expecting is the logical equivalent of 20Mbit Enet then you are likely to be less than satisfied. If what you wanted is two logically separate 10Mbit Enets then it works just fine. Paul Hyder Network Engineer NCAR
> We just received a Kalpana 1500C Etherswitch and I want to use 2 ethernet > adapters (le0, le1) from the same SS10 (SunOS 4.1.3) on two ports of > the Kalpana. This would give me 2x10Mb/sec throughput on the SS10 and > still use the same network (that is no subnet). > > >From what I read on the archive, it does not seem to be possible because > SunOS will only use le0 in output.
well, SunOS will only use one interface per subnet in output. it doesn't necessarily have to be le0. but you are basically correct, and hence the problem. > They specify that each ethernet adapter should use a different MAC address. > This is configurable with ifconfig by using > 8:0:20:1a:f5:a3 for le0 and > 48:0:20:1a:f5:a3 for le1 > I read that 802.3 permit to "or" 0x40 with the higher address bit to resolve > this issue.
yup doesn't really matter one way or the other so long as you grab some ether addr that isn't in elsewhere on the subnet. > Kalpana also say that internal routing should be disable on the SS10, > so I have to turn off IP_FORWARDING.
shouldn't make any difference, cause the two ports are on the same subnet so it won't forward anyway. > What about the IP address, can I use the same 184.108.40.206 on both > ethernet adapter? (Probably not) Should I use 220.127.116.11 for le0
> and 18.104.22.168 for le1 instead?
this whole area - the idea of having two interfaces on the same subnet - is, shall we say, an unknown. you're not supposed to do it. you're not supposed to be able to do it. Unix doesn't know how to cope with it.
I have some other thoughts on the matter though. Currently working on a paper that I'd like to present at this year's Usenix LISA conference on a way around the problem.
You have to tell the Sun (but NOTHING else) that the network is subnetted, and have half of your clients on one subnet, and half the other (but DON'T tell the clients about the subnetting, so they'll still talk directly to each other, without a router).
while you can do this, you won't get the full benefit of two ethernets for *output*. on input, you will. you need to assign two IP addresses and two names to the interfaces, just like using two nc400s. there is no difference in the config (le, nc, doesn't matter; they've just device names).
make sure that half of your clients use the name on le0 and half use the name on le1, otherwise all traffic will come in on the le0 interface anyway.
-- Real Page ! Real.Page@Matrox.COM Gerant des Systemes Informatiques ! (514) 685-7230 #2359 Systemes Electroniques Matrox ! (514) 685-7030 Fax Dorval, Quebec !
----- End Included Message -----
----- Begin Included Message -----
>From sun-managers Tue Feb 9 05:40:06 1993 Subject: SUMMARY: le0 and le1 on same ethernet
the overall response is that you can better not put the two interfaces on the same ethernet and thus not on the same ip network. My original message is at the end of this mail.
I would like to thanks everyone who responded and especially Hal Stern (who gave the most complete answer to my questions), Mike Raffety and Roar Smith.
stern@sunne.East.Sun.COM (Hal Stern - NE Area Systems Engineer) responded with :
you cannot put two network interfaces on the same IP network. it confuses the routing tables.
each network interface has the same ethernet address. this is because ethernet addresses are assigned *per machine*, not per interface. each network interface has a different network address (IP address). this requires that they be on different networks: you can't have multiple interfaces to the same network.
when you set up le1 on the same network as le0, you create a route to the le0 IP network -- but through le1. different IP address, but same network route. therefore, packets outbound from your machine go over le1 (the most recent route to the IP network). this is likely to confuse things on the other side.
if you want redundant ethernet interfaces, leave le1 turned "off" until you want it enabled. remove the /etc/hostname.le1 file, so it doesn't get ifconfig'd at boot time.
then write a script that periodically tests the network from the machine. if it thinks it should switch over, have it ifconfig le0 down ifconfig le1 up ifconfig le1 hostname ipaddress .....
this will "move" the IP address from le0 to le1, which is what you want.
Mike Raffety <email@example.com> responded with :
No, you can't do this. If you're extremely familiar with TCP/IP and internet addressing, it becomes apparent that this cannot work. Even with different IP addresses on the two interfaces, any BSD-based IP implementation will only be able to use one of the interfaces (due to the way the routing tables are implemented).
I've never yet heard of an Ethernet interface going bad ... if it's really, really important, dual-homed FDDI is the way to go here.
firstname.lastname@example.org (LMD/T/AM Roar Smith) resonded with :
In article AA09130@abcomp.be, wai-kong@abcomp.UUCP (Wai-kong Yu) writes: >Hello everybody, > >Is it allowed on a sunsystem to put both of your ethernetcards (I have >a SS10 with a FSBE/S card, twisted pair) on the same physical ethernet network so that both of them or on the same IP net ? > There should as far as I know be no problem in doing this IF DIFFERENT MEDIA ACCESS CONTROL ADRESSES ARE USED - i.e. different ethernet adresses.
>I tried it and it does work but sometimes I have problems. >e.g. I can not do a telnet or rlogin anymore from my terminal server to >my SS10. The terminal server can find the SS10's ethernetadres but connection >will always fail. If I put the two cards on a different ethernet and IP network, >it does work. Could there be a conflict somewhere ? I know that each >sun has only one ethernet adres no matter how much ethernet cards you >put in your system. Is there a special reason for this ? > The special reason is that Sun is ignoring the one rule, that all ethernet card manufacturers must follow: There must never ever be two ethernet cards (in the whole wide world!) with the same address. It is unthinkable that anyone should break this rule, but alas - the unthinkable happens all too often. The reason (short version) is that the Data Link Layer of your network relies on the MAC (Media Access Code) address of the node to be unique, if this is not the case then the single node (ethernet card) cannot be uniquely addressed on the ethernet.
I have heard (not 100% safe info, I know) that Sun has allocated 20 ethernet adresses for each of their SS10's, so you can/must configure your system with different ethernet addresses on each card, either by hardware configuration (I don't have a SS10, so I don't know if it's possible) or by software configuration. To configure the system with different ethernet addresses on le0 and le1 you can use the "ifconfig" command (in SunOS 4.1.x - I don't know about SunOS 5.x).
Example: -------- Your SS10 has the ethernet address 8:0:20:10:ab:30 (just an example). In the /etc/rc.boot (or rc.local ?) file you use the "ifconfig" command for setting up different ethernet addresses on le0 and le1: # # Set up different ethernet addresses on le0 and le1: # ifconfig le0 ether 8:0:20:10:ab:30 ifconfig le0 ether 8:0:20:10:ab:31 #
That's my $0.02 worth anyway.
My original mail :
Is it allowed on a sunsystem to put both of your ethernetcards (I have a SS10 with a FSBE/S card, twisted pair) on the same physical ethernet network so that both of them or on the same IP net ?
I tried it and it does work but sometimes I have problems. e.g. I can not do a telnet or rlogin anymore from my terminal server to my SS10. The terminal server can find the SS10's ethernetadres but connection will always fail. If I put the two cards on a different ethernet and IP network, it does work. Could there be a conflict somewhere ? I know that each sun has only one ethernet adres no matter how much ethernet cards you put in your system. Is there a special reason for this ?
Another strange thing is that if I ping to le0 or le1 (the onboard ethernet and second ethernet card), it is always le1 who responds. The same for telnet and rlogin. Why ?
I want to be sure that all of my users can still access the SS10 server if there should be a problem with one of my ethernet cards. What is the best system configuration to assure a maximum reability and uptime for my SS10 server ? Online disksuite is alreaddy installed.
Thanks for your help.
A summary will follow later.
e-mail : email@example.com
----- End Included Message -----
------- 8< firstname.lastname@example.org (Eckhard Rueggeberg): > How do we know what the 'real' ethernet address is on the second
interface??? The "real" adress is the same as for any other interface. But you can change it with "ifconfig le1 ether x:x:x:x:x:x:x:x"
Eckhard R|ggeberg email@example.com
------- 8< Michael Assels <firstname.lastname@example.org>: We're doing something similar. We've never been able to find the "real" address of the second interface, but that isn't necessary: You just need something that's distinct from all other addresses in your network. We just modify the first octet of the original address. For example, if the original address of le1 is 8:0:20:a:b:c, change it to 18:0:20:a:b:c as follows:
ifconfig le1 ether 18:0:20:a:b:c ifconfig le1 $hostname netmask ... up
It's probably important to do the address change BEFORE bringing the interface up, so that the switching hub never gets a chance to see the same address on two different ports, but otherwise I'm not aware of any problems.
I hope this helps.
+-------------------------------+----------------------------------+ | Michael Assels, Analyst | Everything is what it is and not | | Comp. Sci., Concordia Univ. | another thing. | | 1455 de Maisonneuve O | - Aristotle (or someone else) | | Montreal, Quebec, H3G 1M8 +----------------------------------+ | Voice: (514) 848-3035 | email@example.com | +-------------------------------+----------------------------------+
------- 8< firstname.lastname@example.org (Dan Lorenzini): Andy,
Yes, ifconfig will do it; use address family "ether" and specify address using the usual 6 hex numbers separated by colons format. It's in the FM. I don't think it causes any problems to have different ethernet addresses on different nets.
I don't know how to find the actual address of the second interface. Before I read your note I had not even thought of it! You might want to ask the vendor how to find out. You could try booting a machiine with only a hostname.le1 and see if it comes up with a different address than you get with both hostname.le0 and le1. At any rate this is not a show stopper, since you can always make up a number and plug it in using ifconfig. As long as it's unique on your net you should have no problems.
I'm interested in the switch that you are using becuase we are going to be getting some switching gear and these are the kinds of things that you only find out after the fact. If you get a chance, I would appreciate your tellimg me a little bit about the switch you have and your experiences with it. THanks.
Dan Lorenzini Greenwich Capital Markets email@example.com 600 Steamboat Road 203-625-7937 Greenwich, CT 06830
------- 8< firstname.lastname@example.org (Glenn Satchell - Uniq Professional Services): There is no "real" ethernet address for second and subsequent interfaces, so just make one up! To set the address use ifconfig like this:
ifconfig le1 ether 8:0:20:xx:xx:xx
The 8:0:20 prefix is assigned to Sun, other manufacturers all have their own prefix as well.
regards, -- Glenn Satchell email@example.com | "This is a unix system. Uniq Professional Services Pty Ltd ACN 056 279 335 | I can do this easy." PO Box 70, Paddington, NSW 2021, (Sydney) Australia | Phone 02 360 7434 Pager 016 287 000 Fax 02 331 2572 | - Lex, Jurassic Park "Sun Accredited System Consultants" |
------- 8< Mike Raffety <firstname.lastname@example.org>: > How is this done??? (Do I have to RTFM? :)
RTFM ("man ifconfig").
> How do we know what the 'real' ethernet address is on the second
There isn't one; the Ethernet address is derived from the same chip on the CPU board with the hostid and serial number.
> Will this cause any problems???
No, as long as it's still unique. But why bother? The switching hub doesn't care. Or maybe I should ask which hub this is? I've tried the Alantec, Lannet, and one other (I think).
------- 8< email@example.com (Dances on keyboards): I can help a little.
How do we know what the 'real' ethernet address is on the second
It really is whatever the host sets it to. The good book of Ethernet specifies that a given node have only a single address, no matter how many Ethernet interfaces it has.
So to determine the real ethernet address on one or more additional interfaces, write down the address of the host they live in, and you're done.
Bikes on the road. Cars in the gutter. Louis M. Brune ANDATACO firstname.lastname@example.org 10140 Mesa Rim Road 619-453-9191 x1601 San Diego, CA. 92121
------- 8< email@example.com (Shelley L. Shostak): Use ifconfig -a to get the "real" ethernet address. I don't think you want to change the ethernet address of the second interface. We had a SS4/65 with three ethernet interfaces. The ethernet number which was used was determined by the location of the SBUS cards. I don't know if that is still true with the newer ROMs. If you "ifconfig -a" as root, you will see the ethernet number of the interfaces.
If you really want to change the ethernet number, ifconfig can be used to do so in one of the rc files.
Good luck, Shelley
-- Shelley L. Shostak (919) 660-2565 Senior Systems Programmer Duke University Dept. of Physics firstname.lastname@example.org Box 90305 Durham, N.C. 27708-0305
------- 8< stern@sunrise.East.Sun.COM (Hal Stern - NE Area Systems Engineer): to change the ethernet address of an interface:
# ifconfig le1 hostname1 ether 48:0:20:a:b:c
what do you mean by the "real" ethernet address? the "real" address is that given to the whole machine. MAC addresses are assigned on a per-machine basis, so all interfaces share the same one. if you change the MAC addresses, ARP takes care of the mapping from IP to MAC addresses for you
This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:08:54 CDT