Well, We finally resolved this issue. As it turns out, having the DEPRECATED flag set on the virtual interface was causing the problem. Removed the flag from the virtuals and everything is working as advertised. Thanks to all who offered advise. -Sean -----Original Message----- From: Cheney, Sean [mailto:sean_cheney@groton.pfizer.com] Sent: Tuesday, August 13, 2002 5:57 PM To: 'sunmanagers@sunmanagers.org' Subject: SYN recieved, no SYN+ACK on Solaris 2.8 Hello sunmanagers, We are having a terrible time troubleshooting a problem. The Issue is, when connecting to virtual address on the machine, the server receives the TCP SYN, but fails to respond. However, and this is the confusing part, the machine will begin responding if, while attempting to connect, you send another traffic stream to either the virtual interface, or the primary interface. We have observed the behaviour via snoop. The snoop is a little hard to read, but follow along. Here's my initial SYN... 0.00989 client-> server TCP D=11009 S=2535 Syn Seq=1905265301 Len=0 Win=16384 Options=<mss 1460,nop,nop,sackOK> here's the second one after no response.... 0.00644 client-> server TCP D=11009 S=2535 Syn Seq=1905265301 Len=0 Win=16384 Options=<mss 1460,nop,nop,sackOK> and another....Note that the Seq number is not changing.... 0.01424 client-> server TCP D=11009 S=2535 Syn Seq=1905265301 Len=0 Win=16384 Options=<mss 1460,nop,nop,sackOK> Ok, so now I send a ping through to the same IP address I am attempting to connect to on port 11009... 0.00373 client-> server ICMP Echo request (ID: 1024 Sequence number: 32512) And here we see the server responding instantly to my ping request.... 0.00002 server-> client ICMP Echo reply (ID: 1024 Sequence number: 32512) Followed immediately by a SYN+ACK to my initial SYN... 0.00004 server -> client TCP D=2535 S=11009 Syn Ack=1905265302 Seq=1054349554 Len=0 Win=64240 Options=<nop,nop,sackOK,mss 1460> And finally by a client ACK, TCP handshake complete... 0.00305 client -> server TCP D=11009 S=2535 Ack=1054349555 Seq=1905265302 Len=0 Win=17520 After this, things flow as expected. Some helpful information.... The process bound to that port is BEA Weblogics(admin port) We are observing this problem on 1.1.1.230 and 1.1.1.164. here is the servers interface information, address changed to protect the innocent(and my job). ^C# ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 1.1.1.170 netmask fffffc00 broadcast 1.1.1.255 groupname link1 ether 0:3:ba:1:de:ad hme0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 1.1.1.171 netmask fffffc00 broadcast 1.1.1.255 hme0:2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 1.1.1.164 netmask fffffc00 broadcast 1.1.1.255 hme0:3: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 1.1.1.230 netmask fffffc00 broadcast 1.1.1.255 hme0:4: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 1.1.1.163 netmask fffffc00 broadcast 1.1.1.255 This interface is into a backup network... qfe0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 2.2.2.18 netmask fffffc00 broadcast 2.2.2.255 ether 8:0:20:c6:fa:35 And the backup wire. qfe1: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STA NDBY,INACTIVE> mtu 1500 index 4 inet 1.1.1.172 netmask fffffc00 broadcast 1.1.1.255 groupname link1 ether 8:0:20:c1:be:ef here is the routing table. # netstat -nr Routing Table: IPv4 Destination Gateway Flags Ref Use Interface -------------------- -------------------- ----- ----- ------ --------- 1.1.1.0 1.1.1.170 U 1 184 hme0 1.1.1.0 1.1.1.170 U 1 0 hme0:1 1.1.1.0 1.1.1.170 U 1 0 hme0:2 1.1.1.0 1.1.1.170 U 1 0 hme0:3 1.1.1.0 1.1.1.170 U 1 0 hme0:4 1.1.1.0 1.1.1.170 U 1 41 qfe1 2.2.2.0 2.2.2.18 U 1 19 qfe0 224.0.0.0 1.1.1.170 U 1 0 hme0 default 1.1.1.253 UG 1 552 127.0.0.1 127.0.0.1 UH 2 928 lo0 Stuff I have ruled out, but am willing to look into again. L3 routing issues ARP and CAM table issues There are several hundred other servers on this subnet working fine, so we are not of the belief that there is a larger network problems happening here. To summarize, when connecting to the virtual interfaces, the server fails to send the SYN+ACK unless some other traffic flow(from the same source IP address) goes through. It takes about 20 minutes for the problem to re-occur if you stop sending traffic to the machine. This machine is functioning perfectly in every other respect and is otherwise healthy. I am looking for anyone who has witnessed behaviour like this. We are at our wits end with this problem, if anyone can help us out it would be great. Cheers, Sean LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately. _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagers _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagersReceived on Thu Aug 15 21:15:26 2002
This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:42:52 EST