This problem was caused by a few misbehaving HP-UX NFS clients. I found Solaris bug 6538387, which describes an incompatibility between HP-UX NFS v3 clients and Solaris NFS servers running ZFS. The solution was to umount the NFS file system from the five Hp-UX clients and re-mount it using NFSv2. The CPU load on the NFS server immediatly dropped to near zero and performance is now fine on all NFS clients. Thanks everyone for your help nailing this problem down. Josh On Thu, Aug 7, 2008 at 7:56 AM, Joshua Clark <jclark42@gmail.com> wrote: > Thank you all for your responses so far. I have made significant > progress with this issue. I believe I have narrowed the problem to the > few HPUX NFS clients on our network. We have 120+ NFS clients, a mix > of Solaris, Linux, BSD, HP-UX, AIX, and some others like VMS, etc. > > Looking at "nfsstat -s" I see a huge number of SETATTR calls: > > bash-3.00# nfsstat > <snip> > Server nfs: > calls badcalls > 1488894256 3222 > <snip> > Version 3: (1488672395 calls) > null getattr setattr lookup > access > 2 0% 160776192 10% 1313974792 88% 10634878 0% 1092862 0% > readlink read write create > mkdir > 5336 0% 871124 0% 283688 0% 60968 0% 7594 0% > ... > > I ran snoop for 10 seconds on the GigE interface. It captured 30,059 > packets. Of these, 12,284 were SETATTR calls, and a total of 29,969 > were between the NFS server and our five HP-UX clients. I checked > these systems- there is little or no activity that would warrant this > much traffic. I believe there is some sort of problem between the > HP-UX NFS clients and the Sun NFS server that is causing a packet > storm which is driving the CPU usage on the NFS server. > > Has anyone had any problems similar to this? Any ideas where to turn next? > > Thanks again, > Josh _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagersReceived on Thu Aug 7 10:42:08 2008
This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:44:11 EST