Troubleshooting common NFS issues in Linux

The post discusses most commonly occurring NFS issues in Linux and how to resolve them.

1. Error: “Server Not Responding”

The Network File System (NFS) client and server communicate using Remote Procedure Call (RPC) messages over the network. Both the host->client and client->host communication paths must be functional. Use common tools such as ping, traceroute or tracepath to verify that the client and server machines can reach each other. If not, examine the network interface card (NIC) settings using either ifconfig or ethtool to verify the IP settings.

The NFS file system also reports “server not responding” when a heavy server or network loads cause the RPC message responses to time out. Use the “timeo=N” mount option on the client to increase the timeout. Check “man mount” for more information.

2. Error: “No route to host”

The “no route to host” error can be reported when the client attempts to mount an NFS file system, even if the client can successfully ping the server:

# mount NFS-Server:/data /data_remote
mount: mount to NFS server 'NFS-Server' failed: System Error: No route to host.

This can be caused by the RPC messages being filtered by either the host firewall, the client firewall, or a network switch. Verify if a firewall is active and if NFS traffic is allowed. Normally nfs is using port 2049. As a quick test one can switch the firewall off by:

# service iptables stop

on both the client and the server. Try mounting the NFS directory again. Do not forget to switch it back on and configure it correctly to allow NFS traffic/

3. Error: “mount clntudp_create: RPC: Port mapper failure – RPC: Unable to receive”

The Linux NFS implementation requires that both the NFS service and the portmapper (RPC) service be running on both the client and the server. Check it like this:

# rpcinfo -p
  program vers proto   port
  100000    2   tcp    111  portmapper [portmap service is started.]
  100000    2   udp    111  portmapper
  100011    1   udp    881  rquotad
  100011    2   udp    881  rquotad
...
# service portmap status
portmap (pid 7428) is running...   [portmap service is started.]

If not, start it with the commands give below.

# chkconfig portmap on
# service portmap start

4. Error: “NFS Stale File Handle”

A program uses the open(2) system call to access an NFS file in the same way the application opens a local file. This system call returns a file descriptor, or “handle”, that the program subsequently uses in I/O commands to identify the file to be manipulated.

Unlike traditional Linux file systems that allow an application to access an open file even if the file has been deleted using unlink or rm, NFS does not support this feature. An NFS file is deleted immediately. Any program which attempts to do further I/O on the deleted file will receive the “NFS Stale File Handle” error. For example, if your current working directory is an NFS directory and is deleted, you will see this error at the next shell prompt.

To refresh the client’s state with that of the server you may forcely unmount the mount point:

# umount -f /mnt/mount_point

or kill the process, which references the mounted file system:

# fuser -k [mounted-filesystem]

5. Error: “Access Denied” or “Permission Denied”

Check the export permissions for the NFS file system. You can do this from the client:

# showmount -e server_name

or from server:

# exportfs -a

If you see unexpected export permissions, check the /etc/exports file on the server. Make sure there is no syntax error such as space between the permitted host and the permissions. There is a significant difference in the line:

/home *(ro)

and the line:

/home * (ro)

because the second exports /home read-write to all systems: not what was intended. Note that the line still has correct syntax, so NFS will not complain about it.

6. Error: “rpc mount export: RPC: Timed out”

Error message:

Unable to access file system at [NFS SERVER]: rpc mount export: RPC: Timed out

This is caused by DNS name resolution issue. NFS(RPC) needs reverse name resolution. If NFS server or client cannot resolve their name, this error occurs. In case gets the error message, check DNS configuration and /etc/hosts configuration.

Related Post