Recently I was unfortunate to witness huge filesystem loop on a cPanel server that was caused by combination of VirtFS and Idera’s Server Backup software (previously known as R1soft CDP backup). To cut the chase, I had to disable and remove VirtFS for all users on the server. cPanel has documented how to do this on their wiki, but they failed to mention couple of very important things that will help you not to destroy your server.
So, cPanel provided us with nice little script that can unmount orphaned VirtFS mounts. The script is located at /scripts/clear_orphaned_virtfs_mounts and you can get to know it better if you run it with --help switch. Running the script with no arguments will simply unmount obsolete VirtFS mounts (e.g. for terminated account), but if you have to remove VirtFS for an account that is still active, you will have to take a different approach.
The first thing you have to do is check if your account still has JailShell as default shell.
# grep username /etc/passwd
If it does, you’ll need to change it to Bash or any other shell. For example:
# usermod -s /bin/bash username
Now you must check if the user has JailShell defined in its crontab.
# crontab -l -u username
If you notice SHELL="/usr/local/cpanel/bin/jailshell" in the output, you’ll need to edit user’s crontab and delete that line. To access user’s crontab while logged in as root, use:
# crontab -e -u username
OK. Now you are certain that the user has new default shell and that his cron jobs won’t be executed in JailShell. Before going any further, you should check if the user has been logged in the whole time and it’s still logged in in JailShell.
# w | grep -i username
If it is, you should inform him to log out or, if you feel like it, you can log him out by yourself. After all, everybody should bow before you, for you are root.
# skill -KILL -t pts/###
Note that you will have to replace ### with user’s pts number.
Right, now you can unmount all active VirtFS mounts.
# /scripts/clear_orphaned_virtfs_mounts --clearall
Don’t worry. Unmounting VirtFS isn’t non-reversible procedure. If the user logs back into the server and has JailShell as his default shell, or if his cron job (with defined JailShell) executes, JailShell will automatically remount VirtFS. Actually, this is the reason why you went through the steps explained so far.
If cPanel’s script did it’s job, VirtFS for your user should be unmounted. You can verify this with:
# grep -i username /proc/mounts
If grep didn’t return any matches, you can continue to final check. Even though you unmounted VirtFS, there will still be some files and directories in /home/virtfs/username/. This is OK. But to make be completely sure there aren’t any bind mounts or rogue hard links, it’s wise to power up find command and look for files that have more than one link (it checks inodes).
# find /home/virtfs/username/ -type f -links +1 -ls
The expected output should be, well, no output at all. Meaning that there aren’t any files with multiple hard links.
Finally, you can delete user’s VirtFS directory (/home/virtfs/username/).