ProFTPD mini-HOWTO

Debugging Problems

Users of ProFTPD will often encounters problems. It happens with all software, not just ProFTPD. How, then, does the user track down the cause of the problem, and fix it? This is the art of debugging. When users post these problems are the mailing lists, it is extremely helpful to include the following bits of information to help find the answer. Even better is when the user follows these steps and determines the solution for themselves.

Know the Version
Various problems afflict various versions of the code, so when tracking down problems, it is good to know the version being used:

  proftpd -vv

It is possible that the problem you are encountering is due to some bug that may already be fixed in a more current version, fixed in the CVS repository, or has a bug report with an attached patch. Searching http://bugs.proftpd.org will often yield useful information, depending on the keywords used in the search.

Know the Modules
System administrators who compile and install the server from the souce code distribution will probably already know this, but administrators who install using RPMs or other package formats may not know the specifics of the contained pre-built binary. To list the modules compiled into the server:

  proftpd -l

Knowing the modules helps to pinpoint the source of error messages (e.g. mod_tls and certificate files).

Perform Configuration Check
When making changes to the configuration file, it is often helpful to make sure that your changes are valid. The easiest way to do this is to do an informative syntax check:

  proftpd -td5

The -t option directs the server to only do a syntax check, to parse the configuration file but stop before actually starting its operations as a server. The -d5 will cause the server to display debugging messages during this testing of the configuration file. Another useful command is:

  proftpd -c /path/to/new/config/file -td5

which lets you test the syntax of some new configuration file before it is put into production. If you are still having problems, and you have verified that your proftpd.conf is correct, the next step is see what debugging messages are generated during an FTP session, as described next.

Collect Debug Information
proftpd has built-in debug information reporting capabilities - the trick is in enabling that reporting, and tracking down where it is sent. The easiest way to get the debugging information is to start the server from the command line using:

  proftpd -nd6

Note: make sure that no other proftpd instances are running before using this command, otherwise you will see:

  Failed binding to 0.0.0.0, port 21: Address already in use

This message means that some other program is already bound to the socket at that address/port. This message will also appear if you attempt the debugging command with a ServerType of inetd; this is described below.

Once working, the above debugging command will display lots of information on the connected terminal's screen, both as the server starts up and during the servicing of any clients. If clients are having trouble logging in or authenticating, the debug messages reported by the server when a client connects are much more useful than knowing the messages displayed by the client, as the client does not know why it cannot log in. If asked to send debugging information to the mailing list, you can send the relevant snippets (if you know what the relevant debug messages are), or you can capture the debug output to a file:

  proftpd -nd5 2>&1 >& /path/to/debug/file

and send that file, compressed, along with your post.

The above method works if you have ServerType standalone in your configuration file. If you run the server in inetd mode instead, and are unable or unwilling to make the changes necessary to run the server in standalone mode for the sake of debugging, then use of the SystemLog configuration directive is necessary for capturing the debug information. Add this directive to your configuration file, and add -d5 to your /etc/inetd.conf's ftp line, or to the server_args tag in your xinetd configuration file for the server. Be sure to restart inetd/xinetd so that your configuration changes will take effect.

Note that use of the SystemLog directive is not necessarily confined to inetd mode servers. If you are interested in letting your standalone server run unattended and want to have that debugging information in the log file, use SystemLog and add -d5 (or whatever your preferred debug level is) to the server startup script.

As of version 1.2.8rc1, ProFTPD supports a DebugLevel configuration directive. This lets you set a debugging level in your proftpd.conf file, without needing to edit inetd.conf or xinetd configuration file.

Locate Log Files
A common response on the mailing lists to a posted question is: "What do your server logs say?" Locating the server's log files can be troublesome, depending on your configuration. If the SystemLog configuration directive is in effect, you know exactly where the server's log file is. If not, then by default the server uses syslog for logging. The location of syslog'd log files is set in your system's /etc/syslog.conf file. You may need to read your system's man pages for syslog.conf or syslogd to understand the format of that file. Note that the server will log using a syslog facility of daemon (and level debug when debugging) for most of its messages; during authentication, messages are logged using the authpriv facility.

Debugging Segfaults
If you see a message like the following in your logs:

  golem.castaglia.org (127.0.0.1[127.0.0.1]) - ProFTPD terminating (signal 11)

then you have most likely encountered a segfault. This is a particular bad error, and can be hard to track down.

In version 1.3.1rc1, ProFTPD gained the ability to provide better logging to help track down these sorts of bugs. To enable this ability, you will need to configure proftpd with something like:

  ./configure --enable-devel=stacktrace ...

and run your proftpd like normal. If a segfault occurs, the logs should show something like this:

  golem.castaglia.org (127.0.0.1[127.0.0.1]) - ProFTPD terminating (signal 11)
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - FTP session closed.
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - -----BEGIN STACK TRACE-----
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [0] ./proftpd [0x809b1e1]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [1] ./proftpd(call_module+0x53) [0x8072c63]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [2] ./proftpd(strftime+0x14cf) [0x8051bef]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [3] ./proftpd(pr_cmd_dispatch+0x167) [0x8051f2f]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [4] ./proftpd(strftime+0x1fd3) [0x80526f3]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [5] ./proftpd [0x8053e12]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [6] ./proftpd [0x805484d]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [7] ./proftpd [0x8057975]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [8] ./proftpd(main+0x9d1) [0x8058625]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [9] /lib/i686/libc.so.6(__libc_start_main+0x93) [0x40076507]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - [10] ./proftpd(strcpy+0x31) [0x8051001]
  golem.castaglia.org (127.0.0.1[127.0.0.1]) - -----END STACK TRACE-----

These stack symbols are generated automatically by glibc, and may or may not contain the function names.

The key here for tracking down the location of the segfault is that [0] frame, and the memory address: 0x809b1e1. Using that address and a very handy command called addr2line, you can determine the location of that address in the source code:

  addrline -e ./proftpd 0x809b1e1

In this example, I saw:

  golem/tj>addr2line -e ./proftpd 0x809b1e1
  /home/tj/proftpd/cvs/proftpd/modules/mod_auth.c:1723

which is the location of test code added to trigger the segfault.

This feature is specific to glibc systems, so keep that in mind. Other things to mention about this feature: it disables the changing of the process title that proftpd normally does. The obtaining of the stack symbols that glibc does uses the process title, and if this feature did not disable that process title changing, the output would be much less legible. Also, it does not work if the process which receives the SIGSEGV signal is chrooted. (Blame glibc for its bad handling of being chrooted.) To work around this, I would recommend using the mod_vroot module, just for a short period of time, in order to get a useful automatic stack trace.

Common Problems
One common question is: "I changed the configuration file, but the new configuration is not being seen!" The solution depends on your configured ServerType. Almost certainly it will be standalone, as inetd-mode servers pick up configuration changes almost instantly (the server is started from the ground up for each connection). For configuration changes to be seen by a standalone server, you need to either stop, then start the server (the hard way), or to send the HUP signal the the daemon process.

Another common question involves use of ProFTPD's <Limit> directive to restrict certain FTP commands. These limits always function in addition to the normal filesystem permissions, not instead of them. If having problems writing, deleting, or updating files, check your directory and file permissions first.

Once you have the debug output and various other information, and are still in need of help, search the FAQ, Userguide, and mailing list archives for material related to the problem. If you're unable to find anything helpful in these sources, post your question to the appropriate mailing list. Be sure to include the version used, your proftpd.conf, and possibly any debug information.

The following document describes how to ask good questions that are likely to be answered:

  http://www.catb.org/~esr/faqs/smart-questions.html

Last Updated: $Date: 2006/04/16 22:35:55 $