How to Set Up Authentication for Sendmail SMTP server

Posted on by

Sendmail is a mail server that supports many different protocols including SMTP.  SMTP protocol handles delivering emails (but not receiving them).

This article details how to setup Plain authentication on a sendmail server on Fedora.

Install sendmail

yum install sendmail

Install SASL. SASL handles the authentication for sendmail.

yum install cyrus-sasl
yum install cyrus-sasl-plain

Add an OS user. The following creates a “fake” user that does not have a home directory and is unable to login to the OS. Its credentials are only used to authenticate with the SMTP server.

adduser -M -s /sbin/nologin userName

Start your SASL authentication daemon if it hasn’t already started.

service saslauthd start

Start your sendmail server if it hasn’t already started.

service sendmail start

Test SMTP Server

You can verify the SMTP server is working by sending yourself an email.  Use telnet to give direct SMTP commands to your SMTP server via commandline.  See http://www.freebsdwiki.net/index.php/SMTP,_testing_via_Telnet#LOGIN_method.

You will need to encode your SMTP username and password in base64 encoding. You could use openssl, or the perl encode_base64 function, but I find it easier to type your text into the graphical encoder from http://www.webpan.com/Customers/Email/base64_conversion.htm and click “encode”.

Troubleshooting

  • If telnet can not establish a connection with your SMTP server, you may be blocked by a firewall.  Try executing telnet directly from the machine hosting the SMTP server.  Also, use localhost instead of the server name.  Ensure that the SMTP port is not blocked by iptables, ufw, or any other firewall systems.  The SMTP port is 25 by default.
  • If you keep getting authentication issues, try different base64 encoders or different parameters to your base64 encoder.  I found http://www.webpan.com/Customers/Email/base64_conversion.htm was eaiest to use and did not require any parameter manipulation.
  • If you use strace to monitor the system calls from sendmail and you notice PAM error messages, then the SMTP user does not have an OS account properly setup.  Ensure that the SMTP user has an OS account with the same username and password.

Logging

  • sendmail logs to /var/log/maillog by default.  You can alter what, where, and how much gets logged by executing 
    sendmail -D (logfile) -d(flag).(level)

    The flag specifies what gets logged. The level specifies how verbose. See http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/sendmail_debugflags.htm for details.