Mail Gateway
From Halon Security
This document is about the Web Administration's Mail Gateway sections. The mail gateway sections are available in both the SPG and VSP, since they are per definition; mail gateways.
Block chart overview
The SPG/VSP are specialized mail (SMTP) gateway products, which focuses around the problem of assuring that every genuine e-mail is delivered, while preventing attacks, spam and viruses. Below is a chart that outlines which part of the H/OS 2 operating system is involved during each stage of an SMTP session.
Sections of the Mail Gateway administration
The following sections describe each of the mail gateway sections, found in the Web Administration.
Incoming SMTP Listeners
The section incoming allows users to configure mail listeners (recievers), and display the incoming queue. The incoming queue lists messages that are awaiting processing by the mailscanner process. A mail listener is essentially a mail server, listening to a port (usually 25).
An incoming SMTP listener is a server object, bound to listen (accept connections on) either all addresses configured on an appliance, or just a few addresses. They are pre-configured to listen on port 25, since it is the mail delivery (SMTP) port by definition. A listener only accepts a message if it's recipient address' domain is configured on the appliance, and assigned to that specific listener.
Direct processing
Normally, a message is scanned "inline", in order for you to reject it (give an error message like "550 We think this is spam" response). However, by disabling direct processing, messages can be placed in the incoming queue, available at Mail Gateway → Activity → Incoming tab. The incoming queue is continuously processed by the mailscanner, according to the assigned mail content flow.
Connection-level protection
If a IP Policy is used, a connection must be allowed by the IP Policy flow before it's accepted; providing connection-level filtering. IP Policies are configured on Security → IP Policy, providing lots of features such as rate control, black and white list, DNSBL and GlobalView.
GlobalView is a subscription service, that typically stop more that 80% spam on connection level. This means that the spammer does not even get to talk to the appliance; it is instantly blocked. GlobalView also protects against botnets and other threats.
SSL/TLS Support
To activate TLS, goto Mail Gateway → Incoming and activate the advanced option "Support TLS" for your incoming listener. Read more about the use of TLS and PKI.
You may also use TLS for outgoing traffic, see TLS Certificates Section.
SASL Authentication
You may require users to authenticate before they will be able to send mail outbound (or inbound) just like a SMTP server. Since there is no user database in the SPG/VSP we provide ways to externally authenticate users using: SMTP Forwarding Authentication, LDAP, FTP/FILE or HTTP. We support PLAIN and LOGIN authentications.
This is a two step process. In the Authentication Flow, the users tries to authenticate, and if succeeded you may process his email differently in the Recipient Flow. Below follows an example for SMTP Forwarding Authentication only for you to understand that process behind it, you are normally not required to use scripting modules to achieve this since it can be done using other modules.
Authentication Flow:
// SMTP Forwarding Authentication
if (smtp_lookup_auth("mailtransport:1", $saslusername, $saslpassword) == 1)
{
echo "$saslusername successfully SMTP authenticated";
Accept();
}
Reject("You failed ($saslusername)");
Recipient Flow:
// Only allow if user is authenticated
if ($saslauthed)
Accept();
else
Reject("This server require authentication");
SMTP Forwarding Authentication
SMTP Forwarding Authentication work in the way that it's using an existing mailtransport (lookup-mx is not supported) and tries to authenticate itself usign the provided username and password.
Example
Script:
if (smtp_lookup_auth("mailtransport:1", $saslusername, $saslpassword) == 1)
{
echo "$saslusername successfully SMTP authenticated";
Accept();
}
LDAP Authentication
LDAP Authentication validates a user by trying to bind the provided username and password.
LDAP Block Example
In the Authentication Flow, the first block is a LDAP block. You can select pre-defined LDAP profiles from the Mail Gateway -> LDAP section. What will happen, is that $saslusername will overwrite the "Username (DN)" field, and $saslpassword the "Password" field. Since you can authenticate against LDAP using the "User Principal Name" (UPN) format (user@domain) as well as the DN format, this works well.
Script Example
Script:
if (ldap_bind("ldap:4", $saslusername, $saslpassword))==1)
{
echo "$saslusername successfully LDAP authenticated";
Accept();
}
File Authentication
File Authentication can be used to lookup the username/password in a file.
File Block Example
In the Authentication Flow, the (currently, in version 1.0.7.1) third block is a File block. Simply upload a ASCII-like text file to the appliance's FTP (put it the FTP-root) with the following syntax:
username1 password1 username2 password2 username3 password3
and type the filename, (for example "auth.txt") into the File blocks text field, press "Done", and then "Save".
Script Example
Upload a file called accounts.txt to the FTP (put it the FTP-root) with the following content...
username1 password1 username2 password2 username3 password3
Script:
if (count($info=in_file($saslusername,"file://accounts.txt")) == 2 and $info[1] == $saslpassword)
{
echo "$saslusername successfully FILE authenticated";
Accept();
}
HTTP Authentication
HTTP Authentication can be used to lookup the username/password against a web service.
Example
if (http("http://test.example.org/smtp_authentication.php?username=$1&password=$2", 10, $saslusername, $saslpassword) == "OK")
{
Accept();
}
Domains
In order for an incoming SMTP listener to know which domains to relay for, domain objects needs to be defined and associated with a listener. Even when using LDAP or a text file as recipient database for the incoming listener, domains are still required since the mailscanner (employing the process flows) needs to know which one to use. A so-called "any" domain is available, if no domain-related associations shall be made.
Domains Tab
A domain is the core of the mail configuration, it connects all parts together (incoming, flow, transport) in a unique combination of incoming and domain. That means that you can have the same domain defined multiple times on different incoming listeners. The search of the domain to use is narrowed down by searching for the most specific listeners and then domain to use; a defined IP is matched before "any", same goes for domains, a defined domain is matched before "any" (just like an IP routing table).
Recipient Flows
In the recipient flow, you may at an early stage check SPF record, lookup users against LDAP etc. If no "Recipient Flow" has been configured for a listener, it accepts mail messages to all users on the domains assigned to it. The appliance can however look up specific users on a domain, by issuing either LDAP or a text file or Mail Recipient flow. Recipient flows are chosen per domain; but the default "SPF and Recipient" flow is very good, and can usually be chosen for all domains.
Domain Alias Tab
The relationship between domain alias and parent domain is that the domain alias inherit all settings from the parent domain. Also, quarantine users with equal names will be automatically joined between the alias and parent domain.
Mail Content Flows
Mail Content Flows are selected per domain, on the "Domains" section. They process the mail, once the mail data (the SMTP command "DATA") is available. Hence, mail content flows have more parameters available than for example IP Policy Flows, which only has a IP packet object to operate on.
Objects (modules) in a mail content flow can be added, removed or re-ordered. If the pre-defined objects prove insufficient, you can use the "Script" module to write your own HSL code, with the commands available in the Core and Mail Content extension.
Custom Icon on Script Blocks
In the first comment specify an icon using --webui-icon=http://url.
// My first block --webui-icon=http://example.org/images/internet-mail.png echo "Hello World";
Quarantine
The quarantine temporary store messages in order for end-users to release them.
Outgoing
The outgoing section contains SMTP transport configuration and monitoring. An SMTP transport is handled by the queueprocessord service, responsible for delivering mail upon the Deliver() script function; e.g. the mail was neither deleted or quarantined.
Outgoing SMTP Transport Tab
The final step in the process of processing mail is the delivery. If you are using the SPG for inbound delivery you should provide the IP-address of your receiving mail server, since it's very likely the MX record points to the SPG, and anything else would cause a mail loop. If you're using the SPG for outbound delivery, you probably want to use the "Use MX Record" feature.
Lookup MX
If the SPG/VSP is used in outbound mode you may want to use lookup-mx. The lookup-mx has two additional features. It can append a zone prefix and you may specify which DNS server to use for all DNS queries, if you for some reason don't want to use the system-wide default.
In eg. larger "point-your-mx-to-us-and-we-scan" hosting installations, instead of adding a transport for each customers real mail server. You may trick the lookup-mx command to do look ups against a DNS server storing this information and get the MX you want.
lookup-mx:zone=customer.example.org;query=ns1.example.org,ns2.example.org
Lookup MX by appending a ZONE
The SPG/VSP may automatically append a zone prefix to all DNS queries for a transport. This is useful if you want to store MX transport information in your existing DNS server. You should be aware of that anyone who knows your "system" may find your customers real mail server, therefore in bind you may restrict a zone to only the SPG.
lookup-mx:zone=customer.example.org
Lookup MX using alternative DNS servers
If you're using an alternative DNS server, there is no need to put the domain in a specific zone. You may use the example above as a reference but it is not a complete working configuration.
lookup-mx:query=ns1.example.org,ns2.example.org
Retry and Give Up
If the SPG failed to deliver a mail it will be queued for a delivery at a later time depending on the "Retry Delay" and "Retry Count" settings. Once the Retry Count is exceeded the SPG can either Generate a DSN (see the "Generate DSN" feature) or just discard the mail and do no further actions. The RFC for SMTP propagates long delays and large retry counts; that is since the SMTP protocol is a very old protocol and back in the days of dial-up internet mail servers was not always online, so large values prevented mails from being lost and that they had to be resent all the time.
We think it's a good idea to lower these values to have at most 1-2 days before giving up. If you know your mail server will be down for more you can always pause the delivery process in the SPG; and once the server is back online you can continue like nothing happened.
Values are given in minutes.
Outbound Delivery
If you are using the SPG for outbound delivery, it's recommended to use a very small "Retry Delay" (eg. 1,3,5) and "Retry Count" (eg. 3) and use the "Generate DSN" feature in order to notify the sending client that we failed to deliver the mail within minutes.
TLS Certificates
While delivering mail you may choose to use TLS, in order for TLS to prevent a man-in-the-middle attacks you should use "Require and Verify", this verification requires you to install the remote public certificate on your system. Goto Security -> Certificates and install a new "Remote Certificate" containing the public certificate. You may fetch a certificate using [OpenSSL] (warning, this is not a secure way to obtain a certificate if you already is affected by a man-in-the-middle attack) instead you should extract the certificate from the original certificate installed on the server, but that is not covered in this documentation.
openssl s_client -starttls smtp -connect smtp.google.com:25 -showcerts
Use the first "-----BEGIN CERTIFICATE-----" section.
SASL Authentication
Some server/relay servers require you to authenticate yourself using SASL in order by be able to send outbound mail. We support CRAM-MD5, LOGIN and PLAIN authentication. Username and Password are specified for each outgoing transport.
LDAP
If you're using Microsoft Exchange or Icewarp/Merak a LDAP profile may look like this.
| Parameter | Value | Exchange (IAS) Example | IceWarp/OpenLDAP |
|---|---|---|---|
| Name | User defined name | My LDAP | My LDAP |
| Server Address | Address of LDAP Server | 10.0.0.5 | 10.0.0.5 |
| Username (DN) | Distinguished Name | cn=username, ou=company, dc=example, dc=org | cn=admin, dc=root |
| Password | Password | mysecretpassword | mysecretpassword |
| Search Base (DN) | Distinguished Name | dc=example, dc=org | dc=root |
| Query Filter | Query Filter | (proxyAddresses=smtp:%s) | (mail=%s) |
Logging and History
Activity
Incoming Queue
The Incoming Queue consists of mail that has not yet been scanned by the mail scanning process. This queue can receive mail much faster than the mail scanning process can process mail, helping the SPG to handle large bursts of mail messages. Once the storage is getting full, it will start to defer messages until the SPG has delivered and removed messages.
Outgoing Queue
The Outgoing Queue consists of mail that has been scanned by the mail scanning process but not yet delivered. Once the mail has been delivered it will be moved to the History. If a mail cannot be delivered the reason can be shown by pressing the explanation mark button. If a mail is stuck in this queue is might be because.
* The mail server is down * The mail server has a virus/spam protection that denies the delivery * The mail server has a more strict policy (that can be. "sender domain must exists")
If your queue gets filled up with mail that cannot be delivered to the mail server, it might be a good idea to lower the "Retry Count" since the chance of that the mail will be accepted in the near future is not very likely.
History
This page shows messages that has been delivered, in combination with some information about spam score etc etc. By pressing the "Show in log" button it will be found in the log.
Logging
This will show the log for a message, spawn over the mailscanner and queueprocessor. Syntax is messageid, messageid:queueid or just queueid.
| Syntax | Example | Description |
|---|---|---|
| 127.0.0.1 | 127.0.0.1 | Searches using a plain free text search |
| messageid | 43de929d-cc22-11dd-90ef-0048546ae42b | Searches for a message id |
| messageid:queueid | 43de929d-cc22-11dd-90ef-0048546ae42b:432360 | Searches for a message and a specific queue id |
| queueid | 432360 | Searches for a queue id |
| /<regexp>/ | /127\.0\.0\.1/ | Searches using a Regular Expression |
