Tadeu Bento

Fetching E-mail Into a Local Server

@Haui@discuss.tchncs.de asked me an interesting question: how could he pull emails from multiple providers such as Gmail and Outlook into his home server? This article assumes you have working experience with Linux and can set up a local Dovecot and Postfix server to handle your e-mail.

Why would anyone do this?

Cloud providers impose storage limits, and you may have dozens of email accounts while also have concerns about entrusting your data to those providers. The idea behind this request is to set up a local server on a low-power device like a Raspberry Pi and have this server regularly download all new e-mails from Gmail and Outlook into its local storage. Subsequently, the local server will act as the user’s primary server for both receiving and sending emails.

Why pull instead of forwarding?

Most providers allow users to forward e-mail to other accounts and while this feature could be used to forward the e-mail to the user’s home server, it would require the the latter to have a valid e-mail address exposed to the internet, an SMTP port open, a good uptime and a very strong protection against SPAM. None of those are easy to setup nor to maintain and some ISPs don’t even allow it.

To circumvent those limitations, we can simply implement polling. This involves regularly querying Gmail/Outlook to check for new e-mails, and if any are found, download them to the local server. There are multiple programs that can accomplish this task:

The first step to install go-getmail as described on the link above and the configuration file should look like this:

Accounts:

- Name: GMail
  Source:
    IMAP:
      Server: imap.gmail.com:993
      Username: your-emai@gmail.com
      Password: your-gmail-password-or-app-password
      Mailbox: Inbox
  Target:
    IMAP:
      Server: localhost:993
      Username: local-email-address@homeserver.example.org
      Password: your-local-email-password
      Mailbox: INBOX

Note that Google doesn’t always allow direct access IMAP with your account password. In some cases you’re required to create an “app password” as described in here.

Routing Outgoing E-Mail

In order to route outgoing e-mail through the right provider based on the from address we can use a Postfix feature called “sender dependent relayhost maps”. Here’s an example of a postfix configuration:

main.cf:

smtp_sasl_auth_enable = yes
smtp_sender_dependent_authentication = yes
smtp_sasl_password_maps = mysql:/etc/postfix/virtual/mysql-external-alias-credentials.cf
sender_dependent_relayhost_maps = mysql:/etc/postfix/virtual/mysql-external-alias-relay-hosts.cf
smtp_sasl_security_options = noanonymous

mysql-external-alias-credentials.cf:

user = XXXXX
password = XXXXXX
hosts = 127.0.0.1
dbname = mailserver
query = SELECT concat(relay_user, ':', relay_passwd) AS credential FROM `Virtual_Alias_External` WHERE alias='%s' AND active = 1;

mysql-external-alias-relay-hosts.cf:

user = XXXXX
password = XXXXXX
hosts = 127.0.0.1
dbname = mailserver
query = SELECT relay_host AS transport FROM `Virtual_Alias_External` WHERE alias='%s' AND active = 1;

MySQL table structure:

CREATE TABLE `Virtual_Alias_External` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `alias` varchar(70) NOT NULL,
  `owner` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  `active` tinyint(1) NOT NULL DEFAULT 0,
  `relay_host` varchar(70) NOT NULL DEFAULT '',
  `relay_user` varchar(70) NOT NULL,
  `relay_passwd` varchar(70) NOT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci ROW_FORMAT=COMPACT;

Your entries should look like this:

alias: your-emai@gmail.com
owner: local-email-address@homeserver.example.org
active: 1
relay_host: [smtp.gmail.com]:587
relay_user:  your-emai@gmail.com
relay_passwd: your-gmail-password-or-app-password

Whenever you send an e-mail and the from is set as your-emai@gmail.com Postfix will route the email through Gmail’s SMTP server using the stored credentials. If this is done correctly (smtpd_sasl_authenticated_header = no) no references to local-email-address@homeserver.example.org will show up on the outgoing e-mail headers.

Exit mobile version