Automating Linux Package Upgrades Using Cron Apt and Postfix

As the number of servers under our control grows, we want to be able to improve our control over package upgrades.

The problems here are two-fold:

  • we want to know which packages are ready to be installed
  • we want to have these packages already downloaded so we can install them when we're ready

However, we don't want to fully automate the upgrading of these packages, as:

  • there may be packages that want to overwrite existing configuration, so we'll need to figure out what the best couse of action is
  • there may be packages that we don't want to install

As a solution, we'll use cron-apt in conjunction with the postfix mail server to inform us of package updates that are ready to apply.

cron-apt

cron-apt allows us to automate apt-get commands. The documentation is a little sparse, though some writeups do exist here and here.

Installing cron-apt:

apt-get install cron-apt

In /etc/cron-apt/config:

# Configuration for cron-apt. For further information about the possible
# configuration settings see /usr/share/doc/cron-apt/README.gz.
MAILTO="mebooks.support@gmail.com"
#Send us an email when upgrades are readt to apply
MAILON="upgrade"

We'll want to make sure that the cron job is set up correctly in /etc/cron.d/cron-apt:

#
# Regular cron jobs for the cron-apt package
#
# Every night at 4 o'clock.
0 4 * * *   root    test -x /usr/sbin/cron-apt && /usr/sbin/cron-apt
# Every hour.
# 0 *   * * *   root    test -x /usr/sbin/cron-apt && /usr/sbin/cron-apt /etc/cron-apt/config2
# Every five minutes.
# */5 * * * *   root    test -x /usr/sbin/cron-apt && /usr/sbin/cron-apt /etc/cron-apt/config2

postfix

Possibly the most difficult part of setting up cron-apt is configuring the mail server to allow cron-apt to send out its emails about packages ready to be upgraded (presuming you don't already have a mail server set up).

In our case, we'll configure postfix to route outbound emails via Mandrill, an SMTP email relaying service.

Our postfix configuration file (as generated by our ansible role) /etc/postfix/main.cf looks like the following:

# Ansible managed: /Users/jasondarwin/workspace/ansible-digitalocean/roles/postfix/templates/main-cf.j2 modified on 2015-04-03 15:37:33 by jasondarwin on hare.lan

smtpd_banner = $myhostname ESMTP $mail_name
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# Enable SASL authentication
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_use_tls = yes

# General
myhostname = host1
myorigin = $mydomain
mydestination =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = loopback-only
relayhost = [smtp.mandrillapp.com]
inet_protocols = ipv4
sender_canonical_maps = hash:/etc/postfix/sender_canonical

Note the following:

  1. We use SASL authentication to authenticate with Mandrill, so have a /etc/postfix/sasl_passwd which contains something like:

     [smtp.mandrillapp.com]    jcdarwin@gmail.com:WHATEVERYOURAPIKEYIS

This file is compiled as follows:

    /usr/sbin/postmap /etc/postfix/sasl_passwd

  1. We specify Mandrill as our relayhost:

     relayhost = [smtp.mandrillapp.com]

  2. We restrict the protocol used to IPV4, as Mandrill doesn't yet seem to support IPV6:

     inet_protocols = ipv4

  3. We use sender_canonical_maps to specify the email sender address.

     sender_canonical_maps = hash:/etc/postfix/sender_canonical

    and this file contains something like

     root root@mebooks.co.nz

    This file is compiled as follows:

     /usr/sbin/postmap /etc/postfix/sender_canonical

    If this file didn't exist, cron-apt will send email as root, meanign our From address would look like root@localdomain, and Mandrill will refuse to relay these messages.

Testing postfix

We can use sendmail to test postfix:

sendmail mebooks.support@gmail.com
Here is the message body
CTRL+D

As no From: is specified, postfix should use the address specified for root in /etc/postfix/sender_canonical, and we should be able to see this in our mail log by running cat /var/log/mail.log:

Apr  5 02:17:32 mebooks1 postfix/qmgr[26754]: 95A76142FF1: removed
Apr  5 02:20:06 mebooks1 postfix/master[26750]: terminating on signal 15
Apr  5 02:20:07 mebooks1 postfix/master[26940]: daemon started -- version 2.11.0, configuration /etc/postfix
Apr  5 02:20:21 mebooks1 postfix/pickup[26943]: 2529F142FF1: uid=0 from=<root>
Apr  5 02:20:21 mebooks1 postfix/cleanup[26951]: 2529F142FF1: message-id=<20150405062021.2529F142FF1@host1>
Apr  5 02:20:21 mebooks1 postfix/qmgr[26944]: 2529F142FF1: from=<root@mebooks.co.nz>, size=250, nrcpt=1 (queue active)
Apr  5 02:20:21 mebooks1 postfix/smtp[26953]: 2529F142FF1: to=<mebooks.support@gmail.com>, relay=smtp.mandrillapp.com[52.74.52.47]:25, delay=6.8, delays=6.8/0.04/0.03/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 5B9082A00BD)
Apr  5 02:20:21 mebooks1 postfix/qmgr[26944]: 2529F142FF1: removed

If nothing appears in /var/log/mail.log, have a look in /var/log/mail.err for any messages.

Wrapping up

Presuming that cron-apt and postfix are working correctly, we should receive an email before too long notifying us that there are packages to upgrade.

Upgrading these is simply a matter of running the following on the server:

apt-get dist-upgrade