See how-to instructions and a video in another article.
This script was written for Google Cloud Platform and Debian. It should be fairly straightforward to port to any other cloud platform. They have a similar way of accessing virtual machine metadata. The only other GCP specific function is gsutil rsync in the backup script.
Porting to other Linux distributions or Unices would take some more effort. If the destination system doesn’t use systemd then the services and timers should be rewritten. The script also uses apt-get to install packages, but that is fairly easy to edit. Ubiquiti only provides a dpkg package for the controller, though.
I have tried to write the script in open style without shortcuts. A lot of the ifs could be streamlined, but I value legibility. The resulting system is also easy to reconfigure, if you are familiar with Unix command line. The list of files is at the end of the article.
# # Set up logging for unattended scripts and UniFi's MongoDB log # Variables $LOG and $MONGOLOG are used later on in the script. # LOG="/var/log/unifi/gcp-unifi.log" if [ ! -f /etc/logrotate.d/gcp-unifi.conf ]; then cat > /etc/logrotate.d/gcp-unifi.conf <<_EOF $LOG { monthly rotate 4 compress } _EOF echo "Script logrotate set up" fi MONGOLOG="/usr/lib/unifi/logs/mongod.log" if [ ! -f /etc/logrotate.d/unifi-mongod.conf ]; then cat > /etc/logrotate.d/unifi-mongod.conf <<_EOF $MONGOLOG { weekly rotate 10 copytruncate delaycompress compress notifempty missingok } _EOF echo "MongoDB logrotate set up" fi
Although most users won’t ever log on to the virtual machine, I still find it worthwhile to write a log. This log will only contain unattended script actions like database repairs or certificate updates every 3 months or so. Not much volume, but it is still good practice to set up logrotate. Four months will cover the last certificate update. For some reason Ubiquiti doesn’t set up logrotate for their MongoDB instance so we’ll cover that as well.
# # Turn off IPv6 for now # if [ ! -f /etc/sysctl.d/20-disableIPv6.conf ]; then echo "net.ipv6.conf.all.disable_ipv6=1" > /etc/sysctl.d/20-disableIPv6.conf sysctl --system > /dev/null echo "IPv6 disabled" fi
I found out that occasionally the UniFi controller will try to bind to IPv6 only. GCP doesn’t route IPv6 to the virtual machines at this time, so we can just as well turn it off.
# # Update DynDNS as early in the script as possible # ddns=$(curl -fs -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/ddns-url") if [ ${ddns} ]; then curl -fs ${ddns} echo "Dynamic DNS accessed" fi
If the user supplied a dynamic DNS service URL we’ll call it as early as possible so the DNS has a couple of minutes to settle before we’ll need it for Let’s Encrypt certificate at the end of the script.
# # Create a swap file for small memory instances and increase /run # if [ ! -f /swapfile ]; then memory=$(free -m | grep "^Mem:" | tr -s " " | cut -d " " -f 2) echo "${memory} megabytes of memory detected" if [ -z ${memory} ] || [ "0${memory}" -lt "2048" ]; then fallocate -l 2G /swapfile chmod 600 /swapfile mkswap /swapfile >/dev/null swapon /swapfile echo '/swapfile none swap sw 0 0' >> /etc/fstab echo 'tmpfs /run tmpfs rw,nodev,nosuid,size=400M 0 0' >> /etc/fstab mount -o remount,rw,nodev,nosuid,size=400M tmpfs /run echo "Swap file created" fi fi
If the memory of the virtual machine is less than 2GB (or we can’t find out) we’ll create a 2GB swap file and register it on /etc/fstab. The size of default tmpfs mounted at /run also depends on the amount of memory. On a small machine /run may be too small to create backups so we’ll set it to 400M. Tmpfs will be swapped out as needed, so it won’t consume real memory.
# # Set the time zone # tz=$(curl -fs -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/timezone") if [ ${tz} ] && [ -f /usr/share/zoneinfo/${tz} ]; then apt-get -qq install -y dbus >/dev/null if ! systemctl start dbus; then echo "Trying to start dbus" sleep 15 systemctl start dbus fi if timedatectl set-timezone $tz; then echo "Localtime set to ${tz}"; fi systemctl reload-or-restart rsyslog fi
If the user supplied a timezone reference we’ll check whether it exists on Debian. To set the timezone properly we need DBus. It is finicky and often won’t start on first try. We retry right away and if needed a third time 15 seconds later. RSysLog is already running but it needs to be made aware of the new local time.
# # Add Unifi to APT sources # if [ ! -f /etc/apt/trusted.gpg.d/unifi-repo.gpg ]; then echo "deb http://www.ubnt.com/downloads/unifi/debian stable ubiquiti" > /etc/apt/sources.list.d/unifi.list curl -Lfs -o /etc/apt/trusted.gpg.d/unifi-repo.gpg https://dl.ubnt.com/unifi/unifi-repo.gpg echo "Unifi added to APT sources"; fi
If the file doesn’t exist we’ll add the Ubiquiti repo and its GPG key to APT sources.
# # Add backports if it doesn't exist # release=$(lsb_release -a 2>/dev/null | grep "^Codename:" | cut -f 2) if [ ${release} ] && [ ! -f /etc/apt/sources.list.d/backports.list ]; then cat > /etc/apt/sources.list.d/backports.list <<_EOF deb http://deb.debian.org/debian/ ${release}-backports main deb-src http://deb.debian.org/debian/ ${release}-backports main _EOF echo "Backports (${release}) added to APT sources" fi
CertBot should be installed from the backports repo. The repo definition needs the codename for the release, so we need to figure it out first. Google provided images should contain this by default, but we’ll check for it just in case.
# # Install stuff # if [ ! -f /usr/share/misc/apt-upgraded ]; then apt-get -qq update -y >/dev/null apt-get -qq upgrade -y >/dev/null touch /usr/share/misc/apt-upgraded echo "System upgraded" fi
Update the APT package list and upgrade the system to the latest on the first run only. We’ll leave a flag file in /var/share/misc so the next boots won’t be delayed. Unattended Upgrades will keep the system up to date on a daily basis.
haveged=$(dpkg-query -W --showformat='${Status}\n' haveged 2>/dev/null) if [ "x${haveged}" != "xinstall ok installed" ]; then if apt-get -qq install -y haveged >/dev/null; then echo "Haveged installed" fi fi
Haveged provides entropy for the random number generator needed for encryption. Cloud based virtual machines don’t have access to entropy sources like ordinary computers do, which may delay booting 30 minutes or more.
certbot=$(dpkg-query -W --showformat='${Status}\n' certbot 2>/dev/null) if [ "x${certbot}" != "xinstall ok installed" ]; then if (apt-get -qq install -y -t ${release}-backports certbot >/dev/null) || (apt-get -qq install -y certbot >/dev/null); then echo "CertBot installed" fi fi
CertBot is the tool to acquire and renew Let’s Encrypt certificates. According to EFF docs it should be installed from backports, but this occasionally gives errors. We try the backports first and then the default repos if it fails.
unifi=$(dpkg-query -W --showformat='${Status}\n' unifi 2>/dev/null) if [ "x${unifi}" != "xinstall ok installed" ]; then if apt-get -qq install -y unifi >/dev/null; then echo "Unifi installed" fi systemctl stop mongodb systemctl disable mongodb fi
UniFi Controller is the beef.
UniFi Controller will install and enable MongoDB, even though the service is never used. UniFi Controller will start its own MongoDB instance instead. By stopping and disabling the stock MongoDB we’ll save some memory and processor cycles.
httpd=$(dpkg-query -W --showformat='${Status}\n' lighttpd 2>/dev/null) if [ "x${httpd}" != "xinstall ok installed" ]; then if apt-get -qq install -y lighttpd >/dev/null; then cat > /etc/lighttpd/conf-enabled/10-unifi-redirect.conf <<_EOF \$HTTP["scheme"] == "http" { \$HTTP["host"] =~ ".*" { url.redirect = (".*" => "https://%0:8443") } } _EOF systemctl reload-or-restart lighttpd echo "Lighttpd installed" fi fi
Install Lighttpd and configure it to redirect plain http requests to https on port 8443. Lighttpd needs to be reloaded after the configuration change.
f2b=$(dpkg-query -W --showformat='${Status}\n' fail2ban 2>/dev/null) if [ "x${f2b}" != "xinstall ok installed" ]; then if apt-get -qq install -y fail2ban >/dev/null; then echo "Fail2Ban installed" fi if [ ! -f /etc/fail2ban/filter.d/unifi-controller.conf ]; then cat > /etc/fail2ban/filter.d/unifi-controller.conf <<_EOF [Definition] failregex = ^.* Failed .* login for .* from \s*$ _EOF cat > /etc/fail2ban/jail.d/unifi-controller.conf <<_EOF [unifi-controller] filter = unifi-controller port = 8443 logpath = /var/log/unifi/server.log _EOF fi # The .local file will be installed in any case cat > /etc/fail2ban/jail.d/unifi-controller.local <<_EOF [unifi-controller] enabled = true maxretry = 3 bantime = 3600 findtime = 3600 _EOF systemctl reload-or-restart fail2ban fi
Fail2Ban will protect the controller from brute-force login attacks. We need three files for it: the filter contains the regex, the jail file contains the generic settings and the local jail file for local setup. The two latter could be combined but I have submitted the first two upstream to Fail2Ban maintainers. If they are accepted they will eventually be installed automatically. Then we only need to create the local jail file.
# # Set up unattended upgrades after 04:00 with automatic reboots # if [ ! -f /etc/apt/apt.conf.d/51unattended-upgrades-unifi ]; then cat > /etc/apt/apt.conf.d/51unattended-upgrades-unifi <<_EOF Acquire::AllowReleaseInfoChanges "true"; Unattended-Upgrade::Origins-Pattern { "o=Debian,a=stable"; "c=ubiquiti"; }; Unattended-Upgrade::Remove-Unused-Dependencies "true"; Unattended-Upgrade::Automatic-Reboot "true"; _EOF cat > /etc/systemd/system/timers.target.wants/apt-daily-upgrade.timer <<_EOF [Unit] Description=Daily apt upgrade and clean activities After=apt-daily.timer [Timer] OnCalendar=4:00 RandomizedDelaySec=30m Persistent=true [Install] WantedBy=timers.target _EOF systemctl daemon-reload systemctl reload-or-restart unattended-upgrades echo "Unattended upgrades set up" fi
By Google default only the security updates are installed automatically. This file will add all Debian stable updates and Ubiquiti updates to the automatic update schedule. If the update requires a restart or reboot, it will occur after 04:00 local time. This is the reason for setting the timezone.
# # Set up automatic repair for broken MongoDB on boot # if [ ! -f /usr/local/sbin/unifidb-repair.sh ]; then cat > /usr/local/sbin/unifidb-repair.sh <<_EOF #! /bin/sh if ! pgrep mongod; then if [ -f /var/lib/unifi/db/mongod.lock ] \ || [ -f /var/lib/unifi/db/WiredTiger.lock ] \ || [ -f /var/run/unifi/db.needsRepair ] \ || [ -f /var/run/unifi/launcher.looping ]; then if [ -f /var/lib/unifi/db/mongod.lock ]; then rm -f /var/lib/unifi/db/mongod.lock; fi if [ -f /var/lib/unifi/db/WiredTiger.lock ]; then rm -f /var/lib/unifi/db/WiredTiger.lock; fi if [ -f /var/run/unifi/db.needsRepair ]; then rm -f /var/run/unifi/db.needsRepair; fi if [ -f /var/run/unifi/launcher.looping ]; then rm -f /var/run/unifi/launcher.looping; fi echo >> $LOG echo "Repairing Unifi DB on \$(date)" >> $LOG su -c "/usr/bin/mongod --repair --dbpath /var/lib/unifi/db --logappend --logpath ${MONGOLOG} 2>>$LOG" unifi fi else echo "MongoDB is running. Exiting..." exit 1 fi exit 0 _EOF chmod a+x /usr/local/sbin/unifidb-repair.sh cat > /etc/systemd/system/unifidb-repair.service <<_EOF [Unit] Description=Repair UniFi MongoDB database at boot Before=unifi.service mongodb.service After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/local/sbin/unifidb-repair.sh [Install] WantedBy=multi-user.target _EOF systemctl enable unifidb-repair.service echo "Unifi DB autorepair set up" fi
This section will create a script to in /usr/local/sbin to repair MongoDB if necessary. Then it will create a system.d unit to launch the script at every boot before Unifi and MongoDB are started.
In the script the first condition checks if MongoDB is running. The script should be run by systemd at boot before UniFi is started, but the check is there just in case someone runs this as a command. The script is executable by anyone and /usr/local/sbin is in root’s path so this is a possibility.
If the MongoDB is shut down improperly it will leave lock files, which indicate a repair operation is needed. UniFi controller may add flag files that the startup is looping or the database is broken. If any of these files is found, they are all removed and MongoDB repair is attempted.
At last we’ll create the unit file for a service and enable it so this will be run automatically at boot.
# # Set up daily backup to a bucket after 01:00 # bucket=$(curl -fs -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/bucket") if [ ${bucket} ]; then cat > /etc/systemd/system/unifi-backup.service <<_EOF [Unit] Description=Daily backup to ${bucket} service After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://$bucket _EOF cat > /etc/systemd/system/unifi-backup.timer <<_EOF [Unit] Description=Daily backup to ${bucket} timer [Timer] OnCalendar=1:00 RandomizedDelaySec=30m [Install] WantedBy=timers.target _EOF systemctl daemon-reload systemctl start unifi-backup.timer echo "Backups to ${bucket} set up" fi
If the user has specified a bucket we’ll create two systemd unit files. One for a timer to run once a day and a service to actually execute the rsync command.
The unit files are rewritten at every boot in case the user has changed the destination bucket. One could argue that there is no point in enabling the service, since it is also started at every boot. Enabling does very little harm and perhaps one day the script isn’t loaded for some error.
# # Set up Let's Encrypt # dnsname=$(curl -fs -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/attributes/dns-name") if [ -z ${dnsname} ]; then exit 0; fi privkey=/etc/letsencrypt/live/${dnsname}/privkey.pem pubcrt=/etc/letsencrypt/live/${dnsname}/cert.pem chain=/etc/letsencrypt/live/${dnsname}/chain.pem caroot=/usr/share/misc/ca_root.pem
Setting up Let’s Encrypt reliably turned out to be messy. At first we check whether the user supplied a DNS name and if not then we are done. We also set some file paths for shorthand.
if [ ! -f $caroot ]; then cat > $caroot <<_EOF -----BEGIN CERTIFICATE----- MIIDSjCCAjKgAwIBAgIQRK+wgNajJ7qJMDmGLvhAazANBgkqhkiG9w0BAQUFADA/ MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT DkRTVCBSb290IENBIFgzMB4XDTAwMDkzMDIxMTIxOVoXDTIxMDkzMDE0MDExNVow PzEkMCIGA1UEChMbRGlnaXRhbCBTaWduYXR1cmUgVHJ1c3QgQ28uMRcwFQYDVQQD Ew5EU1QgUm9vdCBDQSBYMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB AN+v6ZdQCINXtMxiZfaQguzH0yxrMMpb7NnDfcdAwRgUi+DoM3ZJKuM/IUmTrE4O rz5Iy2Xu/NMhD2XSKtkyj4zl93ewEnu1lcCJo6m67XMuegwGMoOifooUMM0RoOEq OLl5CjH9UL2AZd+3UWODyOKIYepLYYHsUmu5ouJLGiifSKOeDNoJjj4XLh7dIN9b xiqKqy69cK3FCxolkHRyxXtqqzTWMIn/5WgTe1QLyNau7Fqckh49ZLOMxt+/yUFw 7BZy1SbsOFU5Q9D8/RhcQPGX69Wam40dutolucbY38EVAjqr2m7xPi71XAicPNaD aeQQmxkqtilX4+U9m5/wAl0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNV HQ8BAf8EBAMCAQYwHQYDVR0OBBYEFMSnsaR7LHH62+FLkHX/xBVghYkQMA0GCSqG SIb3DQEBBQUAA4IBAQCjGiybFwBcqR7uKGY3Or+Dxz9LwwmglSBd49lZRNI+DT69 ikugdB/OEIKcdBodfpga3csTS7MgROSR6cz8faXbauX+5v3gTt23ADq1cEmv8uXr AvHRAosZy5Q6XkjEGB5YGV8eAlrwDPGxrancWYaLbumR9YbK+rlmM6pZW87ipxZz R8srzJmwN0jP41ZL9c8PDHIyh8bwRLtTcm1D9SZImlJnt1ir/md2cXjbDaJWFBM5 JDGFoqgCWjBH4d1QB7wCCZAA62RjYJsWvIjJEubSfZGL+T0yjWW06XyxV3bqxbYo Ob8VZRzI9neWagqNdwvYkQsEjgfbKbYK7p2CNTUQ -----END CERTIFICATE----- _EOF fi
This cross-signed certificate is required for the import process. You should check it matches the original (courtesy of Steve Jenkins).
Next we’ll write a deploy hook that CertBot will automatically run every time it has renewed a certificate. We need to massage and then import the new certificate to Java keystore for UniFi.
if [ ! -d /etc/letsencrypt/renewal-hooks/deploy ]; then mkdir -p /etc/letsencrypt/renewal-hooks/deploy fi cat > /etc/letsencrypt/renewal-hooks/deploy/unifi <<_EOF #! /bin/sh if [ -e $privkey ] && [ -e $pubcrt ] && [ -e $chain ]; then echo >> $LOG echo "Importing new certificate on \$(date)" >> $LOG p12=\$(mktemp) if ! openssl pkcs12 -export \\ -in $pubcrt \\ -inkey $privkey \\ -CAfile $chain \\ -out \${p12} -passout pass:aircontrolenterprise \\ -caname root -name unifi >/dev/null ; then echo "OpenSSL export failed" >> $LOG exit 1 fi if ! keytool -delete -alias unifi \\ -keystore /var/lib/unifi/keystore \\ -deststorepass aircontrolenterprise >/dev/null ; then echo "KeyTool delete failed" >> $LOG fi if ! keytool -importkeystore \\ -srckeystore \${p12} -srcstoretype PKCS12 \\ -srcstorepass aircontrolenterprise \\ -destkeystore /var/lib/unifi/keystore \\ -deststorepass aircontrolenterprise \\ -destkeypass aircontrolenterprise \\ -alias unifi -trustcacerts >/dev/null; then echo "KeyTool import failed" >> $LOG exit 2 fi systemctl stop unifi if ! java -jar /usr/lib/unifi/lib/ace.jar import_cert \\ $pubcrt $chain $caroot >/dev/null; then echo "Java import_cert failed" >> $LOG systemctl start unifi exit 3 fi systemctl start unifi rm -f \${p12} echo "Success" >> $LOG else echo "Certificate files missing" >> $LOG exit 4 fi _EOF chmod a+x /etc/letsencrypt/renewal-hooks/deploy/unifi
There are many steps where this may fail. The exit codes and the log will hint what went wrong. It is still important to restart the UniFi controller to be of any service. The only error which won’t cause premature exit is keytool delete. If it fails, then most of the time the keytool import will also fail, causing exit.
Next we’ll create a script in /usr/local/sbin to run the CertBot. This way we can use it in a timer if we don’t succeed the first time:
cat > /usr/local/sbin/certbotrun.sh <<_EOF #! /bin/sh extIP=\$(curl -fs -H "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip") dnsIP=\$(getent hosts ${dnsname} | cut -d " " -f 1) echo >> $LOG echo "CertBot run on \$(date)" >> $LOG if [ \${extIP} = \${dnsIP} ]; then if [ ! -d /etc/letsencrypt/live/${dnsname} ]; then systemctl stop lighttpd if certbot certonly -d $dnsname --standalone --agree-tos --register-unsafely-without-email >> $LOG; then echo "Received certificate for ${dnsname}" >> $LOG fi systemctl start lighttpd fi if /etc/letsencrypt/renewal-hooks/deploy/unifi; then systemctl stop certbotrun.timer echo "Certificate installed for ${dnsname}" >> $LOG fi else echo "No action because ${dnsname} doesn't resolve to ${extIP}" >> $LOG fi _EOF chmod a+x /usr/local/sbin/certbotrun.sh
First we need to figure out the external IP of the VM and the IP address of the user-supplied DNS name. If they match and we don’t yet have a certificate for the domain, we’ll try. We need to shutdown Lighttpd for the time because CertBot may need port 80 as well. If we succeed we need to run the deploy hook manually, since it is run automatically only for renewals. In case this was a timer job, we’ll stop the timer after success.
If we don’t succeed on the first try, we’ll run the CertBot once an hour as a timer job. This requires two unit files and a systemd reload:
cat > /etc/systemd/system/certbotrun.timer <<_EOF [Unit] Description=Run CertBot hourly until success [Timer] OnCalendar=hourly RandomizedDelaySec=15m [Install] WantedBy=timers.target _EOF systemctl daemon-reload cat > /etc/systemd/system/certbotrun.service <<_EOF [Unit] Description=Run CertBot hourly until success After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/local/sbin/certbotrun.sh _EOF
This short part will set the Let’s Encrypt machinery rolling:
if [ ! -d /etc/letsencrypt/live/${dnsname} ]; then if ! /usr/local/sbin/certbotrun.sh; then echo "Installing hourly CertBot run" systemctl start certbotrun.timer fi fi
If there is no certificate yet, we’ll run the script manually. If it fails, we’ll start the timer.
List of files
These files are created or modified by the script. You can edit these to your needs once the system is up. In the order of appearance:
- /etc/logrotate.d/gcp-unifi.conf
- /etc/logrotate.d/unifi-mongod.conf
- /etc/sysctl.d/20-disableIPv6.conf
- /etc/fstab
- /etc/apt/sources.list.d/unifi.list
- /etc/apt/sources.list.d/backports.list
- /etc/lighttpd/conf-enabled/10-unifi-redirect.conf
- /etc/fail2ban/filter.d/unifi-controller.conf
- /etc/fail2ban/jail.d/unifi-controller.conf
- /etc/fail2ban/jail.d/unifi-controller.local
- /etc/apt/apt.conf.d/51unattended-upgrades-unifi
- /etc/systemd/system/timers.target.wants/apt-daily-upgrade.timer
- /usr/local/sbin/unifidb-repair.sh
- /etc/systemd/system/unifidb-repair.service
- /etc/systemd/system/unifi-backup.service (*)
- /etc/systemd/system/unifi-backup.timer (*)
- # /var/lib/unifi/system.properties (*)
- /etc/letsencrypt/renewal-hooks/deploy/unifi (*)
- /usr/local/sbin/certbotrun.sh (*)
- /etc/systemd/system/certbotrun.timer
- /etc/systemd/system/certbotrun.service
The starred files are rewritten at every boot if bucket or dnsname metadata fields are defined. If you want to edit them you need to remove the startup-script-url from the metadata.
System.properties file will be edited in place for xms and xmx (once Ubiquiti fixes a bug). You can edit other lines as required.
Could you please put a link to this script to download. I would like to use it and I’m having a hard time setting the script up with it broken up in chunks. I’m getting strange errors. Greatly appreciated!!
Good point. The link was already on the actual how-to page. You should take a look there, but I added the link to the end of the article.
Thank you sir, Great work!
I got an error on this line: “systemctl reload-or-restart unattended-upgrades”
Failed to reload-or-restart unattended-upgrades.service: Unit unattended-upgrades.service not found.
When I changed it to: “systemctl reload-or-restart apt-daily-upgrade”
No error was printed.
I wonder if the unattended-upgrades was installed in the first place. Could you please log in via the SSH button in the Google console and type “apt list –installed” without the quotes. Does it appear in the list?
I’ll need to test. The packages change once in a while and I haven’t spun up a new VM for a few months. Thanks for the heads up!
I have spun up a couple of test servers but I couldn’t reproduce this. Are you running your controller on Debian Stretch as instructed?
Just found your script. Thanks, this is awesome. Some changes I made
This line:
apt-get -qq install -y unifi
Changed to:
apt-get -qq install -y –allow-unauthenticated unifi
For some reason these directories were not created which caused issues, I added these lines near the top
if [ ! -d /var/log/unifi ]; then
mkdir -p /var/log/unifi
echo “Created /var/log/unifi”
fi
if [ ! -d /usr/lib/unifi/logs ]; then
mkdir -p /usr/lib/unifi/logs
echo “Created /usr/lib/unifi/logs”
fi
Thanks! I’ll do some testing and probably add your improvements to the script.
Hi again,
I enabled StackDriver logging so you can view the system logs from the Cloud Console mobile app or the Logging menu item inside GCP.
https://cloud.google.com/logging/docs/agent/
Here is what I added to the script (keep in mind I’m a bash noob):
if [ ! -f /usr/lib/logging-agent/install-logging-agent.sh ]; then
echo “Installing StackDriver agent”
if [ ! -d /usr/lib/logging-agent ]; then
mkdir -p /usr/lib/logging-agent
echo “Created /usr/lib/logging-agent”
fi
curl -Lfs -o /usr/lib/logging-agent/install-logging-agent.sh https://dl.google.com/cloudagents/install-logging-agent.sh
bash /usr/lib/logging-agent/install-logging-agent.sh
echo “StackDriver agent installed”
fi
No need to configure the /var/log/messages file that unifi uses as this file is already reported by default.
I look forward to your updated script. Thanks.
Great script and instructions. I was able to install on my Google Cloud. On the same VM I would like to install UNMS with instructions form this page:
https://help.ubnt.com/hc/en-us/articles/115012196527
I run this command:
curl -fsSL https://unms.com/install > /tmp/unms_inst.sh && sudo bash /tmp/unms_inst.sh
It complains of needing at least 1GB of RAM. I stopped the VM and increased the RAM to 1GB.
After running the command again it continues for a good while but then stops with this message:
Status: Downloaded newer image for ubnt/unms-nginx:0.12.2
Building docker images.
fluentd uses an image, skipping
postgres uses an image, skipping
redis uses an image, skipping
unms uses an image, skipping
rabbitmq uses an image, skipping
nginx uses an image, skipping
Checking available ports
Creating docker-compose.yml
Deploying templates
Writing config file
no crontab for unms
Deleting old firmwares from /home/unms/data/firmwares/unms/*
Starting docker containers.
Creating network “unms_internal” with the default driver
Creating network “unms_public” with the default driver
Creating unms-fluentd
Creating unms-rabbitmq
Creating unms-redis
Creating unms-postgres
Creating unms
Creating unms-nginx
ERROR: for nginx Cannot start service nginx: driver failed programming external connectivit
y on endpoint unms-nginx (735ee03658a75f1e494cafe47af542df3f83366bc5ce0114408a9e709cb00394):
Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.
Failed to start docker containers
What can I do to fix this error?
That is the Lighttpd reserving port 80. You can remove Lighttpd with commands
sudo systemctl stop lighttpd
andsudo systemctl disable lighttpd
. The first one will stop the currently running Lighttpd and the second one will prevent Lighttpd from starting after next reboot. Without Ligttpd you’ll need to use :8443 to access your UniFi controller.My script was intended for users without Linux or GCP skills to automate setting up their UniFi controller. You can use it as a starting point for your own customized version, but I cannot support all possible variations. You are on your own after you diverge. It isn’t necessarily bad, it is a standard Debian system and you can get support for it as it is.
Got everything setup and running fine and migrated from my existing Cloud Key to it fine.
But now I can’t directly access the url @ 8443 at all. The redirect from 80 to 8443 works but it never connects. Running netstat -l on the VM shows it’s listening on 8443 but I don’t get the login screen.
Do you mean you first could connect to 8443 when you restored the backup? Then it stopped? In that case the problem isn’t VPC firewall restrictions that came first to my mind. I would suggest you delete the VM and create a new one with the same IP. Troubleshooting VMs is not worthwhile. If you do want to take a look then the controller log is at /var/log/unifi/server.log. Please let me know how it went.
So I borked networking on my GCP controller and SSH times out, here’s what the GCP serial console prints:
Jan 27 13:56:38 unifi rc.local[644]: Warning: Overwriting existing alias unifi in destination keystore
[ 11.642732] rc.local[644]: Warning:
Jan 27 13:56:38 unifi rc.local[644]: Warning:
[ 11.643488] rc.local[644]: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using “keytool -importkeystore -srckeystore /var/lib/unifi/keystore -destkeystore /var/lib/unifi/keystore -deststoretype pkcs12”.
Jan 27 13:56:38 unifi rc.local[644]: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using “keytool -importkeystore -srckeystore /var/lib/unifi/keystore -destkeystore /var/lib/unifi/keystore -deststoretype pkcs12”.
Jan 27 13:56:38 unifi systemd[1]: Stopped unifi.
[ OK ] Stopped unifi.
Jan 27 13:56:38 unifi systemd[1]: Starting unifi…
Starting unifi…
Jan 27 13:56:38 unifi unifi.init[866]: Starting Ubiquiti UniFi Controller: unifi (already running) failed!
[ OK ] Started unifi.
Jan 27 13:56:38 unifi systemd[1]: Started unifi.
[ OK ] Started /etc/rc.local.
[ OK ] Started Serial Getty on ttyS0.
[ OK ] Started Getty on tty1.
[ OK ] Reached target Login Prompts.
Jan 27 13:56:38 unifi systemd[1]: Started /etc/rc.local.
Jan 27 13:56:38 unifi systemd[1]: Started Serial Getty on ttyS0.
Jan 27 13:56:38 unifi systemd[1]: Started Getty on tty1.
Jan 27 13:56:38 unifi systemd[1]: Reached target Login Prompts.
Jan 27 13:56:38 unifi unifi.init[732]: Starting Ubiquiti UniFi Controller: unifi
Jan 27 13:56:43 unifi instance-setup: ERROR GET request error retrieving metadata. .
Debian GNU/Linux 9 unifi ttyS0
unifi login:
Since I did not set a password, I am unable to login via the serial console. Is there any way I could add a password for the default user via Metadata/GC Console? Any other way I could log back in and fix it?
I don’t know what you mean by borked networking. There is something really odd going on it is definitely not solely a networking issue.
You should be able log in via the SSH button. GCP will create a temp account and transfer the keys to it. However, I don’t recommend troubleshooting it. My recommendation is to start over. Delete the current instance, go through your VPC settings once more and create a new controller. It will take 15 minutes instead of hours spent resurrecting this. You do have the backup safe, don’t you.
In my case, the VM lost internet connection, my problem was get the config files from SSD disk. What I’ve done?
1º – I created a new VM with own disk,
2º – I edit this new VM and attached the old SSD disk as additional disk (not boot).
3º – I accessed the new VM using SSH and mount the old disk partition using the command: /dev/sdb1 /mnt (in my case was sdb1)
4º – I opened the folder /mnt and I get what I needed
Well done! However, those steps require Unix/Linux skills that most people lack.
Hey Petri, first of all thanks for putting this together, really appreciate it. That said I’m having a couple of issues getting my APs to inform the controller running in the GC. It appears the controller is not listening on http://server:8080/inform therfore the APs can’t find the controller.
I have disabled the lighttpd as advised above and also edited the unifi system.properties file. However after issuing a `systemctl start unifi ` command the system.properties file is over-written with the original contents and my changes are lost.
Would you have any idea what is happening?
Thanks again, Andy!
Either the controller is not listening or there is a firewall blocking the connection. The former is easy to verify with
sudo netstat -tulpn
Why did you disable lighttpd? (Disabling it won’t hurt anything, though)
Why and what did you change in system.properties?
I wrote the script for users without Linux skills so they wouldn’t ever need to log in. I can’t support all possible configurations. I may be able to give you some hints, but mostly you are on your own.
Petri,
Thank you for great work. Your guide perfectly worked for me in 2 hours migration from on-premise UniFi Controller to GCP. I am an average Linux user.
My first attempt was 100% following your guide and let the machines do the rest. After long reboot nothing happened, maybe your script could not run.
2nd attempt is to read through this post and copy paste lines from startup.sh to terminal console. That monkey imitation helped me to understand the whole process of installation.
3nd attempt was downloading your startup script, chmod +x and fired it. A few little corrections and everything runs smoothly.
Just a few more commands to change device settings and now I can bury my old Windows Server box with Unifi Controller required restarting and mongod –repair every 6 hours or so.
I’d like to suggest you should add some lines about java prerequisites unifi does not install it as default required dependencies.
Thank you,
Sounds like you were installing it on an existing Linux machine. The script is designed to run when a new VM is created using startup-script-url meta data key. Did you read the main post at Set up UniFi Controller on Google Cloud Platform?
thanks for the script code, very helpfull, can i share it?
Certainly!
This is really great. It worked right away. The upload of the backup to restore took quite a while on my slow connection. I wonder if there is a clever way to grab it from a bucket or something..
Thanks for contributing this code and all of the time you give!
No. The way it works is that your browser sends the backup to the controller. In theory, you could mount the bucket on your PC and upload from there, but in that case your browser would download and upload simultaneously, slowing the process even more.
Petri,
Thank you for your hard work on this script and tutorial. I have having a problem when restoring my backup file to the new VM. It says the backup is from a newer version than the Controller. My backup is from version 5.11.39, released 19 Aug 2019. How can I upgrade the VM to run that version controller?
Many thanks and Cheers!
Matt
The repo is still at 5.10. Usually the release is moved to the repo in a week. I guess this time there has been too many complaints and I guess they are working on a quick fix release.
You can open an SSH session, download the package file and install it manually to get going. The automatic updating will continue from there when there will be a later release in the repo.
Perhaps look at adding rng-tools5 in addition to haveged.
rng-tools5 includes a version of rngd that will take advantage of the RDRAND / RDSEED CPU instruction available to GCP VMs to improve the quality of the entropy pool, as compared to just haveged.
Thank you! I will.
Hello! Thank you for the excellent guide. I’ve followed it and all appears to be working except one thing. I’m still running Unifi Controller 5.14.23. It appears that people who have upgraded to the latest version, 6.0.43, have had some very serious issues. I’ve been studying your startup.sh file all night and cannot figure out how to remove the auto-update. Any chance you could post another file without the auto-update setting? I’m new to this but do know how to do updates via ssh. Thank you kindly, Mike
Remove line 214: “c=ubiquiti”;
The script will install the latest version by default. If you don’t want the controller installed at all then comment out lines 137-139. Then you can install the version of your choice.
That’s exactly what I was looking for! Thank you SO much for the fast response. I’m grateful to have found your work and that you still check on it. Thanks for documenting it in such detail.