Fix GCP UniFi Controller certificate issue

If you are using my script to Set up UniFi Controller on Google Cloud Platform and you have been affected by the Let’s Encrypt root certificate roll over, you will need this fix.

Just Stop and then Start your virtual machine in Google Cloud Console. This will update the VM with new code. Presuming of course that you still have the startup-script-url in your metadata. The fix won’t be immediate, however. It will only take effect after the next certificate renewal, which might take up to 60 days.

If you want to fix it right away then you need to log in to the VM. You can do that with the SSH button in the Console. After you get a prompt type sudo certbot renew --force-renewal You will get a warning about the keystore format and an error Unable to import the certificate into keystore. I haven’t found the cause for the error, but the certificate is imported nonetheless.

Special thanks for Djo and Derek for providing the fix. It took me time to get the script to do the same and I also tried to get rid of the error. The FireFox workaround was too good so it took longer than it should have. Enjoy!

GCP Unifi Controller startup script explained

This is a walkthrough of the Google Cloud Platform virtual machine startup script to launch full featured Unifi Controllers on demand.

See how-to instructions and a video in another article.

This script was written for Google Cloud Platform and Debian. It should be fairly straightforward to port to any other cloud platform. They have a similar way of accessing virtual machine metadata. The only other GCP specific function is gsutil rsync in the backup script.

Porting to other Linux distributions or Unices would take some more effort. If the destination system doesn’t use systemd then the services and timers should be rewritten. The script also uses apt-get to install packages, but that is fairly easy to edit. Ubiquiti only provides a dpkg package for the controller, though.

I have tried to write the script in open style without shortcuts. A lot of the ifs could be streamlined, but I value legibility. The resulting system is also easy to reconfigure, if you are familiar with Unix command line. The list of files is at the end of the article.

# Set up logging for unattended scripts and UniFi's MongoDB log
# Variables $LOG and $MONGOLOG are used later on in the script.
if [ ! -f /etc/logrotate.d/gcp-unifi.conf ]; then
	cat > /etc/logrotate.d/gcp-unifi.conf <<_EOF
$LOG {
	rotate 4
	echo "Script logrotate set up"
if [ ! -f /etc/logrotate.d/unifi-mongod.conf ]; then
	cat > /etc/logrotate.d/unifi-mongod.conf <<_EOF
	rotate 10
	echo "MongoDB logrotate set up"

Although most users won’t ever log on to the virtual machine, I still find it worthwhile to write a log. This log will only contain unattended script actions like database repairs or certificate updates every 3 months or so. Not much volume, but it is still good practice to set up logrotate. Four months will cover the last certificate update. For some reason Ubiquiti doesn’t set up logrotate for their MongoDB instance so we’ll cover that as well.

# Turn off IPv6 for now
if [ ! -f /etc/sysctl.d/20-disableIPv6.conf ]; then
	echo "net.ipv6.conf.all.disable_ipv6=1" > /etc/sysctl.d/20-disableIPv6.conf
	sysctl --system > /dev/null
	echo "IPv6 disabled"

I found out that occasionally the UniFi controller will try to bind to IPv6 only. GCP doesn’t route IPv6 to the virtual machines at this time, so we can just as well turn it off.

# Update DynDNS as early in the script as possible
ddns=$(curl -fs -H "Metadata-Flavor: Google" "")
if [ ${ddns} ]; then
	curl -fs ${ddns}
	echo "Dynamic DNS accessed"

If the user supplied a dynamic DNS service URL we’ll call it as early as possible so the DNS has a couple of minutes to settle before we’ll need it for Let’s Encrypt certificate at the end of the script.

# Create a swap file for small memory instances and increase /run
if [ ! -f /swapfile ]; then
	memory=$(free -m | grep "^Mem:" | tr -s " " | cut -d " " -f 2)
	echo "${memory} megabytes of memory detected"
	if [ -z ${memory} ] || [ "0${memory}" -lt "2048" ]; then
		fallocate -l 2G /swapfile
		chmod 600 /swapfile
		mkswap /swapfile >/dev/null
		swapon /swapfile
		echo '/swapfile none swap sw 0 0' >> /etc/fstab
		echo 'tmpfs /run tmpfs rw,nodev,nosuid,size=400M 0 0' >> /etc/fstab
		mount -o remount,rw,nodev,nosuid,size=400M tmpfs /run
		echo "Swap file created"

If the memory of the virtual machine is less than 2GB (or we can’t find out) we’ll create a 2GB swap file and register it on /etc/fstab. The size of default tmpfs mounted at /run also depends on the amount of memory. On a small machine /run may be too small to create backups so we’ll set it to 400M. Tmpfs will be swapped out as needed, so it won’t consume real memory.

# Set the time zone
tz=$(curl -fs -H "Metadata-Flavor: Google" "")
if [ ${tz} ] && [ -f /usr/share/zoneinfo/${tz} ]; then
	apt-get -qq install -y dbus >/dev/null
	if ! systemctl start dbus; then
		echo "Trying to start dbus"
		sleep 15
		systemctl start dbus
	if timedatectl set-timezone $tz; then echo "Localtime set to ${tz}"; fi
	systemctl reload-or-restart rsyslog

If the user supplied a timezone reference we’ll check whether it exists on Debian. To set the timezone properly we need DBus. It is finicky and often won’t start on first try. We retry right away and if needed a third time 15 seconds later. RSysLog is already running but it needs to be made aware of the new local time.

# Add Unifi to APT sources
if [ ! -f /etc/apt/trusted.gpg.d/unifi-repo.gpg ]; then
	echo "deb stable ubiquiti" > /etc/apt/sources.list.d/unifi.list
	curl -Lfs -o /etc/apt/trusted.gpg.d/unifi-repo.gpg
	echo "Unifi added to APT sources";

If the file doesn’t exist we’ll add the Ubiquiti repo and its GPG key to APT sources.

# Add backports if it doesn't exist
release=$(lsb_release -a 2>/dev/null | grep "^Codename:" | cut -f 2)
if [ ${release} ] && [ ! -f /etc/apt/sources.list.d/backports.list ]; then
	cat > /etc/apt/sources.list.d/backports.list <<_EOF
deb ${release}-backports main
deb-src ${release}-backports main
	echo "Backports (${release}) added to APT sources"

CertBot should be installed from the backports repo. The repo definition needs the codename for the release, so we need to figure it out first. Google provided images should contain this by default, but we’ll check for it just in case.

# Install stuff
if [ ! -f /usr/share/misc/apt-upgraded ]; then
	apt-get -qq update -y >/dev/null
	apt-get -qq upgrade -y >/dev/null
	touch /usr/share/misc/apt-upgraded
	echo "System upgraded"

Update the APT package list and upgrade the system to the latest on the first run only. We’ll leave a flag file in /var/share/misc so the next boots won’t be delayed. Unattended Upgrades will keep the system up to date on a daily basis.

haveged=$(dpkg-query -W --showformat='${Status}\n' haveged 2>/dev/null)
if [ "x${haveged}" != "xinstall ok installed" ]; then 
	if apt-get -qq install -y haveged >/dev/null; then
		echo "Haveged installed"

Haveged provides entropy for the random number generator needed for encryption. Cloud based virtual machines don’t have access to entropy sources like ordinary computers do, which may delay booting 30 minutes or more.

certbot=$(dpkg-query -W --showformat='${Status}\n' certbot 2>/dev/null)
if [ "x${certbot}" != "xinstall ok installed" ]; then
if (apt-get -qq install -y -t ${release}-backports certbot >/dev/null) || (apt-get -qq install -y certbot >/dev/null); then
		echo "CertBot installed"

CertBot is the tool to acquire and renew Let’s Encrypt certificates. According to EFF docs it should be installed from backports, but this occasionally gives errors. We try the backports first and then the default repos if it fails.

unifi=$(dpkg-query -W --showformat='${Status}\n' unifi 2>/dev/null)
if [ "x${unifi}" != "xinstall ok installed" ]; then
	if apt-get -qq install -y unifi >/dev/null; then
		echo "Unifi installed"
	systemctl stop mongodb
	systemctl disable mongodb

UniFi Controller is the beef.

UniFi Controller will install and enable MongoDB, even though the service is never used. UniFi Controller will start its own MongoDB instance instead. By stopping and disabling the stock MongoDB we’ll save some memory and processor cycles.

httpd=$(dpkg-query -W --showformat='${Status}\n' lighttpd 2>/dev/null)
if [ "x${httpd}" != "xinstall ok installed" ]; then
	if apt-get -qq install -y lighttpd >/dev/null; then
		cat > /etc/lighttpd/conf-enabled/10-unifi-redirect.conf <<_EOF
 \$HTTP["scheme"] == "http" {
	\$HTTP["host"] =~ ".*" { 
		url.redirect = (".*" => "https://%0:8443")
		systemctl reload-or-restart lighttpd
		echo "Lighttpd installed"

Install Lighttpd and configure it to redirect plain http requests to https on port 8443. Lighttpd needs to be reloaded after the configuration change.

f2b=$(dpkg-query -W --showformat='${Status}\n' fail2ban 2>/dev/null)
if [ "x${f2b}" != "xinstall ok installed" ]; then 
	if apt-get -qq install -y fail2ban >/dev/null; then
			echo "Fail2Ban installed"
	if [ ! -f /etc/fail2ban/filter.d/unifi-controller.conf ]; then
		cat > /etc/fail2ban/filter.d/unifi-controller.conf <<_EOF
failregex = ^.* Failed .* login for .* from \s*$
		cat > /etc/fail2ban/jail.d/unifi-controller.conf <<_EOF 
filter = unifi-controller 
port = 8443 
logpath = /var/log/unifi/server.log 

# The .local file will be installed in any case cat > /etc/fail2ban/jail.d/unifi-controller.local <<_EOF
enabled  = true
maxretry = 3
bantime  = 3600
findtime = 3600
	systemctl reload-or-restart fail2ban

Fail2Ban will protect the controller from brute-force login attacks. We need three files for it: the filter contains the regex, the jail file contains the generic settings and the local jail file for local setup. The two latter could be combined but I have submitted the first two upstream to Fail2Ban maintainers. If they are accepted they will eventually be installed automatically. Then we only need to create the local jail file.

# Set up unattended upgrades after 04:00 with automatic reboots
if [ ! -f /etc/apt/apt.conf.d/51unattended-upgrades-unifi ]; then
	cat > /etc/apt/apt.conf.d/51unattended-upgrades-unifi <<_EOF
Acquire::AllowReleaseInfoChanges "true";
Unattended-Upgrade::Origins-Pattern {
Unattended-Upgrade::Remove-Unused-Dependencies "true";
Unattended-Upgrade::Automatic-Reboot "true";
	cat > /etc/systemd/system/ <<_EOF
Description=Daily apt upgrade and clean activities
	systemctl daemon-reload
	systemctl reload-or-restart unattended-upgrades
	echo "Unattended upgrades set up"

By Google default only the security updates are installed automatically. This file will add all Debian stable updates and Ubiquiti updates to the automatic update schedule. If the update requires a restart or reboot, it will occur after 04:00 local time. This is the reason for setting the timezone.

# Set up automatic repair for broken MongoDB on boot
if [ ! -f /usr/local/sbin/ ]; then
	cat > /usr/local/sbin/ <<_EOF
#! /bin/sh
if ! pgrep mongod; then
	if [ -f /var/lib/unifi/db/mongod.lock ] \
	|| [ -f /var/lib/unifi/db/WiredTiger.lock ] \
	|| [ -f /var/run/unifi/db.needsRepair ] \
	|| [ -f /var/run/unifi/launcher.looping ]; then
		if [ -f /var/lib/unifi/db/mongod.lock ]; then rm -f /var/lib/unifi/db/mongod.lock; fi
		if [ -f /var/lib/unifi/db/WiredTiger.lock ]; then rm -f /var/lib/unifi/db/WiredTiger.lock; fi
		if [ -f /var/run/unifi/db.needsRepair ]; then rm -f /var/run/unifi/db.needsRepair; fi
		if [ -f /var/run/unifi/launcher.looping ]; then rm -f /var/run/unifi/launcher.looping; fi
		echo >> $LOG
		echo "Repairing Unifi DB on \$(date)" >> $LOG
		su -c "/usr/bin/mongod --repair --dbpath /var/lib/unifi/db --logappend --logpath ${MONGOLOG} 2>>$LOG" unifi
	echo "MongoDB is running. Exiting..."
	exit 1
exit 0
	chmod a+x /usr/local/sbin/

	cat > /etc/systemd/system/unifidb-repair.service <<_EOF
Description=Repair UniFi MongoDB database at boot
Before=unifi.service mongodb.service
	systemctl enable unifidb-repair.service
	echo "Unifi DB autorepair set up"

This section will create a script to in /usr/local/sbin to repair MongoDB if necessary. Then it will create a system.d unit to launch the script at every boot before Unifi and MongoDB are started.

In the script the first condition checks if MongoDB is running. The script should be run by systemd at boot before UniFi is started, but the check is there just in case someone runs this as a command. The script is executable by anyone and /usr/local/sbin is in root’s path so this is a possibility.

If the MongoDB is shut down improperly it will leave lock files, which indicate a repair operation is needed. UniFi controller may add flag files that the startup is looping or the database is broken. If any of these files is found, they are all removed and MongoDB repair is attempted.

At last we’ll create the unit file for a service and enable it so this will be run automatically at boot.

# Set up daily backup to a bucket after 01:00
bucket=$(curl -fs -H "Metadata-Flavor: Google" "")
if [ ${bucket} ]; then
	cat > /etc/systemd/system/unifi-backup.service <<_EOF
[Unit] Description=Daily backup to ${bucket}
[Service] Type=oneshot
ExecStart=/usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://$bucket

	cat > /etc/systemd/system/unifi-backup.timer <<_EOF
Description=Daily backup to ${bucket} timer
	systemctl daemon-reload
	systemctl start unifi-backup.timer
	echo "Backups to ${bucket} set up"

If the user has specified a bucket we’ll create two systemd unit files. One for a timer to run once a day and a service to actually execute the rsync command.

The unit files are rewritten at every boot in case the user has changed the destination bucket. One could argue that there is no point in enabling the service, since it is also started at every boot. Enabling does very little harm and perhaps one day the script isn’t loaded for some error.

# Set up Let's Encrypt
dnsname=$(curl -fs -H "Metadata-Flavor: Google" "")
if [ -z ${dnsname} ]; then exit 0; fi

Setting up Let’s Encrypt reliably turned out to be messy. At first we check whether the user supplied a DNS name and if not then we are done. We also set some file paths for shorthand.

if [ ! -f $caroot ]; then
	cat > $caroot <<_EOF

This cross-signed certificate is required for the import process. You should check it matches the original (courtesy of Steve Jenkins).

Next we’ll write a deploy hook that CertBot will automatically run every time it has renewed a certificate. We need to massage and then import the new certificate to Java keystore for UniFi.

if [ ! -d /etc/letsencrypt/renewal-hooks/deploy ]; then
	mkdir -p /etc/letsencrypt/renewal-hooks/deploy
cat > /etc/letsencrypt/renewal-hooks/deploy/unifi <<_EOF 
#! /bin/sh
if [ -e $privkey ] && [ -e $pubcrt ] && [ -e $chain ]; then 
	echo >> $LOG
	echo "Importing new certificate on \$(date)" >> $LOG
	if ! openssl pkcs12 -export \\
	-in $pubcrt \\
	-inkey $privkey \\
	-CAfile $chain \\
	-out \${p12} -passout pass:aircontrolenterprise \\
	-caname root -name unifi >/dev/null ; then
		echo "OpenSSL export failed" >> $LOG
		exit 1
	if ! keytool -delete -alias unifi \\
	-keystore /var/lib/unifi/keystore \\
	-deststorepass aircontrolenterprise >/dev/null ; then
		echo "KeyTool delete failed" >> $LOG
	if ! keytool -importkeystore \\
	-srckeystore \${p12} -srcstoretype PKCS12 \\
	-srcstorepass aircontrolenterprise \\
	-destkeystore /var/lib/unifi/keystore \\
	-deststorepass aircontrolenterprise \\
	-destkeypass aircontrolenterprise \\
	-alias unifi -trustcacerts >/dev/null; then
		echo "KeyTool import failed" >> $LOG
		exit 2
	systemctl stop unifi
	if ! java -jar /usr/lib/unifi/lib/ace.jar import_cert \\
	$pubcrt $chain $caroot >/dev/null; then
		echo "Java import_cert failed" >> $LOG
		systemctl start unifi
		exit 3
	systemctl start unifi
	rm -f \${p12}
	echo "Success" >> $LOG
	echo "Certificate files missing" >> $LOG
	exit 4
chmod a+x /etc/letsencrypt/renewal-hooks/deploy/unifi

There are many steps where this may fail. The exit codes and the log will hint what went wrong. It is still important to restart the UniFi controller to be of any service. The only error which won’t cause premature exit is keytool delete. If it fails, then most of the time the keytool import will also fail, causing exit.

Next we’ll create a script in /usr/local/sbin to run the CertBot. This way we can use it in a timer if we don’t succeed the first time:

cat > /usr/local/sbin/ <<_EOF 
#! /bin/sh 
extIP=\$(curl -fs -H "Metadata-Flavor: Google" "") 
dnsIP=\$(getent hosts ${dnsname} | cut -d " " -f 1) 
echo >> $LOG
echo "CertBot run on \$(date)" >> $LOG
if [ \${extIP} = \${dnsIP} ]; then
	if [ ! -d /etc/letsencrypt/live/${dnsname} ]; then
		systemctl stop lighttpd
		if certbot certonly -d $dnsname --standalone --agree-tos --register-unsafely-without-email >> $LOG; then
			echo "Received certificate for ${dnsname}" >> $LOG
		systemctl start lighttpd
	if /etc/letsencrypt/renewal-hooks/deploy/unifi; then
		systemctl stop certbotrun.timer
		echo "Certificate installed for ${dnsname}" >> $LOG
	echo "No action because ${dnsname} doesn't resolve to ${extIP}" >> $LOG
chmod a+x /usr/local/sbin/

First we need to figure out the external IP of the VM and the IP address of the user-supplied DNS name. If they match and we don’t yet have a certificate for the domain, we’ll try. We need to shutdown Lighttpd for the time because CertBot may need port 80 as well. If we succeed we need to run the deploy hook manually, since it is run automatically only for renewals. In case this was a timer job, we’ll stop the timer after success.

If we don’t succeed on the first try, we’ll run the CertBot once an hour as a timer job. This requires two unit files and a systemd reload:

cat > /etc/systemd/system/certbotrun.timer <<_EOF 
[Unit] Description=Run CertBot hourly until success 
[Timer] OnCalendar=hourly 

systemctl daemon-reload cat > /etc/systemd/system/certbotrun.service <<_EOF
Description=Run CertBot hourly until success

This short part will set the Let’s Encrypt machinery rolling:

if [ ! -d /etc/letsencrypt/live/${dnsname} ]; then
	if ! /usr/local/sbin/; then
		echo "Installing hourly CertBot run"
		systemctl start certbotrun.timer

If there is no certificate yet, we’ll run the script manually. If it fails, we’ll start the timer.

List of files

These files are created or modified by the script. You can edit these to your needs once the system is up. In the order of appearance:

  • /etc/logrotate.d/gcp-unifi.conf
  • /etc/logrotate.d/unifi-mongod.conf
  • /etc/sysctl.d/20-disableIPv6.conf
  • /etc/fstab
  • /etc/apt/sources.list.d/unifi.list
  • /etc/apt/sources.list.d/backports.list
  • /etc/lighttpd/conf-enabled/10-unifi-redirect.conf
  • /etc/fail2ban/filter.d/unifi-controller.conf
  • /etc/fail2ban/jail.d/unifi-controller.conf
  • /etc/fail2ban/jail.d/unifi-controller.local
  • /etc/apt/apt.conf.d/51unattended-upgrades-unifi
  • /etc/systemd/system/
  • /usr/local/sbin/
  • /etc/systemd/system/unifidb-repair.service
  • /etc/systemd/system/unifi-backup.service (*)
  • /etc/systemd/system/unifi-backup.timer (*)
  • # /var/lib/unifi/ (*)
  • /etc/letsencrypt/renewal-hooks/deploy/unifi (*)
  • /usr/local/sbin/ (*)
  • /etc/systemd/system/certbotrun.timer
  • /etc/systemd/system/certbotrun.service

The starred files are rewritten at every boot if bucket or dnsname metadata fields are defined. If you want to edit them you need to remove the startup-script-url from the metadata. file will be edited in place for xms and xmx (once Ubiquiti fixes a bug). You can edit other lines as required.


Set up UniFi Controller on Google Cloud Platform

Google offers one free virtual machine on Google Cloud Platform. This script will make setting up a UniFi Controller on GCP a breeze and it includes all the goodies. No command line required, everything is done in the GCP Console and it takes 15 minutes total and that includes transferring your current sites to the cloud. You should try this!

[Note 2021/12] Regarding the Log4j vulnerability: Your controller should automatically have upgraded to 6.5.54. No action required.

[Note 2021/12] If you are getting a certificate error then see Fix GCP UniFi Controller certificate. It is finally solved [with help from Djo and Derek]! Basically you just need to stop and start your VM.

[Note 2021/09] If you are still being charged for egress traffic, check your network tier. Apparently Premium is on the free offer and Standard is not! [Thanks, gman!]

[Note 2020/01] If Google has charged you for egress traffic to some remote areas of the globe, you can stop Lighttpd as a fix. Log in to the VM and type sudo systemctl stop lighttpd and sudo systemctl disable lighttpd. That means you will need to use the full https://my.ctrl.dns:8443 URL, but your browser should already know it.

The script I have created will set up an UniFi Controller on GCP with these extras:

  • Automatically acquire and install a Let’s Encrypt certificate to relieve you from HTTPS certificate warnings. For a certificate you need to have a domain name for your controller, dynamic or static.
  • Dynamic DNS support to help you set up the domain name.
  • You don’t need to type https://your.server.dns:8443 every time. You just type the DNS name of your controller and it will be redirected to HTTPS port 8443 automatically.
  • Backup files will be copied to a Google Storage bucket for offline storage.
  • Fail2Ban protects your controller from attackers trying to brute force the password: the IP address of the offender will be denied access for an hour after every three failed attempts.
  • The free tier GCP micro instance comes with only 600MB of memory. The script will create a swap file to accommodate the controller.
  • The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases.
  • If the MongoDB database supporting UniFi Controller needs repairing, a reboot will do the trick. There is a support script to run the repair commands as needed before the controller starts.
  • HAVEGEd will store entropy for encryption. Without sufficient entropy available rebooting the server may take 30 minutes or more.

None of this will cost a dime to try out. With a new account you will get credit for the first year. At least you can watch this 10 minute video to see the whole procedure including transferring your current controller:

(There are subtitles available under the YouTube gear icon)

1. Preliminaries

Take a backup of your current controller in Settings > Maintenance. This is always a good idea. You may also want first to improve your password. In Google Cloud your controller can be accessed by anyone who guesses your username and password. You don’t want that. Fail2Ban will lock the IP address for an hour after three failed attempts, but it won’t help if your password is guessable.

A proper SSL certificate requires a DNS name that points to your controller. You can register your own domain or use one of the free ones that dynamic DNS providers offer. If you want to use a dDNS service, you need to set it up first. There are links to some providers at the end. If you control your DNS yourself you won’t need dDNS.

Next you have to create an account on Google Cloud Platform. You will need a credit card for this, but the micro instance that Google offers for free is sufficient for a small UniFi Controller and the free 5GB on Cloud Storage is plenty for the backups. Google does not guarantee this offer will run forever, but it has run for years now and they will certainly let you know if this changes. The current price for a micro instance is $4.28 per month, if you need to run another for example.

First of all, use a good password or better yet, set up two factor authentication for your Google account. If someone breaks into your account the intruder could create a huge bill in just a few hours. You can also set a spending limit in Billing > Budgets & Alerts. Create a new budget for the whole billing account and set a budget. $50 or even $10 should be enough, if you ever want to do some testing on an extra controller for a few hours. The budget feature won’t protect you from an intruder breaking into your account, however. The first thing the intruder would do is to remove the limit.

If you set up budget alerts (50%, 90% and 100% by default) you need to nominate yourself to a billing administrator to receive the alert emails. This is done in IAM & Admin > IAM section. Locate your identity in the list and click in the Roles column. Check Billing > Project Billing Manager to add this role to your identity.

[UPDATE:] Some readers have reported that they needed to enable the storage API in APIs & Services > Dashboard > Enable APIs & Services > search for Google Cloud Storage JSON API > click Google Cloud Storage JSON API tile > click Enable button. My account had this enabled by default, but my account is older and perhaps the default has been changed. [Thank you Limbo and Chrysilis!]

2. Create a Storage Bucket (optional)

UniFi Controller will create backups automatically. The default is once a month, but you can change this in UniFi Controller > Settings > Auto Backup. I run backups once a week and keep the last 26 or a half year’s worth. These backups are stored on the computer the controller runs on. If this computer is lost for some reason, so are the backups. You should store them somewhere else and a Google Storage bucket is the perfect place. If you create a bucket the script will sync the backups to the bucket daily.

In GCP Console switch to Storage > Cloud Storage > Browser and create a new bucket. The bucket name has to be globally unique, so all the obvious ones are already in use. Use something like petri-office-unifi-backup. For the free offer select Region as the Location type and choose a Location in the U.S. I am in Europe and East Coast is marginally closer, so I use it.

3. Set Up the Virtual Network

This is something you will need to do only once. All your future UniFi controllers will use the same network settings.

Switch to Networking > VPC Network > Firewall. Create a new firewall rule and name it allow-unifi for example, the name isn’t important. In the Target tags field enter unifi. You can use any name for the tag, but you will need to remember it for the next step. In the Source IPv4 ranges type to denote any address. In the Protocols and ports section select Specified protocols and ports and copy these into the TCP and UDP fields:

tcp: 8443,8080,8880,8843,6789,443,80
udp: 3478

Switch to  VPC Networks > External IP Addresses and click on Reserve Static Address. You need to give the address a name, choose the region where your virtual machine is going to run and click Reserve. I suggest you use the same region as the Storage bucket. You can’t attach the address yet, since you haven’t yet created the virtual machine. (Google will charge for the static addresses while they are not in use. The plan is to run the controller non-stop so this is not an issue. Just remember to release the address if you don’t need it any longer.) If you manage your DNS yourself, add an A record for this IP at this point.

4. Create the Virtual Machine

In Compute > Compute Engine > VM Instances create a new instance. You can name it anything you like. Choose between Regions us-west1, us-central1 or us-east1 for the free offer. You need to place the VM in a Zone that is in the same region as the static address. The Machine type should be E2 series e2-micro for the free instance.

Click on Change for the Boot disk.  The default operating system is Debian  GNU/Linux 10 (Buster), but UniFi Controller is not supported on Buster. Change the version to Debian GNU/Linux 9 (Stretch) [Kudos to James!]. The default size is 10GB which suffices, but it is tricky to expand afterwards. The free offer is 30GB but I prefer to set the boot disk to 15GB in case I need to run another VM. To change the size, click on Change and set the size. Also change the Boot disk type to Standard persistent disk (not Balanced) to stay on the free tier [Thanks Steve!]  and then you are ready to click Select to save.

If you created a bucket for the backups, you need to allow the virtual machine to write to Google Storage. The default permission is Read Only. In Identity and API access choose Set access for each API. In the list that appears change Storage to Read Write.

No need to select the HTTP and HTTPS options in the Firewall section. You created your own firewall rules already.

Click on Networking, disks, security, management, sole-tenancy to open advanced options. In the Networking section add unifi to the Network Tags field. This tag ties the firewall rules you created earlier to this virtual machine instance. This step is easy to forget, but missing it will prevent access to your controller! Leave the hostname empty. Click on Default under Network Interfaces and choose the static IP address you created earlier as the External IP. Set Network Service Tier to Premium [apparently Premium is on the Free Tier. Thanks, gman!].

In the Management section add the following Metadata key/value pairs. The first one is mandatory: the script that does the magic. All the others are optional and the order doesn’t matter. There is no way to do any error checking or report script failures, some feature will just not work if the metadata is erroneous. You can come back and change them if necessary. The changes will take effect after next reboot. See the Maintenance section at the end for how to reboot.

Key Value Purpose
startup-script-url gs://petri-unifi/ Required!
ddns-url http://your.dyn.dns/update.php?key=xxxxx Helps you access the Controller
timezone Europe/Helsinki Lets you adjust the reboot hour
dns-name your.server.dns Required for HTTPS certificate
bucket your-bucket-name-here Required for offline backups

Some users have reported problems with the gs: style URL for the script. You can try instead [Thank you Miczita]

Click Create. Go get coffee. Even if the virtual machine appears to be running, the script is not done yet. It used to run in minutes, but give it a half an hour these days.

5. Set Up the Controller

Connect to your new controller. On the first screen of the UniFi wizard click restore from a previous backup and upload the latest backup. Wait for the restore to complete. Log on to the new controller using your old username and password. In Settings > Controller add the DNS name or the IP address of the new controller to Controller Hostname/IP field and check Override inform host with controller hostname/IP. Confirm the change. Click Apply Settings.

Connect to your old controller. Do the same change in Settings > Controller and add the DNS name or the IP address to Controller Hostname/IP field and check Override inform host with controller hostname/IP. This setting is for the current devices that are already associated with the current controller. Now they will start to contact the new controller in Google Cloud.

Connect to your new controller again. Check the devices page to see whether the devices come online eventually. If they don’t, you may have to force reprovisions them on the old controller. In that case go to the Devices page on the old controller, select each device in turn and on the Config tab click Provision under Manage Device.

[UPDATE:] The controller doesn’t log failed login attempts by default any more. Logging is required for Fail2Ban protection. You need to change the logging level of the controller: Settings > Maintenance > Services > Log level and change Mgmt to More and click Apply. [Thank you Gabe!]

How to Set Up a New Network with a GCP Controller

If you are starting from scratch you won’t have any controller backup to transfer to the GCP based UniFi Controller. In that case you have three alternatives:

  1. Install a temporary controller on a local computer, set up the network and transfer the controller to GCP as outlined above.
  2. If you want to do a pure cloud based installation, you’ll need to tell the devices the address of the controller. If you have an UniFi Security Gateway, connect its WAN interface to the Internet and your laptop directly to the LAN interface. Connect to the USG with a browser ( and configure the inform URL to http://your.server.dns:8080/inform. Once you have adopted the USG in the controller other UniFi devices will appear in the controller as you start to connect them.
  3. If you have some other router than an USG, you can either set your DNS server to resolve name unifi to the IP address of your cloud controller or set the DHCP option 43 in your DHCP server. There are examples of both in Ubiquiti’s L3 Adoption Guide. The last resort is to SSH to each device in turn and use command
    set-inform http://your.server.dns:8080/inform.

Maintenance and troubleshooting

If your devices won’t switch to the new controller the reason is usually either in the firewall or DNS. Is your VPC firewall rule correct and does the network tag of the virtual machine match the tag on the rule? Is your DNS name correct both in the metadata and in both controllers’ Override Inform Host fields and does the DNS name point to the correct IP address? In either case the devices will try to connect for several minutes before reconnecting to the old controller. Sometimes it takes hours for DNS to settle, depending on the time-to-live settings. In that case let it sit overnight and force provision the devices next morning.

Let’s Encrypt limits the number of certificates issued per domain to five per week (at the time). This usually isn’t an issue, but if you use the free domains offered by dDNS providers, the most popular domains will have used this week’s five certificates already. In this case the script will try to acquire a certificate once an hour until it succeeds. You will see certificate warnings at first but the address bar should eventually turn green. You can also anticipate this and choose a less popular domain (or register your own).

Don’t set up two controllers to back up to the same bucket. They will delete each other’s backups. You can either create a new bucket or create folders inside the bucket. In the metadata set bucket to your-bucket/foldername.

Google blocks sending email directly to SMTP port 25 from within GCP because they don’t want all spammers on board. If you want to receive email notifications you need to set up authenticated SMTP on port 587 or 465 in UniFi’s Settings > Controller > Mail Server. You can use your personal email account or even use Google’s email engine if you can prove you are not a spammer.

The Cloud Console will typically start displaying a recommendation to increase performance by upgrading to a bigger virtual machine (at cost), even though the CPU utilisation graph typically stays below 15%. Apparently the updates the devices send cause very short spikes, because occasionally the devices report that the controller is busy. This may cause some inaccuracies in the reports, I am not certain. If you can live with that, just dismiss the suggestion. In my experience the micro instance can serve at least the same 30 devices a Cloud Key can. You can also use this script on any virtual machine type you like. When you select the machine type you see the cost per month. You get a better price if you can commit to use the resource for 1 or 3 years.

If the controller appears to be malfunctioning the first remedy is to reboot it. A reboot is also the way to re-run the script if you made changes to the metadata. The safe way to reboot is to Stop and then Start the virtual machine. This won’t change the IP address since you have reserved a static address. Don’t use the Reset button. A Reset will immediately restart the virtual machine, which may damage the UniFi database and/or the Linux filesystem (if you are really unlucky).

The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases. If an update requires a server restart or a reboot it will occur after 04AM UTC. Set the timezone metadata to adjust the reboot to 04AM local time. A reboot will make a captive wireless guest portal inaccessible for a couple of minutes so you don’t want it to happen during peak hours.

Automatically updating the system is a risk. For example there could be an update to Java or MongoDB that is incompatible with the installed UniFi controller. There is no way you could prevent the installation of the update. However, I decided that NOT updating the system is even bigger risk. I don’t trust the intended audience (you) to log on regularly over SSH and run apt-get. In any case, if the risk realizes I trust that it will be transient. The incompatibility will affect a lot of other users as well and there will be another update to solve the issue. This other update will be also automatically installed. If you are familiar with Linux you can also log on the VM and edit /etc/apt/apt.conf.d/51unattended-upgrades-unifi to your requirements.

The automatic update won’t do major Linux version upgrades. Every couple of years it is wise to upgrade the underlying operating system. The Cloud Way is to create a new UniFi controller:

  1. Take a backup of the old controller.
  2. Create a new bucket (or folder).
  3. Remove the static IP address from the old virtual machine (VPC Network > External IP Addresses and Change to None)
  4. Create a new virtual machine as outlined above and restore the backup
  5. Delete the old virtual machine
  6. Done! Since the IP address is the same there is no need to update any settings and the switchover is immediate.

If your controller starts to act up and rebooting doesn’t help, just create a new one. Just like upgrading the underlying Linux, create a new virtual machine. If you cannot connect to the old controller to retrieve the backup, use the latest auto backup in the storage bucket. That’s what they are for. Don’t fall in love with your virtual machine, you should treat them like pencils: very useful but replaceable.

If you are concerned about the security of the script you better read it through first (the link is at the end). Be aware that the script is in my bucket so I can update it as necessary. I could also make malicious changes that would take effect on your server after next boot (after a system update for instance). If you are cautious you should remove the whole startup-script-url row from the metadata as soon as you are happy with the other metadata settings. The script doesn’t do anything once the system is configured. If you ever need to reconfigure something you can just add the startup-script-url back, Stop and Start the VM to run the script and then remove the row again.

Google currently offers for free one micro instance with 30GB of disk space and 5GB of bucket storage in the U.S. regions. Both come with monthly 1GB of free egress (i.e. outbound) data transfer with China and Australia excluded. Egress data is used when you download backups to your computer and when your devices download upgrades from the controller. The static IP address is free as long as it is attached to a running virtual machine. You will be charged if you exceed these limits, but typically the charges are minuscule. For example when you upgrade your Linux you will run another micro instance for a while. The budget feature should protect you from nasty surprises.


Some dynamic DNS providers and how their single line update URLs look like:

  • DNSPark
  • DynDNS
    https://{user}:{updater client key}{hostname}
  • Dynu [thank you Moses!]
  • EasyDNS
  • NameCheap[host]&domain=[domain_name]&password=[ddns_password]
  • Sitelutions
  • ZoneEdit

How to build a working WiFi in an older residential house

Modern WiFi coverage is needed in older residential houses as well. It isn’t always easy to achieve. Here are a couple of design points to consider.

In a smaller, wooden house a single access point may provide good coverage for the entire building. Nice and easy. But if you have a multifloor brick and/or concrete construction, you are in for trouble. Many houses are built like bunkers: reinforced concrete floors and sturdy walls that kill all WiFi signals. You will need multiple access points per floor. For the access points you will need a wired distribution network. Wireless is only for the last hop to the device, the access point needs a wired feed. Unfortunately only 21st century houses tend to have high speed network cabling built in. In older buildings the distribution network is often the main challenge.

Wired Network

Using the power lines for networking is a popular alternative, if you search for solutions. Unfortunately the results are most often not even close to the expectations. This is easy to test. Just plug the powerline adapters to adjacent power sockets and test the throughput. It might be a measly 30 Mbps instead of the hundreds advertised on the box. With more distance the speed will drop even lower. If the sockets are in separate fuse circuits you need an electrician to bridge the circuits. But still you won’t get the throughput you need.

Another widely offered alternative is to use wireless for the distribution network. It is called Mesh WiFi, WiFi repeater or Wireless Distribution System (a.k.a. WDS). The inefficiency will cause problems because there will be so much wireless traffic going back and forth. The net effect is a slow network. In the best mesh access points there are two radios: one for the clients and one for the backhaul, but they tend to be pricey.

Most houses have telephone cables connecting the floors. In the best case the cabling is not used anymore at all. If the telephone cables are installed in tubes inside the walls, you can easily replace them with Ethernet cables. Every access point will need its own feed (cable), but there is not room for many cables in the tube. The professional solution is to put a network switch on every floor and run only one Ethernet cable between the floors. If you only need a couple of access points on a floor, you might get by using twin Ethernet cables. Actually, you can use a single Ethernet cable to transfer two 100 Mbps connections because only from gigabit up are all 8 wires used. With twin cables you can run 4 x 100 Mbps in a single tube. Is it reasonable to install 100 Mbps Ethernet today? In residential use it should be plenty for the next 10 years or so (unless you have fibre connection from the house). Even if the phone cables are installed on the surface, you may still use four wires to get speeds up to 100 Mbps. Either way, you can still use the old phone system to get a neat and inexpensive distribution network. If that is not possible, you need to run cables through ducts, a stairwell or on outer walls, increasing the amount of work to be done.

In every case the cabling job is best left to professionals. They have the tools to install and test the connections. Most electrical, telephone or data service shops do cabling. Ask for a couple of tenders and take your pick. Just make certain that all the cables will be tested and you get the report. A proper tester will cost well over a thousand Euros, so the handyman next door may not have one. All professionals do.

Wireless network

You will need multiple access points per floor to get good coverage through brick walls. Even if you can crank up an access point to cover the whole floor, the cell phone in the last room can’t reach back to the access point. With multiple access points the administration needs to be organized. Manually keeping just five access points in synch regarding all settings and updates is a chore. Ubiquiti has affordable access points and the central controller software is included. The Ruckus brand XClaim would be another alternative, but XClaim doesn’t seem to have a distributor in Finland at the time of writing.

The access points need power to operate. The neatest solution is to feed the power over the Ethernet (a.k.a. PoE). The network switch provides the power, so the switch needs to have a power supply big enough for all the access points. There are two standards for PoE: older 802.3af and later 802.3at (a.k.a. PoE+). Latest 802.11ac access points are power hungry and often require a PoE+ switch. Some vendors have their own, non-standard PoE solutions (often called “passive”). In that case it is best to stick with the same brand for access points and switches to ensure compatibility.