Set up UniFi Controller on Google Cloud Platform

Google offers one free virtual machine on Google Cloud Platform. This script will make setting up a UniFi Controller on GCP a breeze and it includes all the goodies. No command line required, everything is done in the GCP Console and it takes 15 minutes total and that includes transferring your current sites to the cloud. You should try this!

The script I have created will set up an UniFi Controller on GCP with these extras:

  • Automatically acquire and install a Let’s Encrypt certificate to relieve you from HTTPS certificate warnings. For a certificate you need to have a domain name for your controller, dynamic or static.
  • Dynamic DNS support to help you set up the domain name.
  • You don’t need to type https://your.server.dns:8443 every time. You just type the DNS name of your controller and it will be redirected to HTTPS port 8443 automatically.
  • Backup files will be copied to a Google Storage bucket for offline storage.
  • Fail2Ban protects your controller from attackers trying to brute force the password: the IP address of the offender will be denied access for an hour after every three failed attempts.
  • The free tier GCP micro instance comes with only 600MB of memory. The script will create a swap file to accommodate the controller.
  • The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases.
  • If the MongoDB database supporting UniFi Controller needs repairing, a reboot will do the trick. There is a support script to run the repair commands as needed before the controller starts.
  • HAVEGEd will store entropy for encryption. Without sufficient entropy available rebooting the server may take 30 minutes or more.

None of this will cost a dime to try out. At least you can watch this 10 minute video to see the whole procedure including transferring your current controller:

(There are subtitles available under the YouTube gear icon)

1. Preliminaries

Take a backup of your current controller in Settings > Maintenance. This is always a good idea. You may also want first to improve your password. In Google Cloud your controller can be accessed by anyone who guesses your username and password. You don’t want that. Fail2Ban will lock the IP address for an hour after three failed attempts, but it won’t help if your password is guessable.

A proper SSL certificate requires a DNS name that points to your controller. You can register your own domain or use one of the free ones that dynamic DNS providers offer. If you want to use a dDNS service, you need to set it up first. There are links to some providers at the end. If you control your DNS yourself you won’t need dDNS.

Next you have to create an account on Google Cloud Platform. You will need a credit card for this, but the micro instance that Google offers for free is sufficient for a small UniFi Controller and the free 5GB on Cloud Storage is plenty for the backups. Google does not guarantee this offer will run forever, but it has run for years now and they will certainly let you know if this changes. The current price for a micro instance is $4.28 per month, if you need to run another for example.

First of all, use a good password or better yet, set up two factor authentication for your Google account. If someone breaks into your account the intruder could create a huge bill in just a few hours. You can also set a spending limit in Billing > Budgets & Alerts. Create a new budget for the whole billing account and set a budget. $50 or even $10 should be enough, if you ever want to do some testing on an extra controller for a few hours. The budget feature won’t protect you from an intruder breaking into your account, however. The first thing the intruder would do is to remove the limit.

If you set up budget alerts (50%, 90% and 100% by default) you need to nominate yourself to a billing administrator to receive the alert emails. This is done in IAM & Admin > IAM section. Locate your identity in the list and click in the Roles column. Check Billing > Project Billing Manager to add this role to your identity.

UPDATE: Some readers have reported that they needed to enable the storage API in APIs & Services > Dashboard > Enable APIs & Services > search for Google Cloud Storage JSON API > click Google Cloud Storage JSON API tile > click Enable button. My account had this enabled by default, but my account is older and perhaps the default has been changed. Thank you Limbo and Chrysilis!

2. Create a Storage Bucket (optional)

UniFi Controller will create backups automatically. The default is once a month, but you can change this in UniFi Controller > Settings > Auto Backup. I run backups once a week and keep the last 26 or a half year’s worth. These backups are stored on the computer the controller runs on. If this computer is lost for some reason, so are the backups. You should store them somewhere else and a Google Storage bucket is the perfect place. If you create a bucket the script will sync the backups to the bucket daily.

In GCP Console switch to Storage > Browser and create a new bucket. The bucket name has to be globally unique, so all the obvious ones are already in use. Use something like petri-office-unifi-backup. For the free offer select Regional as the Storage class and choose a Regional Location in the U.S. I am in Europe and East Coast is marginally closer, so I use it.

3. Set Up the Virtual Network

This is something you will need to do only once. All your future UniFi controllers will use the same network settings.

Switch to VPC Network > Firewall Rules. Create a new firewall rule and name it allow-unifi for example, the name isn’t important. In the Target tags field enter unifi. You can use any name for the tag, but you will need to remember it for the next step. In the Source IP range type 0.0.0.0/0 to denote any address. In the Protocols and ports section select Specified protocols and ports and copy these into the TCP and UDP fields:

tcp: 8443,8080,8880,8843,6789,443,80
udp: 3478

Switch to  VPC Networks > External IP Addresses and click on Reserve Static Address. You need to give a name for the address, choose the region where your virtual machine is going to run and click Reserve. I suggest you use the same region as the Storage bucket. You can’t attach the address yet, since you haven’t yet created the virtual machine. (Google will charge for the static addresses while they are not in use. The plan is to run the controller non-stop so this is not an issue. Just remember to release the address if you don’t need it any longer.) If you manage your DNS yourself, add an A record for this IP at this point.

4. Create the Virtual Machine

In Compute Engine > VM Instances create a new instance. You can name it anything you like. Select a Zone in the U.S. for the free offer. You need to place the VM in a zone that is in the same region as the static address. The Machine type should be micro for the free instance. By default the Boot disk is 10GB which suffices, but it is tricky to expand afterwards. The free offer is 30GB but I prefer to set the boot disk to 15GB in case I need to run another VM. To change the size, click on Change and set the size. The script is designed to run on Debian Linux so don’t change that. Click Select to save.

If you created a bucket for the backups, you need to allow the virtual machine to write to Google Storage. The default permission is Read Only. In Access Scopes choose Set access for each API. In the list that appears change Storage to Read Write.

No need to select the HTTP and HTTPS options in the Firewall section. You created your own firewall rules already.

Click on Management, disks, networking, SSH keys to open advanced options. On the appearing Management tab add the following Metadata key/value pairs. The first one is mandatory: the script that does the magic. All the others are optional and the order doesn’t matter. There is no way to do any error checking or report script failures, some feature will just not work if the metadata is erroneous. You can come back and change them if necessary. The changes will take effect after next reboot. See the Maintenance section at the end for how to reboot.

Key Value Purpose
startup-script-url gs://petri-unifi/startup.sh Required!
ddns-url http://your.dyn.dns/update.php?key=xxxxx Helps you access the Controller
timezone Europe/Helsinki Lets you adjust the reboot hour
dns-name your.server.dns Required for HTTPS certificate
bucket your-bucket-name-here Required for offline backups

Click on the Networking tab and add unifi to the Network Tags field. This tag ties the firewall rules you created earlier to this virtual machine instance. This step is easy to forget, but missing it will prevent access to your controller! Click on Default under Network Interfaces and choose the static IP address you created earlier as the External IP.

Click Create. Go get coffee. Even if the virtual machine appears to be running, the script is not done yet. It usually takes less than five minutes, but give it ten.

5. Set Up the Controller

Connect to your new controller. On the first screen of the UniFi wizard click restore from a previous backup and upload the latest backup. Wait for the restore to complete. Log on to the new controller using your old username and password. In Settings > Controller add the DNS name or the IP address of the new controller to Controller Hostname/IP field and check Override inform host with controller hostname/IP. Confirm the change. Click Apply Settings.

Connect to your old controller. Do the same change in Settings > Controller and add the DNS name or the IP address to Controller Hostname/IP field and check Override inform host with controller hostname/IP. This setting is for the current devices that are already associated with the current controller. Now they will start to contact the new controller in Google Cloud.

Connect to your new controller again. Check the devices page to see whether the devices come online eventually. If they don’t, you may have to force reprovisions them on the old controller. In that case go to the Devices page on the old controller, select each device in turn and on the Config tab click Provision under Manage Device.

How to Set Up a New Network with a GCP Controller

If you are starting from scratch you won’t have any controller backup to transfer to the GCP based UniFi Controller. In that case you have three alternatives:

  1. Install a temporary controller on a local computer, set up the network and transfer the controller to GCP as outlined above.
  2. If you want to do a pure cloud based installation, you’ll need to tell the devices the address of the controller. If you have an UniFi Security Gateway connect its WAN interface to the Internet and your laptop directly to the LAN interface. Connect to the USG with a browser (https://192.168.1.1) and configure the inform URL to http://your.server.dns:8080/inform. Once you have adopted the USG in the controller other UniFi devices will appear in the controller as you start to connect them.
  3. If you have some other router than an USG, you can either set your DNS server to resolve name unifi to the IP address of your cloud controller or set the DHCP option 43 in your DHCP server. There are examples of both in Ubiquiti’s L3 Adoption Guide. The last resort is to SSH to each device in turn and use command
    set-inform http://your.server.dns:8080/inform.

Maintenance and troubleshooting

If your devices won’t switch to the new controller the reason is usually either in the firewall or DNS. Is your VPC firewall rule correct and does the network tag of the virtual machine match the tag on the rule? Is your DNS name correct both in the metadata and in both controllers’ Override Inform Host fields and does the DNS name point to the correct IP address? In either case the devices will try to connect for several minutes before reconnecting to the old controller. Sometimes it takes hours for DNS to settle, depending on the time-to-live settings. In that case let it sit overnight and force provision the devices next morning.

Let’s Encrypt limits the number of certificates issued per domain to five per week (at the time). This usually isn’t an issue, but if you use the free domains offered by dDNS providers, the most popular domains will have used this week’s five certificates already. In this case the script will try to acquire a certificate once an hour until it succeeds. You will see certificate warnings at first but the address bar should eventually turn green. You can also anticipate this and choose a less popular domain (or register your own).

Don’t set up two controllers to back up to the same bucket. They will delete each other’s backups. You can either create a new bucket or create folders inside the bucket. In the metadata set bucket to your-bucket/foldername.

Google blocks sending email directly to SMTP port 25 from within GCP because they don’t want all spammers on board. If you want to receive email notifications you need to set up authenticated SMTP on port 587 or 465 in UniFi’s Settings > Controller > Mail Server. You can use your personal email account or even use Google’s email engine if you can prove you are not a spammer.

The Cloud Console will typically start displaying a recommendation to increase performance by upgrading to a bigger virtual machine (at cost), even though the CPU utilisation graph typically stays below 15%. Apparently the updates the devices send cause very short spikes, because occasionally the devices report that the controller is busy. This may cause some inaccuracies in the reports, I am not certain. If you can live with that, just dismiss the suggestion. In my experience the micro instance can serve at least the same 30 devices a Cloud Key can. You can also use this script on any virtual machine type you like. When you select the machine type you see the cost per month. You get a better price if you can commit to use the resource for 1 or 3 years.

If the controller appears to be malfunctioning the first remedy is to reboot it. A reboot is also the way to re-run the script if you made changes to the metadata. The safe way to reboot is to Stop and then Start the virtual machine. This won’t change the IP address since you have reserved a static address. Don’t use the Reset button. A Reset will immediately restart the virtual machine, which may damage the UniFi database and/or the Linux filesystem (if you are really unlucky).

The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases. If an update requires a server restart or a reboot it will occur after 04AM UTC. Set the timezone metadata to adjust the reboot to 04AM local time. A reboot will make a captive wireless guest portal inaccessible for a couple of minutes so you don’t want it to happen during peak hours.

Automatically updating the system is a risk. For example there could be an update to Java or MongoDB that is incompatible with the installed UniFi controller. There is no way you could prevent the installation of the update. However, I decided that NOT updating the system is even bigger risk. I don’t trust the intended audience (you) to log on regularly over SSH and run apt-get. In any case, if the risk realizes I trust that it will be transient. The incompatibility will affect a lot of other users as well and there will be another update to solve the issue. This other update will be also automatically installed. If you are familiar with Linux you can also log on the VM and edit /etc/apt/apt.conf.d/51unattended-upgrades-unifi to your requirements.

The automatic update won’t do major Linux version upgrades. Every couple of years it is wise to upgrade the underlying operating system. The Cloud Way is to create a new UniFi controller:

  1. Take a backup of the old controller.
  2. Create a new bucket (or folder).
  3. Remove the static IP address from the old virtual machine (VPC Network > External IP Addresses and Change to None)
  4. Create a new virtual machine as outlined above and restore the backup
  5. Delete the old virtual machine
  6. Done! Since the IP address is the same there is no need to update any settings and the turnover is immediate.

If your controller starts to act up and rebooting doesn’t help, just create a new one. Just like upgrading the underlying Linux, create a new virtual machine. If you cannot connect to the old controller to retrieve the backup, use the latest auto backup in the storage bucket. That’s what they are for. Don’t fall in love with your virtual machine, you should treat them like pencils: very useful but replaceable.

If you are concerned about the security of the script you better read it through first (the link is at the end). Be aware that the script is in my bucket so I can update it as necessary. I could also make malicious changes that would take effect on your server after next boot (after a system update for instance). If you are cautious you should remove the whole startup-script-url row from the metadata as soon as you are happy with the other metadata settings. The script doesn’t do anything once the system is configured. If you ever need to reconfigure something you can just add the startup-script-url back, Stop and Start the VM to run the script and then remove the row again.

Google currently offers for free one micro instance with 30GB of disk space and 5GB of bucket storage in the U.S. regions. Both come with monthly 1GB of free egress (i.e. outbound) data transfer with China and Australia excluded. Egress data is used when you download backups to your computer and when your devices download upgrades from the controller. The static IP address is free as long as it is attached to a running virtual machine. You will be charged if you exceed these limits, but typically the charges are minuscule. For example when you upgrade your Linux you will run another micro instance for a while. The budget feature should protect you from nasty surprises.

Links

Some dynamic DNS providers and how their single line update URLs look like:

  • Afraid.org
    http://freedns.afraid.org/dynamic/update.php?xxxdynamicTokenxxx
  • DNSPark
    https://username:passsword@control.dnspark.com/api/dynamic/update.php?hostname=home.example.com
  • DuckDNS
    https://www.duckdns.org/update?domains={YOURVALUE}&token={YOURVALUE}
  • DynDNS
    https://{user}:{updater client key}@members.dyndns.org/v3/update?hostname={hostname}
  • EasyDNS
    https://username:dynamictoken@api.cp.easydns.com/dyn/generic.php?hostname=example.com
  • NameCheap
    https://dynamicdns.park-your-domain.com/update?host=[host]&domain=[domain_name]&password=[ddns_password]
  • Sitelutions
    https://dnsup.sitelutions.com/dnsup?id=990331&user=myemail@mydomain.com&pass=SecretPass
  • ZoneEdit
    https://<username>:<password>@dynamic.zoneedit.com/auth/dynamic.html?host=<hostname>

 

212 thoughts on “Set up UniFi Controller on Google Cloud Platform”

  1. The biggest update so far didn’t go cleanly. Upgrading UniFi controller 5.6 to 5.7.20 killed the MongoDB server disabling the controller. Starting and stopping the virtual machine recovers it. I can’t control the update package contents and instructions. I can only hope for the best. Unattended upgrading is always a gamble.

  2. Hi Petri, thank you very much for all the work you shared with us.
    I follow your procedure and the video but controller is not working.
    How can I check if it’s been downloading and rouning?
    Can I force via SSH script execution so I can see if any error appears?

    Thank you again.

    1. You can SSH into the virtual machine from the Google Cloud Console. Click the SSH button and a pop-up window will appear (unless your pop-up blocker gets into way). No password is required, you are already authenticated in the console. The SSH session is useful if you want to see the logs in your VM.

      Another option is to click on the name of the virtual machine in the VM Instances list. On the VM Instance Details page you’ll see a small link for “Serial Port 1 (Console)”. That will show you all the startup messages. The screen doesn’t update automatically if you want to watch it in real time. You need to click the Refresh for an update. The lines containing “startup-script:” are produced from my script. Do you see any errors?

      1. Thanks for your Reply Prtri:
        In “Serial Port 1 (Console)” there are 6 lines with startup-script:
        Apr 15 17:45:03 unifi-controller startup-script: INFO Starting startup scripts.
        Apr 15 17:45:03 unifi-controller startup-script: INFO Found startup-script-url in metadata.
        Apr 15 17:45:03 unifi-controller startup-script: INFO Downloading url from gs://petri-unifi/startup.sh to /startup-Vm93gn/tmp792gVc using gsutil.
        Apr 15 17:45:08 unifi-controller startup-script: WARNING Could not download gs://petri-unifi/startup.sh using gsutil. Command ‘[‘gsutil’, ‘cp’, u’gs://petri-unifi/startup.sh’, ‘/startup-Vm93gn/tmp792gVc’]’ returned non-zero exit status 1.
        Apr 15 17:45:08 unifi-controller startup-script: INFO No startup scripts found in metadata.
        Apr 15 17:45:08 unifi-controller startup-script: INFO Finished running startup scripts.

        I don’t explain myselft correctly in my last comment, I wanted to know how to execute your script manually 😉
        I found how to download it and executed with sudo, it seem my VM is not accesible from itself:
        Script logrotate set up
        MongoDB logrotate set up
        IPv6 disabled
        ERROR: Address 35.190.181.102 has not changed.
        Dynamic DNS accessed
        592 megabytes of memory detected
        Swap file created
        Localtime set to Europe/Madrid
        Unifi added to APT sources
        System upgraded
        Haveged installed
        CertBot installed
        Extracting templates from packages: 100%
        Unifi installed
        Synchronizing state of mongodb.service with SysV service script with /lib/systemd/systemd-sysv-install.
        Executing: /lib/systemd/systemd-sysv-install disable mongodb
        Lighttpd installed
        Fail2Ban installed
        Unattended upgrades set up
        Created symlink /etc/systemd/system/multi-user.target.wants/unifidb-repair.service → /etc/systemd/system/unifidb-re
        pair.service.
        Unifi DB autorepair set up
        Backups to bdlsolucinoes-backup set up
        Saving debug log to /var/log/letsencrypt/letsencrypt.log
        Plugins selected: Authenticator standalone, Installer None
        Registering without email!
        Obtaining a new certificate
        Performing the following challenges:
        http-01 challenge for mysub.domain.com
        Waiting for verification…
        Cleaning up challenges
        Failed authorization procedure. mysub.domain.com (http-01): urn:acme:error:connection :: The server could n
        ot connect to the client to verify the domain :: Fetching http://mysub.domain.com/.well-known/acme-challeng
        e/rKudGtFidPxCAHNxR__YzQ3qb8PohwWs38yaS6voXcg: Timeout

        I pinged “mysub.domain.com” from VM and my PC, always reply public GCP IP configured as you explained.

        From my PC http://mysub.domain.com or /.well-known/acme-challeng shows ERR_CONNECTION_TIMED_OUT

        Maybe webserver is not working?
        Thanks.

        1. The first part of your post indicates that the VM couldn’t load the startup.sh script at all. That’s odd. I don’t have any other GCP accounts to try on right now. I’ll look into this.

          The second part where you run the script manually indicates that Let’s Encrypt service cannot connect to your domain from their data center. Does the dns-name metadata resolve to 35.190.181.102 (check for typos)? Is port 80 included in the firewall rule and the same tag applied to the VM? Testing tools are dig, nslookup, getent or host for the former and fetch or curl for the latter question.

          1. Hi Petri: your script works great! It’s awasome.

            Connection to my instance was failing becouse I forgot /0 in the 0.0.0.0 on firewall source… 😀

            About startup-script: I found this in serial log:
            ——
            Apr 18 15:25:43 unifi-controller google_metadata_script_runner[683]: AccessDeniedException: 403 Access Not Configured. Please go to the Google Cloud Platform Console (https://cloud.google.com/console#/project) for your project, select APIs and Auth and enable the Google Cloud Storage JSON API.
            ——

            I finally fixed it.

            I want to ask you something: is it as difficult as I read to set a port mapping in GCP?
            My current ISP blocks 8080 so I’m running CloudKey Unifi at TCP/8082

            To easily migrate all my deviced, as you wrote, I want to change controller hostname. So I need to redirect GCP-public-IP:8082 -> VM:8080

            I read lots of commands to enable this port mapping. Do you know a simple way?

            Thank you again! An incredible job.

  3. You sir, are the man!

    Awesome work, seems you’ve thought of everything.

    This must be the only GCP+Unifi tutorial to exist, and it’s fairly easy to follow, it was an absolute treasure of a find, especially given that GCP just happens to be the most difficult to figure out..

    This is killer, it works perfectly, many thanks for this, much appreciated!

    1. Thanks! If you did set up your system recently, it proves that the script download problem in the previous comment by Limbo is either new or random occurence.

      1. Hi Petri, thanks for putting this together! I had the same error as Limbo “AccessDeniedException: 403 Access Not Configured. Please go to the Google Cloud Platform Console (https://cloud.google.com/console#/project) for your project, select APIs and Auth and enable the Google Cloud Storage JSON API.” Once I enabled the Google Cloud Storage JSON API and restarted the VM all was well.

        1. Do you mean APIs and Services > Dashboard ? I have never changed anything there and all services are enabled (there is a button to disable each). Could it be that my account is older and the defaults have changed? The only API that has any request history is the Compute Engine API. None for the Storage JSON API in the past 30 days. I have no clue.

          1. APIs & Services > Dashboard > Enable APIs & Services > Search for “Google Cloud Storage JSON API” > Click “Google Cloud Storage JSON API” tile > Click Enable button

            Without Google Cloud Storage JSON API enabled the VM could not access the startup script. Note that like for you on the APIs & Services Dashboard I see no requests against Google Cloud Storage JSON API.

            1. Thank you for reporting this. I added an update to the post in the Preliminaries section.

  4. Hi! Petri:
    I followed your article, I found that certbot doesn’t work for my cloud controller on GCP.

    Message of Serial Port 1 (Console) contain “startup-script:” looked like error info to me:
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 2.)
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:
    Apr 22 04:38:04 unifi-controller startup-script: INFO startup-script-url: /usr/local/sbin/certbotrun.sh: 7: [: =: argument expected

    Then I managed to use acme.sh which is an alternative to certbot to apply let’s encrypt SSL certificate:
    https://www.naschenweng.info/2017/01/06/securing-ubiquiti-unifi-cloud-key-encrypt-automatic-dns-01-challenge/

    The certificate then installed successfully.

    Is there anything I need to do with the error info besides certbot?
    Thanks.

    1. Each and every apt-get install will create all those debconf and dpkg-preconfigure errors to the log. I haven’t found a way to suppress them without losing valuable debug info. Apt-get is just trying to communicate with the user and doesn’t find a controlling terminal.
      The last line is significant, though, about the certbotrun.sh. It means that for some reason the script couldn’t retrieve the external IP address of the VM. Does your VM have one? I couldn’t reproduce this. Anyways, I have updated the script (currently at 1.0.3) so that it will output a more useful error message instead.
      I haven’t looked into acme.sh. You may have to import the certificate manually to the Java keystore after every renewal. Or you can edit /etc/letsencrypt/renewal-hooks/deploy/unifi to work with acme.sh.

      1. Hi~ Petri,
        I’ve reserved a static external IP address for my cloud Controller VM on GCP as your tutorial steps.
        I am fine with acme.sh, just created a crontab to perform the auto renewal.
        Thanks for your reply!

  5. How do I use Let’s Encrypt with this ?

    How do I see what is in the gs script ?

    Can this setup be done for UNMS ?

    1. 1) Create a DNS name and point it to your VM instance. Add the DNS name to the VM metadata with key dns-name as instructed. Everything else is automated.
      2) At the end of the article in section Links there is a link “The startup script on Google Storage”
      3) My script won’t do it, but you could write a similar setup for UNMS. You can also read through the script and do the steps manually to apply them to an UNMS instance.

  6. Hi Petri!
    First of all, great job and thank you for share your work!

    I followed your video, step by step, but, when i connect to VM, the browser (Opera and Chrome) says “Not Secure”, there is any way to solve this issue?

    1. If you click on the lock icon and check the certificate details, is it a UBNT self-signed certificate or a Let’s Encrypt one? To me it sounds like your VM didn’t receive a LE certificate.

      The most probable cause is that the domain-name metadata didn’t resolve to the external IP of the VM. If you reboot the VM by stopping and starting and then click the VM name to see the details there is a link to the Serial port 1 (console) where you can see all the output from the boot process. Only lines containing “startup-script” relate to my script. If you see a line “No action because aaaaa.bbb doesn’t resolve to vv.xx.yy.zz.” then that is the cause.

      It may be easier to just delete the old VM and create a new one. It is the Cloud Way. It could be that a DNS glitch made the LE check fail. If you delete the old one first then it won’t cost you but 10 minutes of work.

  7. Thanks for this tutorial, this is great!

    I’d like to modify the script to install unifi-testing rather than stable. It looks like the only area I would need to modify your script is line 88. Can you confirm? If so, I’ll modify your script to include that change and then host it in my own bucket.

    1. Yes, that’s the only place. You can also just copy and paste the whole script to the startup-script textbox in GCP Console. No need to use a bucket, if you only need one VM.

      1. Thanks! Last question. I have a custom config.gateway.json file for use with CloudFlare DDNS. How would you suggest I add this in a scripted way?

        1. Add a line like (if the file is on a web server):
          curl -Lfs -o /var/lib/unifi/config.gateway.json http://www.some.dns/path/setup.json
          or (if the file is in a bucket):
          gsutil cp gs://bucket/setup.json /var/lib/unifi/config.gateway.json
          Disclaimer: I haven’t tested either one. I just typed them off the top of my head. E.g you may need to adjust the permissions as an extra step.

          1. I don’t do a lot of shell scripting, but I added a section like this to the end of your file:

            ###########################################################
            #
            # Add Cloudflare DDNS settings to default config.gateway.json
            #
            if [ ! -f /var/lib/unifi/sites/default/config.gateway.json]; then
            cat > /var/lib/unifi/sites/default/config.gateway.json <<_EOF
            {
            "service": {
            "dns": {
            "dynamic": {
            "interface": {
            "eth0": {
            "service": {
            "custom-cloudflare": {
            "host-name": [
            "sub.domain.com"
            ],
            "login": "email@gmail.com",
            "options": [
            "zone=domain.com"
            ],
            "password": "API_KEY_GOES_HERE",
            "protocol": "cloudflare",
            "server": "www.cloudflare.com"
            }
            }
            }
            }
            }
            }
            }
            }
            _EOF
            echo "Cloudflare DDNS added to default config.gateway.json";
            fi

            1. Ahem.. You are right. It should go under the site directory. Your script looks fine to me. Testing will tell the truth.

            2. Oh, one more thing. Don’t put it at the end! If there is no dns-name metadata my script will exit on line 342. Put your addition ahead of the Let’s Encrypt stuff on line 336 for example (or any line before that).

  8. Wow, great call. Thanks for all your help on this.

    I’m slowly moving my homelab to the cloud. This has been a fun and useful learning experience.

  9. Great job Petri! It really helpme, I create 2 vcpu + 7.5 gb Memory +30gb storage, i need to manage about 150 unifi AP, there is no problem with “The script will create a swap file to accommodate the controller.”?, Is the script limiting the memory capacity?

    1. No, the script won’t. It will create a swap file if there is less than 2GB memory. You have more so there won’t be any swapping. UniFi defaults however do limit the controller memory to 1GB. If you log on via the SSH button in GCP Console and edit system.properties file you can increase the initial (xms) and maximum (xmx) memory size of the controller. See Controller Tuning

      1. ok, thanks, I will try that, under your experience, how many ap can you manage with 1gb memory(or users)?

        1. I haven’t done any real testing. Dozens anyway. Ubiquiti quotes 30 devices for 1GB Cloud Key, but I don’t know whether it is memory or processor bound. I haven’t seen the load go up noticeably. Using the GUI is sluggish, though, and periodic MongoDB chores do eat up the processor causing some device updates to go unnoticed. If you can live with those you should be fine.

      2. Hello Petri, I add about 65 ap on the google cloud controller, but im experimenting some issues on some ap, some users report wlan unestable, appears and dissapear or some ap’s become heartbeat missed and then disconected, after that conecten again I improve controller to 3 gb memory on system.properties,, do you have any idea? could you help me? (i will pay you for your service)

        1. The WLAN works even without a connection to the controller. The missed heartbeats just mean a gap in history stats at most. Only if you run a guest network with hotspot you need continuous controller connection. There is some other problem with your network.

          I am abroad this week so I can’t help you right now. You should ask on the Ubiquiti wireless forum. There you get prompt replies for free.

          Of course you could use a slightly larger VM for 65 APs.

            1. That is plenty for a hundreds of APs. The controller performance can’t be the issue here. Do you monitor the uplink from your network to the Internet? If the uplink is congested that could cause the symptoms you are seeing. I can’t see how I could help you further. You should post of on the UniFi forums or contact Ubiquiti support.

  10. Thanks for this post Petri, very helpful.

    One question. I assume the ddns-url metadata field is unnecessary if you are manually pointing your hostname to the static IP through your DNS provider?

  11. Petri, this was extremely helpful and easy to setup with your instructions. Thanks so much for your work.

    I’m up and running but I just have one question. Did you do something to redirect requests to your controller via hostname automatically to 8443 for management or do you have to enter https://yourdomain.com:8443 each time you want to access your cloud controller?

    1. The script installs Lighttpd which will direct everything from port 80 to 8443. To use it enter the plain yourdomain.com in your browser, no https at the start or 8443 at the end. Catching all the possible combinations would have complicated the script far too much.

  12. Kiitos paljon!
    Works like a charm, now I don’t have to worry about a power outage ruining my Raspberry PI controller.

  13. What’s the controller upgrade process like once a controller has been up and running via this script?

    1. Unattended upgrades will run every night and keep both the Linux and the UniFi controller up to date. You won’t have to do anything. It won’t do major Linux upgrades however. Debian “Buster” is expected next summer but “Stretch” will be supported for some time to come.

      1. So the UniFi controller will auto update to the latest stable release that’s available each night? Is there anyway to stop this behavior so that the controller is only upgraded on demand?

        1. Yes. This setup was intended for users without any Linux or cloud skills. I decided that it is safer to upgrade everything automatically than to leave unmanaged VMs running on the Internet. For the target audience the Linux command line is a real barrier.

          Linux savvy users can create and tailor their setups as they see fit. You can use my script as a starting point. If you want to revert to the default behavior with only Linux security upgrades being automatically installed you should just remove /etc/apt/apt.conf.d/51unattended-upgrades-unifi. If you want to disable all automation apply command systemctl disable unattended-upgrades.

          1. Thank you, that was very helpful. I don’t mind messing with the Linux CLI at all. Your script just makes the initial setup quite hands off which is nice.

          2. Petri,
            can’t thank you enough for this easy setup. Based on your comment here, if I delete /etc/apt/apt.conf.d/51unattended-upgrades-unifi via SSH, I could update the Unifi controller to a version of my choosing and still enjoy your automated linux updates and other fixes applied during the nightly scheduled call?

            Thanks again for this work,

  14. What’s the best way to troubleshoot the autobackups not being sent to the Google storage bucket? It was working when the controller was first setup as I have one file shown in the storage bucket under the autobackup/ directory but none of the subsequent auto backups that are shown in the controller have been copied to the storage bucket.

    1. What does
      sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://your-bucket-name-here
      produce?

      If that works check /etc/systemd/system/unifi-backup.service to see if the bucket name is correct.

      1. I get the following output:

        CommandException: arg (/var/lib/unifi/backup) does not name a directory, bucket, or bucket subdir.

        1. My bad, I didn’t include the sudo. I’ll edit the answer above in case someone else tries it.

          Alright, the rsync works but your VM isn’t authorized. This is progress. Did you allow Read Write access to Storage for the VM?

          Another, later discovery was that some users had to enable the storage API. I added this update to the post:

          UPDATE: Some readers have reported that they needed to enable the storage API in APIs & Services > Dashboard > Enable APIs & Services > search for Google Cloud Storage JSON API > click Google Cloud Storage JSON API tile > click Enable button.

          1. I confirmed the API was already enabled. In Cloud API access scopes I had Storage set to Read only. I set it to Read Write and started the VM backup. I still get the same output as I showed above.

            AccessDeniedException: 403 Insufficient OAuth2 scope to perform this operation.
            Acceptable scopes: https://www.googleapis.com/auth/cloud-platform

            1. Good, you found the cause. You have corrected it, but nothing in the Cloud World happens instantly. Everything is queued and cached. I suggest you wait til tomorrow.

              If it doesn’t work by tomorrow you should spin up another VM with the proper permissions. Don’t work too hard trying to resurrect broken VMs. Creating a fresh one is the Cloud Way. If you want to learn something new you can detach the current virtual disk and attach it to the new VM so you won’t have to install anything. Although running the script is probably easier and faster.

  15. I’m curious if there are some special settings for the Guest Portal access. I noticed today that guests could connect to the guest wifi but they were not automatically redirected to the guest portal. I’ve added the hostname of the controller to the pre-authorization list. Any ideas why they might not be automatically redirected?

    1. None that I know of. Over here in Finland Guest Portals are rarely used, though. The guest portal redirection is layer 3 anyways, so running the controller elsewhere shouldn’t make a difference.

      Check first that your GCP VPC Firewall allows traffic to the guest portal ports 8880 and 8843. You can check it with the browser: http://your.controller.dns:8880 and https://your.controller.dns:8843 both should produce some result.

      What are your settings in UniFi Controller Settings > Guest Policies ? You could also ask this in the UniFi Wireless forum, where there are many more users who could help you.

  16. Thank you! For some reason, i had small problem with java (or maybe with apt). Unifi daemon did not start because java was not found. When entering java –version, system reported openjdk java fine. But for some reason this was not enough for Unifi.

    I had to ssh in and run sudo apt-get install default-jre -y

    Apt wanted to install several packages (like, +30). After everything was installed and rebooted, i was able to log in to Unifi

    1. Was this just recently? With UniFi 5.8.24? I haven’t rolled a new VM with it yet. My script doesn’t install Java by itself. It should get installed as a prerequisite for UniFi Controller. Apparently someone has goofed upstream. Thanks for reporting this, though.

    2. I confirmed this. I will try to get a response from Ubiquiti whether they will fix their package or should I write a workaround. I added an update at the top of the article about the situation.

      1. Wow, quite the response time! Yes, this was this morning. Thanks for all effort you put in, will be big help to everyone who wants install Unifi to Google VM

        1. I decided to solve this by installing JRE8 manually before UniFi Controller. The script is now updated (and some old scruff was polished as well).

  17. I followed these steps but the webpage never seems to load, I just get “This page isn’t working”. I just ran the script this morning and not sure if I’m doing something wrong.. how do I troubleshoot the installation?

    1. This morning there was a red warning at the top of the article about a problem. It was reported yesterday by GJ and I wrote a workaround today. It has been up for an hour now. Your easiest solution would be to delete the VM you created and spin up a new one. You should still have the backup on hand so it won’t take you but 10 minutes.

    2. I do see an error in script..
      unifi-debian unifidb-repair.sh[631]: bash: /var/log/unifi/gcp-unifi.log: Permission Denied

      Should I worry about this? The above seems to be the only error I’m getting.

      1. But no web page? That is something to worry about.
        The permission denied you are seeing is just about logging, nothing to worry about. It is odd though, since the systemd unit is run as root, so something is very wrong there. If the installation is hosed then I still recommend setting up a new one.
        If you still get this with the new VM please add a comment.

  18. Any idea why I am unable to get my controller to enable cloud access? When i attempt to enable the following error is received “There was an error saving the cloud access changes. No cloud license is assigned.”

    I can access the controller directly without any issues so i guess it’s not that big of a deal, however this was never an issue on AWS, RasPi or Windows hosted controllers.

    1. I never figured out. I had it happen on Cloud Keys and other local servers, too. Occasianally disabling and enabling it a couple of times did the trick – or was the trick waiting for a few days in frustration, I never knew. Trying different accounts didn’t help IIRC. After I switched over to remote controllers I no longer bothered with Cloud Access. Now I can manage the network from anywhere with a direct https connection instead of screen sharing magic. Downloading backups works, less broken or stuck connections, less trouble overall. I haven’t missed Cloud Access at all.

      Cloud Access is very useful for a local server, I admit. It’s a luxury to login from home instead of driving to the office to reboot a misbehaving AP. If you want to troubleshoot it you should ask on UniFi forums

    1. For some reason the Ubiquity downloads were unavailable. Testing those links now everything appears to be OK so perhaps it was some transient error? Have you tried again? And I mean deleting that VM and creating a new one today.
      No, you weren’t doing anything wrong and there is nothing you or I could do about Ubiquity downloads repository or GCP network connecting to it.

    2. On a second thought: the NTPd error just before could indicate there is something wrong with the network. The VM did load the startup-script from my bucket, so it is not completely isolated. My script doesn’t set up NTP because Google does it for us. It uses the virtualization host as the NTP server so it should be closer than the Storage bucket. I can’t say. Just give it some more time.

      Perhaps you should switch to some other zone within your region or to another region altogether. I just tested this in us-central1-c and it went smoothly.

    1. You need to set up authenticated SMTP to some server. You can use your ISP’s SMTP or it could even be Google’s. There’s a paragraph on this under Maintenance and troubleshooting.

  19. Thanks, I have this up and running it’s been really useful. Just been working on sending emails, signing up to Sendgrid via GCP appears to offer more than their normal free plan in terms of number of emails you can send and although the GCP instructions talk about using postfix, the Unifi Controller will allow us to specify the port for sending email. So as soon as sendgrid is configured and we can create an API key and use port 2525 which isn’t blocked to send emails. 🙂

    1. +1 for me, signed up for Sendgrid and used smtp.sendgrid.com / 465 / SSL with the auth credentials that Sendgrid prompted me to use from my control panel in the SMTP server config of the UniFi controller. No need for the bits in the Sendgrid guide on the GCP marketplace for Postfix.

  20. Any way to force update to 5.8.28?
    Installed today but got an issue when importing the backup as my PC already has 5.8.28 and the VM now has 5.8.24.

    1. You need to log on to the VM through the Google Cloud Console SSH button. Then download the version of your choice using curl. Install the package via sudo dpkg -i unifi_sysvinit_all.deb

      1. Hi, I’ve rebooted the VM but still running 5.8.24 and it didn’t upgrade to 5.8.28.

        Is that correct – you mentioned the VM would auto upgrade to stable releases so I assume 5.8.28 isn’t considered stable currently.

        1. After a release is promoted from stable candidate to stable it still takes about a week before it is made available in the repository. Most often it appears on a Monday, but that is not a firm rule. When the release is in the repository unattended-upgrade will install it after 04 AM automatically. No reboot is required. See UniFi Controller Releases

          1. Thanks so much for your reply, makes perfect sense to me. I’m in no rush to upgrade, just more wonding to make sure my install is working as it should (auto upgrade etc).

            You’ve done a great job putting all this information together, really appreciate it all.

      2. Thank you! Figured our myself after reading on the forums that it takes one week for the new release to be added to the repo. Used wget and dpkg but the fun thing was that I could do this from my mobile using the Google cloud app which has built in SSH. That was an epic experience 🙂

        Thank you very much for publishing this solution and keeping it updated. It’s working perfect for me now 🙂

    1. You have two alternatives. The easy one is to wait for a few days for the latest release to hit the repos. Unattended-upgrade will install it for you overnight. The other one involves Unix command line and here is a copy of what I answered Balazs Szabo above:
      You need to log on to the VM through the Google Cloud Console SSH button. Then download the version of your choice using curl. Install the package via sudo dpkg -i unifi_sysvinit_all.deb

  21. Petri – you must have put a lot of time and effort into this so thank you very much for that.

    This is the first time I’ve tried Google and your instructions were excellent. I had a controller up and running in no time and I’m pretty impressed by the performance considering it’s free. I’m now using the instance as my test controller having disabled the auto-upgrade job and done a manual upgrade to one of the Beta versions. Working great 🙂

    Thanks again..

    1. I wrote the script and the article because when I suggested people to use cloud based controllers I got two objections: “I don’t know how to use cloud services” and “I don’t know how to use Linux command line”. The first one can be learned in minutes and the second one can be completely circumvented with the script. My idea was that user never needs to log on. Your post reveals that you are not in the intended target group 🙂

      You can use the script as a starting point, though. If you look at GCP Unifi Controller startup script explained you can see what the script does and at the end there is a list of files which were modified during installation. That post is not quite up to date with the current version, but close enough. I have a couple of improvements in the works and then I’ll update it.

      1. No. You may want to look what I answered Rob on 2018/11/21. Rob wanted to install a legacy version. You can do the same to install a beta version. There is no way to auto-update the beta, though. You need to download the new, remove the old and install the new package of your choice.

  22. First thing is thank you for all your work with this script and tutorial!

    Next thing is I wonder how the new billing setup on google cloud will affect the tutorial and then free teir.

    From the email.
    “Hello Google Compute Engine Customer,

    We are writing to let you know that on October 1, 2018, Google Compute Engine instances with predefined machine types will be billed by their individual resource types (i.e., vCPU, Memory) instead of by their machine type (e.g., n1-standard-2).

    What do I need to know?
    In order to provide additional savings and simplicity, starting October 1, 2018, predefined machine types will be reported as individual vCPUs and memory SKUs. This means that you will receive sustained use discounts for your instances based on your use of the individual vCPUs and GB of memory instead of being limited to instances of the same machine type.

    What do I need to do?
    The changes will take effect automatically and will result in either the same or lower effective bill. No action is required on your part.”

    1. I don’t KNOW, but if you trust Google then the last paragraph says the charges won’t increase. Since zero charge can’t decrease either we’ll have to assume it will stay free.
      I hadn’t seen that notice, thanks for the heads-up. [Edit: I received the same email a couple of hours later.]

  23. I followed the instructions, but the webpage won’t load. It just says Unable to Connect. I can ping the IP address, but I cannot access 8843 through the browser. I made sure all the ports were allowed in the firewall rules, but it acts like it basically ignores them all. If I use the Open Port Check Tool, all the ports are closed. #Frustrated!

    Can anyone help me please? Thank you.

    1. Can you access port 80? It should redirect to port 8443 immediately but you can see the redirect in your browsers developer console. Lighttpd is listening on 80 and doing the redirect. Another debugging option is to look at Serial Port 1 (console) to see the output. The link is available on the VM details page.

      Usually it is easier to delete the VM and start over. You have made a small error at some point. Hopefully you won’t make the same error twice. On popular omission is to not assign the correct network tag to the VM. The network tag ties the firewall rules and the VM together and it would appear as you describe. Watch the video once more and make notes of the steps.

  24. I already have my controller running on GCP. But I haven’t been successful in getting a cert applied. Do you have a version that checks to see if a stage is already done and skips those steps? I don’t want to mess up my install. Thank you!

    1. My script checks for most of the steps so it won’t retry them on every boot. It can’t detect what you have done manually since there are so many ways to skin a cat. I didn’t intend my script to be run on top of a manual installation so I don’t recommend it. If the cert is the only problem why not try this script by Steve Jenkins. It does pretty much the same as mine in a single-shot fashion.

  25. Thank you so much for pulling this together Petri, I’m sick of having cloud keys failing so was thrilled to find this tutorial and amazed to have a controller running 45mins after signing up for a GCP account. You’re a legend!

      1. Thanks! Although I have one small problem; I made a typo when entering the value for my storage container bucket so backups aren’t transferring. Is there any way to correct this (eg editing the VM instance) or do i need to set up a new instance and run the script again?

  26. Hello Petri!
    Thank you very much for your hard work!
    A couple of questions: can I use the LTS version of the controller? Will your automatic update script override it with latest stable?
    And am I obliged to use DDNS? If Google gives you a static IP can’t I just use that? Or is it needed for Let’s Encrypt?

    Thanks!

    1. You need to edit the script. After you have edited it to install the LTS version you can either save it in your own bucket or copy-paste it to the startup-script meta-data field in the GCP console. Unattended-upgrades will keep all installed packages up to date. (I have never figured out how to install LTS via apt-get.)

      Another untested solution is to run the script as it is. Then log on and uninstall unifi package and manually install your choice. To prevent unattended upgrades from updating the controller you remove line "c=ubiquiti"; from /etc/apt/apt.conf.d/51unattended-upgrades-unifi

      No. You only need to fill in the meta-data fields you want. If you don’t add a bucket your backups won’t be copied. Without timezone GMT is used. If you have your own DNS records you don’t need ddns. If you don’t include dns-name you won’t get a certificate. Certificates can only be issued to DNS names, not to bare IP addresses.

  27. If anyone is confused on the dynamic dns portion, use the reserved static IP address (reserved in step 3) as your destination address. This is the name on freedns.afraid.org. The name on other sites may vary. Continue as Petri shows (in his video) using the direct URL in the value field of the ddns-url key and the subdomain.domain in the value field of the dns-name key field.

  28. Dear Petri,
    We want to use certbot .. after using your script.
    I did not use the ddns services ..
    What will be the root map for the certification of the ssl .. (var/www/html) .. in your script we have no ‘home-dir’ as like /srv/htdocs/[domainname]
    BTW .. the main page is still running with ssl error, nice script and video !
    Regards,
    Eric

    1. I’m afraid I don’t get what you are referring to with “root map for certification”. Current Let’s Encrypt certificates are in /etc/letsencrypt/live/yourdomain/ (actually there are symlinks to the certs). The script uses certbot’s –standalone plugin which will start its own web server for validation. That’s why I have to shut down Lighttpd for renewals.

      What kind of SSL error you are getting? I’ve never seen one or I would have fixed it 🙂 I occasionally check the setup with SSL Labs test which comes out clean. (I have OpenSSL 1.0 which doesn’t include TLS 1.3 so it gives me a B rating. An upgrade is on my to-do list…)

      1. The next error was coming up after the command:
        #certbot certonly –webroot -w /var/www/html/ -d wifi[domainName]

        Failed authorization procedure. wifi.[domainName] (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching https://wifi.[domainName]:8443: Invalid port in redirect target.
        Only ports 80 and 443 are supported, not 8443

        I will change 8443 to 443 for generating the key.

        1. This article is about the script I wrote. I am trying to help users who have problems with it. If you want to do it on your own you should read https://certbot.eff.org/docs/using.html#getting-certificates-and-choosing-plugins at least.

          If you need a script to fetch the cert then take a look at Steve Jenkin’s script. It is quite close to mine but will only fetch the initial certificate. You need to set up the update cycle and import the renewed cert to the Java key store.

          1. Dear Petri,

            At this moment I had made the correct ssl for port 443.
            When I check this on the url name: https://wifi.[domain_name]/manage/account/login?redirect=%2Fmanage
            But of course I get the error that the page was not forewarded correct.
            It has to go to port 8443.
            What will in combination with the lighttpd webserver the best way to do.
            A standard redirect to port 8443 .. will give the situation that the site is not secured.
            ssl cert. generation for port 8443 is not allowed .. can you explain how you fixed this with the ‘ddns solution’ in your script.
            From their I can hopefully make from their to the correct forwarding from port 443 to 8443, what unifi needs.

            1. Redirections won’t help with a cert error. My guess is that you haven’t imported the cert to the Java keystore as per Steve Jenkin’s script.

              If you need help with Lighttpd configuration and redirects then you should look at Lighttpd documentation, forums and mailing lists.

    1. Yes, it can be done, but my script is GCP specific. It would be quite easy to adapt: how to access the metadata and the rsync command are the only specifics that come to my mind.

      1. Let me first say, thanks for the GCP script.
        It was super, super easy to setup!!

        Is making a Vultr script on your bucket list?

        1. No, I am not going to expand this. I wrote my script as a proof of concept that you actually can install UniFi controller in a cloud VM without any Linux skills. Many part time admins are not comfortable with remote VMs or Linux command line. I chose GCP because you can run the controller there for free. Anyone can fork my script from GitHub and modify it for Azure, AWS or whatever they desire. You could do it for Vultr! The only CGP specific parts are accessing the metadata about the VM (very similar in other system) and rsyncing the backups to a storage bucket (simple as well).

  29. Thanks so much for the script and this amazing guide!

    One question: does the unifi controller software updates by itself? Or do i have to use apt update to get the latest version?

    Thanks

    1. Yes, it will. Not at once though, but after Ubiquiti uploads the latest release to the repositories. It is typically about a week from the announcement. The idea is that the user never needs to log in to the VM. Everything is done within UniFi Controller or GCP Console.

  30. Is there no risk to leaving all these ports open to all ip addresses? I am working on a script that can limit what devices can talk to the server via iptables and specific ports that also auto updates iptables when the dynamic ip changes.

    Also has built in functionality to allow additional ports to be opened up for your dynamic “admin” ips.

    1. Yes, there are risks, but reasonable in my opinion. When a controller adopts devices they exchange credentials, which are used for future device to device communication. The biggest risk lies in the management interface where the user can choose the password. That’s why I stressed the importance of a good password. I also added Fail2Ban to the mix to make password guessing harder. Only the necessary ports are open in the GCP firewall. For example SSH is not and you would also need to take extra steps to allow remote logins to the VM.

      Typically the UniFi devices are behind a NAT device with a dynamic IP. Trying to track this automatically would introduce more points of failure with questionable benefits (since the devices are strongly authenticated). Limiting the IPs where the user can reach the management interface is even more tricky. Most probably the users would be locked out without skills to open the lock by logging to the VM and editing IPTables manually.

      You always need to choose a sweet spot between security and usability. I targeted my script to average, small scale users, with limited technical skills. Yes, you can harden your controller, but it will require more maintenance. If you require high security you should keep the server in your own data center. If you have satellite offices connect them by static VPNs so all connections are controlled and secure. For remote administration you should set up dynamic certificate-based VPN to the controller.

      1. Well said! Thank you for this script btw.. it works awesome! I wasn’t aware of how strong the security was between the devices and the controller.

        1. Do not use the pastebin from above as it will leave you unable to access your server on port 22.

          I have update the below script to take into account how Google Cloud handles SSH access. This script allows you to block all access to Google Cloud Compute Unifi Controller except by the dynamic dns names defined by you in the script. It will run as often as you want via crontab and polls the dynamic IP address and updates the firewall on the Unifi Controller if a new IP address is detected.

          https://pastebin.com/VA6T0hKj

  31. Hi,

    I just restarted the vm but in the console i see this:
    unifi-controller login: Sep 28 00:53:41 unifi-controller google_metadata_script_runner[778]: AccessDeniedException: 403 518175677229-compute@developer.gserviceaccount.com does not have storage.objects.list access to petri-unifi.
    Sep 28 00:53:41 unifi-controller startup-script: WARNING Could not download gs://petri-unifi/startup.sh using gsutil. Command ‘[‘gsutil’, ‘cp’, u’gs://petri-unifi/startup.sh’, ‘/startup-jZMNIO/tmpJaC8dm’]’ returned non-zero exit status 1.
    Sep 28 00:53:41 unifi-controller startup-script: INFO No startup scripts found in metadata.
    Sep 28 00:53:41 unifi-controller startup-script: INFO Finished running startup scripts.

    Can i just ignore this? Or do i have to restart again?
    Thanks

    1. It is my fault, I’m sorry. I did receive some e-mails from Google saying I need to upgrade, but I couldn’t find any hint in the GCP console about this upgrade. So I decided to ignore the e-mails. Apparently they were for real. Now I have “upgraded” my account and everything should be smooth again.

      This incident didn’t affect any controllers already running. The running controllers don’t need the script after the first boot unless you change the metadata. You can even remove the startup-script-url row from the metadata. You just couldn’t create new controllers while the script was not available. If you had your controller already set up you can just ignore the error. Or you can restart to verify the problem is gone, if you want to help.

  32. Awesome work but for the life of me I couldn’t connect. Console log shows:

    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Starting startup scripts.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Found startup-script in metadata.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO startup-script: /startup-J506OP/tmpdAqmV6: line 1: gs://petri-unifi/startup.sh: No such file or directory
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO startup-script: Return code 127.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Finished running startup scripts.

    Which I assumes means I have a lovely Debian VM without your script running

    1. I’m afraid your assumption is correct. Although the script wouldn’t continue running, it would only install what was required and quit. In your case it didn’t run at all.

      I couldn’t reproduce this. I tried with my own account, but then it could be some obscure permissions issue. I got a friend to spin one up on her account and it succeeded as well. Could you please delete the VM and try again? Please be careful with extra characters that may be produced by copy-paste. I can’t see any on the third line of your console output, but it might be the cause. Or Google had a hiccup on their system. As you can read from the comments, many have succeeded with the first try. I believe even more don’t report it in a comment.

        1. Works for me™ Your mileage may vary.. Both upgrading an existing controller and creating a new one. Usually new Controller releases aren’t the problem. The problem lies in the compatibility with prerequisite software. If only Ubiquiti would make their software compatible with current Java and MongoDB versions.

  33. OK, so what needs to be done to allow public key SSH? 🙂

    I’m able to use the SSH button in the cloud console, and I can use ‘gcloud conpute ssh [unifi-instance-name]’. But when I try to ssh from a host a home, it fails. I’ve allowed port 22 on the VCP Network Firewall rules. And I can see that the traffic that matches that rule is allowed and passes to the host.

    Is there anything else in the VM that needs to be changed to allow SSH? (I’ve added my keys in the instance already).

    Thanks and awesome job on this!!

    1. This isn’t about UniFi Controller at all but GCP VM administration. I wrote the script so people would never need to log on to the VM.

      GCP creates a new, temporary user account when you log on through the Console. When you log out the account is removed with the keys you installed. You need to create a new, static user with adduser and install the keys for that user. If you want to use sudo for the new user, you need to create a new file in /etc/sudoers.d with
      username ALL=(ALL:ALL) NOPASSWD:ALL.

  34. Petri, thank you for sharing this with the community, it’s absolutely brilliant.

    I’m all up and running although as I don’t have a domain name to use at the moment I left the SSL config key out. I’ll probably just access the controller via unifi.ubnt.com so can you think of any reason to apply a DNS name / SSL cert (incase there’s something I’m missing).

    Also can you see any issues with turning auto-backups to weekly in the UniFi controller?

    Thank you again for sharing!
    Dan

    1. You can use it without a domain name. You can get a domain name for free, though. I personally had far too much trouble with WebRTC that the cloud service is using. I find it easier to use the controller directly over https. Keep your password long and complex. Anyone can try to guess it over http.

      Your free bucket allowance is 5GB and there is about the same space available on the VM. As long as you don’t fill either one (they are synced) with the backups you should be good to go. I like to keep 26 weekly backups (or 6 months worth) but they are quite small. The network is small so there isn’t that much statistics to gather.

      1. I found an old domain name I had for testing, added the key for the certificate, shutdown and restarted the VM and I’m in business, SSL all added – thank you again!

  35. Thanks!

    Notes: In my case, on Firewall create, there are separate fields for TCP and UDP, had to paste the port list to each separately (tcp: “8443,8080,8880,8843,6789,443,80,22” and udp: “3478”.

    Oddly enough, when editing the same rule, there is only one field: “Specified ports”, containing “tcp:8443,8080,8880,8843,6789,443,80,22; udp:3478” 🙂

    other than that it seems to be working very well. How much can I scale this (if I pay)? Should I set an administrative limit every such-and-such many sites/devices/clients?

    1. The user interface has changed – thanks for the heads up! I’ll update the instructions, but I am not going to shoot the whole video again. Google keeps making these small changes and they never announce them.

      There is very little information on scaling. Take a look at UniFi – Server Hardware & Database Management. It basically says that you should keep monitoring the CPU & RAM usage. They are mostly affected by the number of APs and other network devices unless you run a captive portal as well. In that case the number of client devices starts to matter as well. The good thing about cloud computing is that you can easily move to a bigger instance as needed. You don’t have to buy spare capacity that you expect to need, you can always use exactly the optimal size. Oh, if you are going to pay for the service use the nearest Google data center to keep the latency down.

  36. Hi, I’m new to GCP and of using your script. A couple of questions:
    1. Do I run your script from the GCP command line? (I’m on a Mac and had a bunch of “command not found” when trying to run the script in Terminal.

    2. Do I setup the DDNS after the controller has been created by the script? (How do I know what the IP address of the VM is otherwise?)

    Thanks for all your work on this.

    1. 1) You add a startup-script-url metadata field in the virtual machine config. That way the the VM will run the script automatically at boot. Watch the video one more time.
      2) If you want to set the DDNS statically then you can set it up when you have reserved the IP address. You don’t need to add the dns metadata field in that case.

  37. I’m getting some possible errors when running the script. In particular, a few “is configured multiple times” and “systemctl not found”. I’m running the script from my home directory. I started it, then cancelled to run as root. Screenshot is on Dropbox and linked.

    1. You are apparently running the script manually on your local machine. My script is intended to be run automatically at boot on the remote virtual machine. That’s why it doesn’t work. Did you watch the video one more time as I suggested? There is no command line involved.

  38. I haven’t rewatched the video yet. I promise I will. I setup my VM following the command line info posted on https://www.instructables.com/id/Unifi-Controller-on-Google-Cloud-Platform-GCP/ (he’s how I found your web page). When I follow his last command,
    gcloud compute instances create unifi-controller \
    –machine-type f1-micro \
    –image-family ubuntu-1604-lts \
    –image-project ubuntu-os-cloud \
    –boot-disk-type pd-standard \
    –boot-disk-size 25GB \
    –zone us-central1-c \
    –description “Unifi Controller” \
    –scopes=default,storage-rw \
    –tags unifi-server \
    –address unifi-external \
    –metadata=startup-script-url=gs://petri-unifi/startup.sh,timezone=US/Central,dns-name=your.domain.com,bucket=some-unique-name

    it completes at command line without error. However, when I login to the new VM (I setup the DDNS and it works) the certificate didn’t load (he says the same thing on a question I posted on that page). Any idea what I might do to get the cert to load correctly? Also, does the script run every time the VM is called from a web browser? It’s OK if it does I just want to know for knowledge sake.

    1. You are asking me to help with MallocArray’s instructions? I haven’t ever tried them. I trust him though, but you should ask him for help.

      My script creates a log at /var/log/unifi/gcp-unifi.log Any hints there?

      Does the DDNS name resolve from any computer on the Internet? (Try from work, library, your cell phone using mobile data, neighbor’s..) It must resolve from Let’s Encrypt’s servers in some data center.

      The script runs on every boot in case you change some of the metadata. With no changes it won’t do much anything. You can remove the startup-script-url from the metadata if you don’t want to run it every time. (I trust you did change the metadata on the last line of the command: dns-name and bucket, didn’t you?)

      1. Hi Petri,

        No, not asking for help with MallocArray’s instructions. I worked through his command line stuff pretty easily. Just wanted to show you his command automatically calls your script. (I watched the section of your video a couple times to make sure the metadata was exactly the same. It was.)

        I can run the VM from every Mac and iOS device. So, that works. The DDNS setting is operational. I’ll have a look at the log, and post back if I find some useful info. Thank you so very much. You are appreciated!!

        1. The log only lists this:
          CertBot run on Mon Nov 5 16:54:48 CST 2018
          Certificate files missing

          This would have been about the time I finished with everything on 11/5/2018.

          1. I just created a new VM using my instructions. It got a certificate just fine. So we can sum this up: there are two sets of instructions. One that works (mine) and one that appears to have problems. Is there some reason why you won’t try my instructions? It will only take you ten minutes to create a new VM. You can even transfer the static IP to the new VM so you don’t need to tinker with DDNS.

            I also noted that MallocArrays’s instructions are for Ubuntu 16.04 LTS. I guess you are using Ubuntu, too. I have only tested mine on Debian. Ubuntu is based on Debian, but they do differ. I can only support you if you follow my instructions to the T (or you have some really exceptionally good reason not to).

    2. Watched the video again. I’m reasonably certain I’ve got everything configured properly. My outstanding problem seems to be the certificate. Please have a look at the web page I listed above. You’ll see the fellow who admins that page said he had deployed a new VM in the last couple of weeks and the certificate wasn’t working for him, either.

      1. Hi Petri,

        I’ve been in IT for quite a few years, and command line is not something I’m shy about. The only reason I haven’t followed your instructions to a T is because I was confused about what the script does & doesn’t do. (For whatever reason, I did not realize I had to start with 1. Preliminaries. I was under the impression the script did everything for me. My bad.) Am I correct that I have to follow the step-by-step as outlined in both the video and the verbal post here (beginning with preliminaries)?

        I was also going to try adding CertBot to my existing VM but wasn’t sure of the choice for I’m using (Apache, Nginx, Haproxy, Plesk or None of the above).

        The Ubuntu VM I created uses 25GB, so I’m thinking I’d have to destroy that or register a new Google Cloud account in order to use your instructions. Does that seem correct?

        1. I made the video to give you the big picture and help decide if you will be able to make it through. Creating a virtual Linux machine in some cloud service sounds intimidating if you have never done it. When actually doing the steps it is easier to have them written down instead of pausing the video. Essentially they are the same thing, but there is more detail in the text.

          If you just created your GCP account then you should have received a couple of hundred dollars worth of credit. You can run several VMs side by side for the money. Even if you don’t, it won’t cost you but a few dollars. You can only attach the IP to one VM at a time, though.

          1. Ultimately, I stopped the Ubuntu VM, changed the static IP it was using to None, then deleted the VM. I followed your directions. The only problem was when I got to the Networking section, it didn’t see the original reserved static IP. I released that, created a new one and attached that to the new VM. After I created the new VM and having changed the DDNS IP, I connected to the new VM & YES, a certificate was attached. Thanks! Now, I’m trying to figure out how to get the APs which are still connected to the proper sites in the VM but show as Disconnected probably because they were setup and adopted on the local Unifi Controller. I’m guessing until I’m back onsite with those APs I won’t be able to get them on the new cloud controller.

            1. You followed instructions and succeeded. Good! There might be a lesson in there 🙂

              Did you also go through step 5 to the end? You should first set the Controller Hostname/IP on the GCP controller and then on the old controller. That’s what will make the devices connect to the new controller.

              You probably had the IP in a different region. Static IPs are tied to a region, you can’t move them. Makes perfect sense if you think about it from the routing perspective.

  39. There’s no question there was a lesson in there and the main reason I went thru all of this was to learn. Success is a great motivator!

    I would like to make one suggestion. I think a preamble explaining to very newbies such as myself, that one HAS to go through the steps to make this all work is important. I never think I’m so unique I’m the first to have the ideas or questions I have. So an explanation that the steps need to be used and THEN the script kicks in at the end would have helped me. If this is the very first time someone didn’t understand (me) I will chalk it up to a new experience.

    Thank you for a great tutorial, but just as grateful you are willing to respond to help anyone who asks questions. So, I have one more: In the reply process, I give my email. Is there anyway to get an alert via that email you have responded? Another of my naive expectations. I thought I’d get an email after your responses, but those didn’t come. I didn’t see your first response til the early afternoon yesterday. Regardless, you are a prince! Very big thank yous for teaching me something new!!

    1. Thanks for your input, but I try to minimize edits to old posts. Let’s see if I get more complaints. I also feel I would belittle readers, if I wrote a cookbook and started by explaining that you really need to take all the steps. You can’t just jump to the end and expect to find a roasted turkey in your oven! 😛

      This site is built around WordPress. WordPress used to notify commenters, but apparently the function has been removed. The email address is not verified by any means so it would be very easy to bully someone by generating thousands of comments around the Internet with the victim’s email. It is sad the Internet has become the environment it is today. I have been using the Internet since the early 80’s, when you really could email any textbook author or a professor in any top university and expect a reply.

  40. I’m having a tough time getting this to work. I got it to work initially once, but then I deleted the VM and started over. Since then, just can’t get anything working anymore.

    Looking at the console, here is what I find.

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 26.)

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:

    [ 27.127267] random: crng init done

    [ 27.131931] random: 7 urandom warning(s) missed due to ratelimiting

      1. It seems I’m still getting an issue. It doesn’t look like the script is running.

        Looks like it finds the script and downloads it

        “`
        Nov 13 01:08:49 unifi-controller startup-script: INFO Downloading url from https://storage.googleapis.com/petri-unifi/startup.sh to /startup-CjurIU/tmpgH4bCM using authentication token.
        “`

        “`
        unifi-controller login: Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to stop mongodb.service: Unit mongodb.service not loaded.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to disable unit: No such file or directory

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 168: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/filter.d/unifi-controller.conf: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 172: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/jail.d/unifi-controller.conf: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 180: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/jail.d/unifi-controller.local: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to reload-or-restart fail2ban.service: Unit fail2ban.service not found.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to start dbus.service: Unit dbus.service not found.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Trying to start dbus

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Failed to start dbus.service: Unit dbus.service not found.

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Failed to create bus connection: No such file or directory
        “`

        “`
        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Backups to snunifistorage set up

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Return code 0.

        Nov 13 01:09:09 unifi-controller startup-script: INFO Finished running startup scripts.
        “`

        This is everything I have with startup-script

        1. Which region/zone are you using? I just tested in us-east-1b and it works. It could be a discrepancy between zones or regions. The suggestion to use ‘dpkg –configure -a’ tells me it is the same GRUB bug.

          1. us-east1-b is what its sitting on.

            I will delete the VM and start over again, see if it changes anything.

          2. Just wanted to let you know it works now!

            I wonder if it was loading a cached version of the script then?

    1. The script will do it for you. You just enter the dns-name metadata. If the DNS name matches with the public IP, the script will acquire a certificate. If you want to download it, it is in /etc/letsencrypt/live/

  41. Hi, installed perfectly but the lets encrypt cert is saying its invalid with the reason below

    This CA Root certificate is not trusted. To enable trust,
    install this certificate in the Trusted Root Certification
    Authorities store.

    1. What OS are you using? (Win/macOS/Linux/iOS/Android/..) Which browser?

      Is the certificate really issued by “Let’s Encrypt Authority X3” and the root is “DST Root CA X3”?

      1. Hi Petri,

        cert says issued by unifi, im using windows and have tried all browsers with no luck?

        Certificate
        General Details Certification Path
        Show:
        X

        Field
        Version
        Serial number
        Signature algorithm
        Signature hash algorithm
        Issuer
        Valid from
        Valid to
        n S. ihi^rt_________________________
        Value
        V3
        5be2c937
        sha256RSA
        sha256
        UniFi, UniFi, ubnt.com, San Jo.
        07 November 2018 11:15:03
        04 November 2028 11:15:03
        I IniFi I IniFi i ihnf rnm ?an In
        V3

        1. So it is not the Let’s Encrypt certificate that is not trusted, but the default UniFi certificate. The script couldn’t acquire a Let’s Encrypt certificate. The most probable cause is that Let’s Encrypt server cannot connect to your instance. It could be a firewall issue. My instructions tell to allow TCP to ports 443 and 80 on the VPC for this reason. It could be a DNS issue. Does the dns-name resolve universally to the instance IP address? Universally means that from anywhere, not only on your computer. You should try it at home, work, library or on neighbour’s computer.

            1. Just to verify: You get the bad certificate warning if you just type the DNS name into you browser address bar? Like your.server.dns.

              If you want to debug this further you need to SSH into the VM command line. There is a SSH button in the GCP Console. You can view the script log with command
              sudo more /var/log/unifi/gcp-unifi.log
              Other logs that may help are:
              sudo more /var/log/letsencrypt/letsencrypt.log
              sudo more /var/log/unifi/server.log

              The other way is to spin up a new VM and transfer the IP address and the backup of your UniFi Controller to it. Keeping your fingers crossed that you’ll do something slightly differently this time. If it is a DNS problem it won’t help, though. While you are at the command line you should test how the VM sees the DNS with command
              getent hosts your.server.dns

    1. I am not able to reproduce this. Neither with a new controller or an upgraded one. Did you use my script for your setup?

      Joseph McParland in Willie Howe’s YouTube comments suggested the problem stems from the Windows CRLF combination. My script is pure Unix style LF. Thanks for the heads-up, though!

  42. Hi Petri,
    great video and instructions. Three concerns/questions:
    1. When I type xxx.duckdns.org nothing pops up in my browser (xxx would be the name of my DDNS name)
    2. Can I edit the values once the VM is created?
    3. Can I delete the entire project and start over? What does this do to my Lets Encrypt Cert?

    Thanks

    1. 1) There is something wrong with your DNS. You should check to see where does your DNS name point to. There are some web tools or local command line tools like nslookup and dig.
      2) Yes. The new metadata will take effect when you stop/start your VM.
      3) Yes. Your cert will be deleted with the VM. LE will issue you a new one. However, LE will issue only five certs per week to all nn.duckdns.org servers together.

  43. Hi Petri,
    I”m not sure if I got this to work. I’m not entirely clear on DDNS vs DNS. Right now, I only have a duckdns domain. Let’s say it’s called test.duckns.org Am I also required to have a DNS domain e.g test.net? Right now, my duckdns points to my public IP (at my house).
    Also, I’m starting from zero on this project. Do I need an old Unifi controller setup to proceed? I don’t have one. I was hoping to not have to buy a Cloud Key to get this to work
    Thanks

    1. The difference between dDNS and DNS is that dDNS can be updated dynamically without editing a config file or restarting a server. dDNS is used for dynamic IPs that change frequently or even occasionally. In case of GCP VMs the IP won’t change, but dDNS names are available for free. My idea was that anyone could deploy this solution without spending a nickle.
      No, you don’t need test.net, test.duckdns.org will do fine.
      You need another DNS name that points to the IP of your controller in GCP.
      Have you read the section “How to Set Up a New Network with a GCP Controller” above? In my opinion option A is the easiest (if you have a laptop) – that’s why I put it first.

  44. Petri, amazing script. question for you. I would like to spin up an older version of the controller software, 5.6.39 or .40 because I have some legacy devices that have fallen out of support by the newer controller versions, how easy is it to modify your script to pull a specific version (this could be a useful metadata key, if it is there to pull that version perhaps?) ubnt will EOL their devices every few years and lots of people cant afford a complete upgrade for that reason…

    cheers!

    1. You can do it, but it requires Linux command line. After you have spun up the VM, SSH into it from the GCP Console. Remove the latest UniFi package and install the one you fancy. You need to edit Unattended-Upgrades configuration and remove the line with "c=ubiquiti"; Otherwise the controller will be automatically upgraded to the latest one on the next run. The commands you are going to need:
      sudo apt-get remove unifi
      sudo nano /etc/apt/apt.conf.d/51unattended-upgrades-unifi
      curl -o installer.deb https://dl.ubnt.com/unifi/5.6.40/unifi_sysvinit_all.deb
      sudo apt install ./installer.deb
      rm installer.deb
      sudo systemctl enable unifi
      sudo systemctl start unifi

      This is from the top of my head, not tested. Should be close, though.

      No, I’m not going to make this a metadata option. It would be a potential source for far too many errors. Anyways, you need to log on to the VM to update the controller. My target were users who are never going to SSH to the VM.

      1. you sir are fantastic, thank you very much for the prompt response and a very very very useful script. I will give this a shot, thank you!

  45. Hi Petri,
    I set up my Google VM per your instructions. Just waiting for a cert from let’s encrypt for duckdns. I do have a question: I use OpenVPN TCP443 on my router and duckdns refers to my IP address. WIll there be a conflict if both Open VPN and Google VM use port 443?

    1. You need two DNS names: One for your home router and another one that points to the external IP of the GCP VM.

      1. Sorry for my confusion, let me see if I understand:
        I need one DNS for my router (xxxx.duckdns.org) which I currently use for OpenVPN on my router. I should not use this one when I configure my GCP VM, correct?

        I need another DNS (yyyy.duckdns.org) for my GCP VM, and this does NOT need to point to my router. This one is used only for GCP VM, poining ot the external IP. correct?

        Lastly, If my DDNS is yyyy.duckdns.org, is that also my DNS entry for the GCP VM? This is where I get lost. Thanks

        1. Yes. You need two names for two addresses thousands of miles apart. The yyyy.duckdns.org should point to the external IP of the GCP VM and you use that name to access your UniFi controller.

          1. Ok,
            Thanks. The only problem is it’s not working. I must have done something wrong. When I type yyyy(not real name).duckdns.org, nothing shows up in my browser.

              1. Does ping yyyy.duckdns.org work or can you only ping ip.ad.re.ss ?

                If pinging the name doesn’t work then you need to figure out where the name points to. On a Mac you use dig yyyy.duckdns.org and on a Windows box use nslookup yyyy.duckdns.org from the command line.

                1. When I ping yyyy.duckdns.org (not the IP address) I get a valid return: (Note, the IP address below is fake so I can post it, as is the DDNS name). The actual IP address IS the assigned IP by my GCP VM

                  Microsoft Windows [Version 10.0.17134.407]
                  (c) 2018 Microsoft Corporation. All rights reserved.

                  C:\Users\Bud>ping yyyy.duckdns.org

                  Pinging yyyy.duckdns.org [25.263.251.45] with 32 bytes of data:
                  Reply from 25.263.251.45: bytes=32 time=37ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=24ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=24ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=23ms TTL=55

                  Ping statistics for 25.263.251.45:
                  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
                  Approximate round trip times in milli-seconds:
                  Minimum = 23ms, Maximum = 37ms, Average = 27ms

                  C:\Users\Bud>

                  1. Microsoft Windows [Version 10.0.17134.407]
                    (c) 2018 Microsoft Corporation. All rights reserved.

                    C:\Users\Bud>nslookup yyyy.duckdns.org
                    Server: resolver1.opendns.com
                    Address: 208.67.222.222

                    Non-authoritative answer:
                    Name: yyyy.duckdns.org
                    Address: 25.263.251.45

                    C:\Users\Bud>

                    1. If the name resolves to the external IP of your VM then stop and start the VM. If it doesn’t solve the problem delete the VM do it over. It will take 10 minutes instead of days of troubleshooting.

                    2. Deleted VM and started all over again. Still batting zero. Not sure whatI’m doing wrong.

                      When I type yyyy.duckdns.org into the browser, I get nowhere.

                      This site can’t be reached yyyy.duckdns.org took too long to respond.
                      Try:

                      Checking the connection
                      Checking the proxy and the firewall
                      Running Windows Network Diagnostics
                      ERR_CONNECTION_TIMED_OUT

                    3. What happens if you type the external ip.ad.re.ss into your browser?

                      If you want to debug this further: Create a new VM and when it finishes open it in the GCP console and click on the Serial port 1 (console) link. There you can see the story evolve. You need to click the Refresh button to see more. The lines containing INFO startup-script-url are generated by my script. Don’t worry about the lines where debconf or dpkg whine about there being no terminal.

                  2. The same thing happens when I type in the external IP address (This site can’t be reached). However, if I ping the external IP or my DNS name (yyyy.duckdns.org) I get a valid ping return.

                    Not sure why the browser will not let me in. I’ll try establishing another VM and report back.

                    1. Double check your firewall rules and that you have typed the network tag exactly the same way in the firewall settings and the VM settings. Sounds like the firewall is keeping you out. It has kept Let’s Encrypt out as well so there will be no certificate. Once you have the tags and rules checked you should create a new VM.

                    2. I did see these entries on the start-up script. Normal?

                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 21.)
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:

                    3. Yes. Debian package installer is wondering why there is no terminal. There is no terminal because it is a headless server.

                      Did you doublecheck the firewall rules and tags?

                    4. allow-unifi
                      Description
                      Unifi Firewall Rule
                      Logs
                      Off
                      view
                      Network
                      default
                      Priority
                      1000
                      Direction
                      Ingress
                      Action on match
                      Allow
                      Targets
                      Target tags
                      unifi
                      Source filters
                      IP ranges
                      0.0.0.0/0
                      Protocols and ports
                      tcp:8443
                      tcp:8080
                      tcp:8880
                      tcp:8843
                      tcp:6789
                      tcp:443
                      tcp:80
                      udp:3478
                      Enforcement
                      Enabled

                    5. I “may” have found the problem. My external IP is on
                      us-east1 but my VM is on us-east1-b.

                      Is that what’s causing the problem?

                    6. No. The IP is tied to us-east1 region and your VM is in zone b. You can’t attach an IP across regions.

                      Do you have the network tag “unifi” also on your VM?

                    7. Ok, I deleted everything and started over. Now, when I type in yyyy.duckdns.org I’m taken to the following:

                      Placeholder page
                      The owner of this web site has not put up any web pages yet. Please come back later.
                      You should replace this page with your own web pages as soon as possible.
                      Unless you changed its configuration, your new server is configured as follows:
                      Configuration files can be found in /etc/lighttpd. Please read /etc/lighttpd/conf-available/README file.
                      The DocumentRoot, which is the directory under which all your HTML files should exist, is set to /var/www/html.
                      CGI scripts are looked for in /usr/www/cgi-bin, which is where Debian packages will place their scripts. You can enable cgi module by using command “lighty-enable-mod cgi”.
                      Log files are placed in /var/log/lighttpd, and will be rotated weekly. The frequency of rotation can be easily changed by editing /etc/logrotate.d/lighttpd.
                      The default directory index is index.html, meaning that requests for a directory /foo/bar/ will give the contents of the file /var/www/foo/bar/index.html if it exists (assuming that /var/www is your DocumentRoot).
                      You can enable user directories by using command “lighty-enable-mod userdir”

                    8. You need to have exactly the same network tag on the firewall rule and the VM.

                      Apparently you did something differently this time. You got connected to Lighttpd default page. What does https://yyyy.duckdns.org:8443 yield?

                    9. Good! When you got your DNS name properly configured and matched the network tags.

                      I can’t say why Lighttpd isn’t redirecting you. There is nothing you could have done to cause this. There are no choices in the script for it. It is a glitch, because I have never heard anyone else running into it and I can’t reproduce it either. My suggestion is to create yet another fresh VM now that you know all the steps.

                    10. That was it!!!!! I deleted everything and started from zero.
                      yyyy.duckdns.org now goes to the unifi controller page.

                      Thanks Petri.
                      I am getting a message on my VM :Instance “yyyy-controller” is overutilized. Consider switching to the machine type: g1-small (1 vCPU, 1.7 GB memory).

                      Is this normal?

                      Thanks again.

                    11. Yeah, that is normal. Google wants to sell you a bigger VM. I admit f1-micro is sluggish but I can live with that.

  46. Need some help here –

    I am getting a certbot error –

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO startup-script-url: /usr/local/sbin/certbotrun.sh: 7: [: x192.64.119.136: unexpected operator

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO startup-script-url: Return code 0.

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO Finished running startup scripts.

    Any thoughts?

    1. Sorry for the delay. My ISP goofed big time and stranded my servers. A total blackout for a few days and of course when I wasn’t there.

      I couldn’t reproduce this, so my advice is to delete the VM and create a new one. The error you posted is real, though. It is not from CertBot but a line from my script. It looks like the text in the script was corrupted somehow, that’s why I was concerned. In this case it appears to be a random bit error, although those are really rare.

  47. Petri,
    Question: I do not have a Unifi profile to import yet. However, I wanted to put a username and password on the Unifi splash page for added security (in case someone guessed my DNS). I set one up. Can I still import my Unifi profile into the VM? Is there any way to reset it from zero and start all over again?

    Thanks.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.