Set up UniFi Controller on Google Cloud Platform

Google offers one free virtual machine on Google Cloud Platform. This script will make setting up a UniFi Controller on GCP a breeze and it includes all the goodies. No command line required, everything is done in the GCP Console and it takes 15 minutes total and that includes transferring your current sites to the cloud. You should try this!

[2023/02] If you are impatient and want to upgrade your controller to the latest version, see Jonathan’s comment below. If you want to upgrade your Debian as well, see Derek’s comment. (Use the search in your browser!) I am still waiting for Ubiquiti to come up with an automatic upgrade without the command line.

[2022/12] UniFi Network 7.3 will REQUIRE Java 11. I don’t yet know how smooth the transition will be. I will do my best.

[Note 2021/12] If you are getting a certificate error then see Fix GCP UniFi Controller certificate. It is finally solved [with help from Djo and Derek]! Basically you just need to stop and start your VM.

[Note 2021/09] If you are still being charged for egress traffic, check your network tier. Apparently Premium is on the free offer and Standard is not! [Thanks, gman!]

[Note 2020/01] If Google has charged you for egress traffic to some remote areas of the globe, you can stop Lighttpd as a fix. Log in to the VM and type sudo systemctl stop lighttpd and sudo systemctl disable lighttpd. That means you will need to use the full https://my.ctrl.dns:8443 URL, but your browser should already know it by now.

The script I have created will set up an UniFi Controller on GCP with these extras:

  • Automatically acquire and install a Let’s Encrypt certificate to relieve you from HTTPS certificate warnings. For a certificate you need to have a domain name for your controller, dynamic or static.
  • Dynamic DNS support to help you set up the domain name.
  • You don’t need to type https://your.server.dns:8443 every time. You just type the DNS name of your controller and it will be redirected to HTTPS port 8443 automatically.
  • Backup files will be copied to a Google Storage bucket for offline storage.
  • Fail2Ban protects your controller from attackers trying to brute force the password: the IP address of the offender will be denied access for an hour after every three failed attempts.
  • The free tier GCP micro instance comes with only 600MB of memory. The script will create a swap file to accommodate the controller.
  • The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases.
  • If the MongoDB database supporting UniFi Controller needs repairing, a reboot will do the trick. There is a support script to run the repair commands as needed before the controller starts.
  • HAVEGEd will store entropy for encryption. Without sufficient entropy available rebooting the server may take 30 minutes or more.

None of this will cost a dime to try out. With a new account you will get credit for the first year. At least you can watch this 10 minute video to see the whole procedure including transferring your current controller:

(There are subtitles available under the YouTube gear icon)

1. Preliminaries

Take a backup of your current controller in Settings > Maintenance. This is always a good idea. You may also want first to improve your password. In Google Cloud your controller can be accessed by anyone who guesses your username and password. You don’t want that. Fail2Ban will lock the IP address for an hour after three failed attempts, but it won’t help if your password is guessable.

A proper SSL certificate requires a DNS name that points to your controller. You can register your own domain or use one of the free ones that dynamic DNS providers offer. If you want to use a dDNS service, you need to set it up first. There are links to some providers at the end. If you control your DNS yourself you won’t need dDNS.

Next you have to create an account on Google Cloud Platform. You will need a credit card for this, but the micro instance that Google offers for free is sufficient for a small UniFi Controller and the free 5GB on Cloud Storage is plenty for the backups. Google does not guarantee this offer will run forever, but it has run for years now and they will certainly let you know if this changes. The current price for a micro instance is $4.28 per month, if you need to run another for example.

First of all, use a good password or better yet, set up two factor authentication for your Google account. If someone breaks into your account the intruder could create a huge bill in just a few hours. You can also set a spending limit in Billing > Budgets & Alerts. Create a new budget for the whole billing account and set a budget. $50 or even $10 should be enough, if you ever want to do some testing on an extra controller for a few hours. The budget feature won’t protect you from an intruder breaking into your account, however. The first thing the intruder would do is to remove the limit.

If you set up budget alerts (50%, 90% and 100% by default) you need to nominate yourself to a billing administrator to receive the alert emails. This is done in IAM & Admin > IAM section. Locate your identity in the list and click in the Roles column. Check Billing > Project Billing Manager to add this role to your identity.

[UPDATE:] Some readers have reported that they needed to enable the storage API in APIs & Services > Dashboard > Enable APIs & Services > search for Google Cloud Storage JSON API > click Google Cloud Storage JSON API tile > click Enable button. My account had this enabled by default, but my account is older and perhaps the default has been changed. [Thank you Limbo and Chrysilis!]

2. Create a Storage Bucket (optional)

UniFi Controller will create backups automatically. The default is once a month, but you can change this in UniFi Controller > Settings > Auto Backup. I run backups once a week and keep the last 26 or a half year’s worth. These backups are stored on the computer the controller runs on. If this computer is lost for some reason, so are the backups. You should store them somewhere else and a Google Storage bucket is the perfect place. If you create a bucket the script will sync the backups to the bucket daily.

In GCP Console switch to Storage > Cloud Storage > Browser and create a new bucket. The bucket name has to be globally unique, so all the obvious ones are already in use. Use something like petri-office-unifi-backup. For the free offer select Region as the Location type and choose a Location in the U.S. I am in Europe and East Coast is marginally closer, so I use it.

3. Set Up the Virtual Network

This is something you will need to do only once. All your future UniFi controllers will use the same network settings.

Switch to Networking > VPC Network > Firewall. Create a new firewall rule and name it allow-unifi for example, the name isn’t important. In the Target tags field enter unifi. You can use any name for the tag, but you will need to remember it for the next step. In the Source IPv4 ranges type 0.0.0.0/0 to denote any address. In the Protocols and ports section select Specified protocols and ports and copy these into the TCP and UDP fields:

tcp: 8443,8080,8880,8843,6789,443,80
udp: 3478

Switch to  VPC Networks > External IP Addresses and click on Reserve Static Address. You need to give the address a name, choose the region where your virtual machine is going to run and click Reserve. I suggest you use the same region as the Storage bucket. You can’t attach the address yet, since you haven’t yet created the virtual machine. (Google will charge for the static addresses while they are not in use. The plan is to run the controller non-stop so this is not an issue. Just remember to release the address if you don’t need it any longer.) If you manage your DNS yourself, add an A record for this IP at this point.

4. Create the Virtual Machine

In Compute > Compute Engine > VM Instances create a new instance. You can name it anything you like. Choose between Regions us-west1, us-central1 or us-east1 for the free offer. You need to place the VM in a Zone that is in the same region as the static address. The Machine type should be E2 series e2-micro for the free instance.

Click on Change for the Boot disk.  The default operating system is Debian  GNU/Linux 10 (Buster), but UniFi Controller is not supported on Buster. Change the version to Debian GNU/Linux 9 (Stretch) [Kudos to James!]. The default size is 10GB which suffices, but it is tricky to expand afterwards. The free offer is 30GB but I prefer to set the boot disk to 15GB in case I need to run another VM. To change the size, click on Change and set the size. Also change the Boot disk type to Standard persistent disk (not Balanced) to stay on the free tier [Thanks Steve!]  and then you are ready to click Select to save.

If you created a bucket for the backups, you need to allow the virtual machine to write to Google Storage. The default permission is Read Only. In Identity and API access choose Set access for each API. In the list that appears change Storage to Read Write.

No need to select the HTTP and HTTPS options in the Firewall section. You created your own firewall rules already.

Click on Networking, disks, security, management, sole-tenancy to open advanced options. In the Networking section add unifi to the Network Tags field. This tag ties the firewall rules you created earlier to this virtual machine instance. This step is easy to forget, but missing it will prevent access to your controller! Leave the hostname empty. Click on Default under Network Interfaces and choose the static IP address you created earlier as the External IP. Set Network Service Tier to Premium [apparently Premium is on the Free Tier. Thanks, gman!].

In the Management section add the following Metadata key/value pairs. The first one is mandatory: the script that does the magic. All the others are optional and the order doesn’t matter. There is no way to do any error checking or report script failures, some feature will just not work if the metadata is erroneous. You can come back and change them if necessary. The changes will take effect after next reboot. See the Maintenance section at the end for how to reboot.

Key Value Purpose
startup-script-url gs://petri-unifi/startup.sh Required!
ddns-url http://your.dyn.dns/update.php?key=xxxxx Helps you access the Controller
timezone Europe/Helsinki Lets you adjust the reboot hour
dns-name your.server.dns Required for HTTPS certificate
bucket your-bucket-name-here Required for offline backups

Some users have reported problems with the gs: style URL for the script. You can try https://storage.googleapis.com/petri-unifi/startup.sh instead [Thank you Miczita]

Click Create. Go get coffee. Even if the virtual machine appears to be running, the script is not done yet. It used to run in minutes, but give it a half an hour these days.

5. Set Up the Controller

Connect to your new controller. On the first screen of the UniFi wizard click restore from a previous backup and upload the latest backup. Wait for the restore to complete. Log on to the new controller using your old username and password. In Settings > Controller add the DNS name or the IP address of the new controller to Controller Hostname/IP field and check Override inform host with controller hostname/IP. Confirm the change. Click Apply Settings.

Connect to your old controller. Do the same change in Settings > Controller and add the DNS name or the IP address to Controller Hostname/IP field and check Override inform host with controller hostname/IP. This setting is for the current devices that are already associated with the current controller. Now they will start to contact the new controller in Google Cloud.

Connect to your new controller again. Check the devices page to see whether the devices come online eventually. If they don’t, you may have to force reprovisions them on the old controller. In that case go to the Devices page on the old controller, select each device in turn and on the Config tab click Provision under Manage Device.

[UPDATE:] The controller doesn’t log failed login attempts by default any more. Logging is required for Fail2Ban protection. You need to change the logging level of the controller: Settings > Maintenance > Services > Log level and change Mgmt to More and click Apply. [Thank you Gabe!]

How to Set Up a New Network with a GCP Controller

If you are starting from scratch you won’t have any controller backup to transfer to the GCP based UniFi Controller. In that case you have three alternatives:

  1. Install a temporary controller on a local computer, set up the network and transfer the controller to GCP as outlined above.
  2. If you want to do a pure cloud based installation, you’ll need to tell the devices the address of the controller. If you have an UniFi Security Gateway, connect its WAN interface to the Internet and your laptop directly to the LAN interface. Connect to the USG with a browser (https://192.168.1.1) and configure the inform URL to http://your.server.dns:8080/inform. Once you have adopted the USG in the controller other UniFi devices will appear in the controller as you start to connect them.
  3. If you have some other router than an USG, you can either set your DNS server to resolve name unifi to the IP address of your cloud controller or set the DHCP option 43 in your DHCP server. There are examples of both in Ubiquiti’s L3 Adoption Guide. The last resort is to SSH to each device in turn and use command
    set-inform http://your.server.dns:8080/inform.

Maintenance and troubleshooting

If your devices won’t switch to the new controller the reason is usually either in the firewall or DNS. Is your VPC firewall rule correct and does the network tag of the virtual machine match the tag on the rule? Is your DNS name correct both in the metadata and in both controllers’ Override Inform Host fields and does the DNS name point to the correct IP address? In either case the devices will try to connect for several minutes before reconnecting to the old controller. Sometimes it takes hours for DNS to settle, depending on the time-to-live settings. In that case let it sit overnight and force provision the devices next morning.

Let’s Encrypt limits the number of certificates issued per domain to five per week (at the time). This usually isn’t an issue, but if you use the free domains offered by dDNS providers, the most popular domains will have used this week’s five certificates already. In this case the script will try to acquire a certificate once an hour until it succeeds. You will see certificate warnings at first but the address bar should eventually turn green. You can also anticipate this and choose a less popular domain (or register your own).

Don’t set up two controllers to back up to the same bucket. They will delete each other’s backups. You can either create a new bucket or create folders inside the bucket. In the metadata set bucket to your-bucket/foldername.

Google blocks sending email directly to SMTP port 25 from within GCP because they don’t want all spammers on board. If you want to receive email notifications you need to set up authenticated SMTP on port 587 or 465 in UniFi’s Settings > Controller > Mail Server. You can use your personal email account or even use Google’s email engine if you can prove you are not a spammer.

The Cloud Console will typically start displaying a recommendation to increase performance by upgrading to a bigger virtual machine (at cost), even though the CPU utilisation graph typically stays below 15%. Apparently the updates the devices send cause very short spikes, because occasionally the devices report that the controller is busy. This may cause some inaccuracies in the reports, I am not certain. If you can live with that, just dismiss the suggestion. In my experience the micro instance can serve at least the same 30 devices a Cloud Key can. You can also use this script on any virtual machine type you like. When you select the machine type you see the cost per month. You get a better price if you can commit to use the resource for 1 or 3 years.

If the controller appears to be malfunctioning the first remedy is to reboot it. A reboot is also the way to re-run the script if you made changes to the metadata. The safe way to reboot is to Stop and then Start the virtual machine. This won’t change the IP address since you have reserved a static address. Don’t use the Reset button. A Reset will immediately restart the virtual machine, which may damage the UniFi database and/or the Linux filesystem (if you are really unlucky).

The underlying Linux system and UniFi Controller will be automatically updated to the latest stable releases. If an update requires a server restart or a reboot it will occur after 04AM UTC. Set the timezone metadata to adjust the reboot to 04AM local time. A reboot will make a captive wireless guest portal inaccessible for a couple of minutes so you don’t want it to happen during peak hours.

Automatically updating the system is a risk. For example there could be an update to Java or MongoDB that is incompatible with the installed UniFi controller. There is no way you could prevent the installation of the update. However, I decided that NOT updating the system is even bigger risk. I don’t trust the intended audience (you) to log on regularly over SSH and run apt-get. In any case, if the risk realizes I trust that it will be transient. The incompatibility will affect a lot of other users as well and there will be another update to solve the issue. This other update will be also automatically installed. If you are familiar with Linux you can also log on the VM and edit /etc/apt/apt.conf.d/51unattended-upgrades-unifi to your requirements.

The automatic update won’t do major Linux version upgrades. Every couple of years it is wise to upgrade the underlying operating system. The Cloud Way is to create a new UniFi controller:

  1. Take a backup of the old controller.
  2. Create a new bucket (or folder).
  3. Remove the static IP address from the old virtual machine (VPC Network > External IP Addresses and Change to None)
  4. Create a new virtual machine as outlined above and restore the backup
  5. Delete the old virtual machine
  6. Done! Since the IP address is the same there is no need to update any settings and the switchover is immediate.

If your controller starts to act up and rebooting doesn’t help, just create a new one. Just like upgrading the underlying Linux, create a new virtual machine. If you cannot connect to the old controller to retrieve the backup, use the latest auto backup in the storage bucket. That’s what they are for. Don’t fall in love with your virtual machine, you should treat them like pencils: very useful but replaceable.

If you are concerned about the security of the script you better read it through first (the link is at the end). Be aware that the script is in my bucket so I can update it as necessary. I could also make malicious changes that would take effect on your server after next boot (after a system update for instance). If you are cautious you should remove the whole startup-script-url row from the metadata as soon as you are happy with the other metadata settings. The script doesn’t do anything once the system is configured. If you ever need to reconfigure something you can just add the startup-script-url back, Stop and Start the VM to run the script and then remove the row again.

Google currently offers for free one micro instance with 30GB of disk space and 5GB of bucket storage in the U.S. regions. Both come with monthly 1GB of free egress (i.e. outbound) data transfer with China and Australia excluded. Egress data is used when you download backups to your computer and when your devices download upgrades from the controller. The static IP address is free as long as it is attached to a running virtual machine. You will be charged if you exceed these limits, but typically the charges are minuscule. For example when you upgrade your Linux you will run another micro instance for a while. The budget feature should protect you from nasty surprises.

Links

Some dynamic DNS providers and how their single line update URLs look like:

  • Afraid.org
    http://freedns.afraid.org/dynamic/update.php?xxxdynamicTokenxxx
  • DNSPark
    https://username:passsword@control.dnspark.com/api/dynamic/update.php?hostname=home.example.com
  • DuckDNS
    https://www.duckdns.org/update?domains={YOURVALUE}&token={YOURVALUE}
  • DynDNS
    https://{user}:{updater client key}@members.dyndns.org/v3/update?hostname={hostname}
  • Dynu [thank you Moses!]
    https://api.dynu.com/nic/update?hostname=HOSTNAME&alias=ALIAS&username=USERNAME&password=PASSWORD
  • EasyDNS
    https://username:dynamictoken@api.cp.easydns.com/dyn/generic.php?hostname=example.com
  • NameCheap
    https://dynamicdns.park-your-domain.com/update?host=[host]&domain=[domain_name]&password=[ddns_password]
  • Sitelutions
    https://dnsup.sitelutions.com/dnsup?id=990331&user=myemail@mydomain.com&pass=SecretPass
  • ZoneEdit
    https://<username>:<password>@dynamic.zoneedit.com/auth/dynamic.html?host=<hostname>

1,176 thoughts on “Set up UniFi Controller on Google Cloud Platform”

  1. The biggest update so far didn’t go cleanly. Upgrading UniFi controller 5.6 to 5.7.20 killed the MongoDB server disabling the controller. Starting and stopping the virtual machine recovers it. I can’t control the update package contents and instructions. I can only hope for the best. Unattended upgrading is always a gamble.

    Reply

    3. Hi Petri – thank you for this wonderful post, the guide, the script and most of all the follow-up support. I have everything loaded up and have my personal domain unifi.domain.com with it’s A record pointing to the static/reserved IP address of the VM. Also, I have the dns-name setup with this same value. I am able to reach the Unifi login screen using my domain – however, I’m unable to login. I am using the same username and password as I use locally but no luck logging in.

      During setup, I loaded the backup settings as instructed above. I also don’t see letsencrypt cert for my domain “unifi.domain.com”. I have tried to STOP and START the VM 3 times.

      Any ideas?

      Reply

      1. Your first problem won’t go away with starting and stopping the VM. The user credentials are inside the controller database and somehow they are wrong. You write you can still log in to your old controller with the same credentials. In that case I would suggest to delete the VM, create a new one, download a NEW backup and restore it.

        My script will acquire a Let’s Encrypt certificate if:
        – Your metadata includes a FQDN dns-name
        – The dns-name resolves globally to the public IP of the VM
        – Let’s Encrypt servers from California can connect to the IP via http and https
        Apparently one of the conditions is not met. You need to figure out which one and fix it.

        Reply

        1. The username/password issue was resolved – I had fat fingered something. ugh.

          I have the FQDN unifi.mydomain.com setup in the metadata and properly resolving to https://unifi.mydomain.com:8443/ and I’m able to use all the functions. The IP is properly setup mapping to the VM.

          Do I need to do anything to trigger the LetsEncrypt servers?

          Reply

          1. I’m sorry, this had slipped my attention. You need to stop/start your VM so it sees your new metadata.

            Reply

      1. Hi Petri,
        Thanks for all your work.
        I setup my controller in Google Cloud following your instructions, so far the upgrade went good.
        I am stock in 5.9.29.0 and version 5.10.17 is available.
        What do you recommend to Do?

        Reply

        1. Stop and start the VM to install the latest version of the script. The upgrade should install next night.

          Reply

          1. Any idea off the top of your head on were I would have messed up, I have been running this for 6 months+ and have been getting charged like 2-6¢ a month the last few months… Its not a big deal im just checking.

            Reply

            1. You should be able to dig into the billing details to find what Google is charging for. Google’s billing is complicated so the billing breakdown is also deep. Most users find it contains too much information, so the answer is there.
              My guess is that the charges are for data transfers. If you are running the VM in a different zone than the backup bucket then you’d pay for transferring the backup to the bucket, for example. Or if you keep downloading the backups to your local computer then you’d pay for the outbound traffic. Just browsing the controller will stay below the charging threshold. For example I haven’t been charged for transfers yet.

              Reply

    5. Hi Petri, thank you for an excellent post. Created my first Cloud controller and all seems well.
      I didn’t point/migrated anything to the new cloud controller yet.
      I have only one question, I had a guest portal (voucher based) on my on-premise controller and had my logo appear on vouchers when printing for guests, how and where do I need to place the logo for it to appear again when printing vouchers?

      Reply

      1. I honestly don’t know, never tried customizing the vouchers. If you have done it for your current setup it should work the same way for the GCP based controller. It is just a Debian system so the UniFi Base directory is /usr/lib/unifi. You can start a SSH session from the GCP console to get access.

        Reply

  2. Hi Petri, thank you very much for all the work you shared with us.
    I follow your procedure and the video but controller is not working.
    How can I check if it’s been downloading and rouning?
    Can I force via SSH script execution so I can see if any error appears?

    Thank you again.

    Reply

    1. You can SSH into the virtual machine from the Google Cloud Console. Click the SSH button and a pop-up window will appear (unless your pop-up blocker gets into way). No password is required, you are already authenticated in the console. The SSH session is useful if you want to see the logs in your VM.

      Another option is to click on the name of the virtual machine in the VM Instances list. On the VM Instance Details page you’ll see a small link for “Serial Port 1 (Console)”. That will show you all the startup messages. The screen doesn’t update automatically if you want to watch it in real time. You need to click the Refresh for an update. The lines containing “startup-script:” are produced from my script. Do you see any errors?

      Reply

      1. Thanks for your Reply Prtri:
        In “Serial Port 1 (Console)” there are 6 lines with startup-script:
        Apr 15 17:45:03 unifi-controller startup-script: INFO Starting startup scripts.
        Apr 15 17:45:03 unifi-controller startup-script: INFO Found startup-script-url in metadata.
        Apr 15 17:45:03 unifi-controller startup-script: INFO Downloading url from gs://petri-unifi/startup.sh to /startup-Vm93gn/tmp792gVc using gsutil.
        Apr 15 17:45:08 unifi-controller startup-script: WARNING Could not download gs://petri-unifi/startup.sh using gsutil. Command ‘[‘gsutil’, ‘cp’, u’gs://petri-unifi/startup.sh’, ‘/startup-Vm93gn/tmp792gVc’]’ returned non-zero exit status 1.
        Apr 15 17:45:08 unifi-controller startup-script: INFO No startup scripts found in metadata.
        Apr 15 17:45:08 unifi-controller startup-script: INFO Finished running startup scripts.

        I don’t explain myselft correctly in my last comment, I wanted to know how to execute your script manually 😉
        I found how to download it and executed with sudo, it seem my VM is not accesible from itself:
        Script logrotate set up
        MongoDB logrotate set up
        IPv6 disabled
        ERROR: Address 35.190.181.102 has not changed.
        Dynamic DNS accessed
        592 megabytes of memory detected
        Swap file created
        Localtime set to Europe/Madrid
        Unifi added to APT sources
        System upgraded
        Haveged installed
        CertBot installed
        Extracting templates from packages: 100%
        Unifi installed
        Synchronizing state of mongodb.service with SysV service script with /lib/systemd/systemd-sysv-install.
        Executing: /lib/systemd/systemd-sysv-install disable mongodb
        Lighttpd installed
        Fail2Ban installed
        Unattended upgrades set up
        Created symlink /etc/systemd/system/multi-user.target.wants/unifidb-repair.service → /etc/systemd/system/unifidb-re
        pair.service.
        Unifi DB autorepair set up
        Backups to bdlsolucinoes-backup set up
        Saving debug log to /var/log/letsencrypt/letsencrypt.log
        Plugins selected: Authenticator standalone, Installer None
        Registering without email!
        Obtaining a new certificate
        Performing the following challenges:
        http-01 challenge for mysub.domain.com
        Waiting for verification…
        Cleaning up challenges
        Failed authorization procedure. mysub.domain.com (http-01): urn:acme:error:connection :: The server could n
        ot connect to the client to verify the domain :: Fetching http://mysub.domain.com/.well-known/acme-challeng
        e/rKudGtFidPxCAHNxR__YzQ3qb8PohwWs38yaS6voXcg: Timeout

        I pinged “mysub.domain.com” from VM and my PC, always reply public GCP IP configured as you explained.

        From my PC http://mysub.domain.com or /.well-known/acme-challeng shows ERR_CONNECTION_TIMED_OUT

        Maybe webserver is not working?
        Thanks.

        Reply

        1. The first part of your post indicates that the VM couldn’t load the startup.sh script at all. That’s odd. I don’t have any other GCP accounts to try on right now. I’ll look into this.

          The second part where you run the script manually indicates that Let’s Encrypt service cannot connect to your domain from their data center. Does the dns-name metadata resolve to 35.190.181.102 (check for typos)? Is port 80 included in the firewall rule and the same tag applied to the VM? Testing tools are dig, nslookup, getent or host for the former and fetch or curl for the latter question.

          Reply

          1. Hi Petri: your script works great! It’s awasome.

            Connection to my instance was failing becouse I forgot /0 in the 0.0.0.0 on firewall source… 😀

            About startup-script: I found this in serial log:
            ——
            Apr 18 15:25:43 unifi-controller google_metadata_script_runner[683]: AccessDeniedException: 403 Access Not Configured. Please go to the Google Cloud Platform Console (https://cloud.google.com/console#/project) for your project, select APIs and Auth and enable the Google Cloud Storage JSON API.
            ——

            I finally fixed it.

            I want to ask you something: is it as difficult as I read to set a port mapping in GCP?
            My current ISP blocks 8080 so I’m running CloudKey Unifi at TCP/8082

            To easily migrate all my deviced, as you wrote, I want to change controller hostname. So I need to redirect GCP-public-IP:8082 -> VM:8080

            I read lots of commands to enable this port mapping. Do you know a simple way?

            Thank you again! An incredible job.

            Reply

  3. You sir, are the man!

    Awesome work, seems you’ve thought of everything.

    This must be the only GCP+Unifi tutorial to exist, and it’s fairly easy to follow, it was an absolute treasure of a find, especially given that GCP just happens to be the most difficult to figure out..

    This is killer, it works perfectly, many thanks for this, much appreciated!

    Reply

    1. Thanks! If you did set up your system recently, it proves that the script download problem in the previous comment by Limbo is either new or random occurence.

      Reply

      1. Hi Petri, thanks for putting this together! I had the same error as Limbo “AccessDeniedException: 403 Access Not Configured. Please go to the Google Cloud Platform Console (https://cloud.google.com/console#/project) for your project, select APIs and Auth and enable the Google Cloud Storage JSON API.” Once I enabled the Google Cloud Storage JSON API and restarted the VM all was well.

        Reply

        1. Do you mean APIs and Services > Dashboard ? I have never changed anything there and all services are enabled (there is a button to disable each). Could it be that my account is older and the defaults have changed? The only API that has any request history is the Compute Engine API. None for the Storage JSON API in the past 30 days. I have no clue.

          Reply

          1. APIs & Services > Dashboard > Enable APIs & Services > Search for “Google Cloud Storage JSON API” > Click “Google Cloud Storage JSON API” tile > Click Enable button

            Without Google Cloud Storage JSON API enabled the VM could not access the startup script. Note that like for you on the APIs & Services Dashboard I see no requests against Google Cloud Storage JSON API.

            Reply

            1. Thank you for reporting this. I added an update to the post in the Preliminaries section.

              Reply

  4. Hi! Petri:
    I followed your article, I found that certbot doesn’t work for my cloud controller on GCP.

    Message of Serial Port 1 (Console) contain “startup-script:” looked like error info to me:
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 2.)
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype
    Apr 21 20:33:13 unifi-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:
    Apr 22 04:38:04 unifi-controller startup-script: INFO startup-script-url: /usr/local/sbin/certbotrun.sh: 7: [: =: argument expected

    Then I managed to use acme.sh which is an alternative to certbot to apply let’s encrypt SSL certificate:
    https://www.naschenweng.info/2017/01/06/securing-ubiquiti-unifi-cloud-key-encrypt-automatic-dns-01-challenge/

    The certificate then installed successfully.

    Is there anything I need to do with the error info besides certbot?
    Thanks.

    Reply

    1. Each and every apt-get install will create all those debconf and dpkg-preconfigure errors to the log. I haven’t found a way to suppress them without losing valuable debug info. Apt-get is just trying to communicate with the user and doesn’t find a controlling terminal.
      The last line is significant, though, about the certbotrun.sh. It means that for some reason the script couldn’t retrieve the external IP address of the VM. Does your VM have one? I couldn’t reproduce this. Anyways, I have updated the script (currently at 1.0.3) so that it will output a more useful error message instead.
      I haven’t looked into acme.sh. You may have to import the certificate manually to the Java keystore after every renewal. Or you can edit /etc/letsencrypt/renewal-hooks/deploy/unifi to work with acme.sh.

      Reply

      1. Hi~ Petri,
        I’ve reserved a static external IP address for my cloud Controller VM on GCP as your tutorial steps.
        I am fine with acme.sh, just created a crontab to perform the auto renewal.
        Thanks for your reply!

        Reply

  5. How do I use Let’s Encrypt with this ?

    How do I see what is in the gs script ?

    Can this setup be done for UNMS ?

    Reply

    1. 1) Create a DNS name and point it to your VM instance. Add the DNS name to the VM metadata with key dns-name as instructed. Everything else is automated.
      2) At the end of the article in section Links there is a link “The startup script on Google Storage”
      3) My script won’t do it, but you could write a similar setup for UNMS. You can also read through the script and do the steps manually to apply them to an UNMS instance.

      Reply

  6. Hi Petri!
    First of all, great job and thank you for share your work!

    I followed your video, step by step, but, when i connect to VM, the browser (Opera and Chrome) says “Not Secure”, there is any way to solve this issue?

    Reply

    1. If you click on the lock icon and check the certificate details, is it a UBNT self-signed certificate or a Let’s Encrypt one? To me it sounds like your VM didn’t receive a LE certificate.

      The most probable cause is that the domain-name metadata didn’t resolve to the external IP of the VM. If you reboot the VM by stopping and starting and then click the VM name to see the details there is a link to the Serial port 1 (console) where you can see all the output from the boot process. Only lines containing “startup-script” relate to my script. If you see a line “No action because aaaaa.bbb doesn’t resolve to vv.xx.yy.zz.” then that is the cause.

      It may be easier to just delete the old VM and create a new one. It is the Cloud Way. It could be that a DNS glitch made the LE check fail. If you delete the old one first then it won’t cost you but 10 minutes of work.

      Reply

  7. Thanks for this tutorial, this is great!

    I’d like to modify the script to install unifi-testing rather than stable. It looks like the only area I would need to modify your script is line 88. Can you confirm? If so, I’ll modify your script to include that change and then host it in my own bucket.

    Reply

    1. Yes, that’s the only place. You can also just copy and paste the whole script to the startup-script textbox in GCP Console. No need to use a bucket, if you only need one VM.

      Reply

      1. Thanks! Last question. I have a custom config.gateway.json file for use with CloudFlare DDNS. How would you suggest I add this in a scripted way?

        Reply

        1. Add a line like (if the file is on a web server):
          curl -Lfs -o /var/lib/unifi/config.gateway.json http://www.some.dns/path/setup.json
          or (if the file is in a bucket):
          gsutil cp gs://bucket/setup.json /var/lib/unifi/config.gateway.json
          Disclaimer: I haven’t tested either one. I just typed them off the top of my head. E.g you may need to adjust the permissions as an extra step.

          Reply

          1. I don’t do a lot of shell scripting, but I added a section like this to the end of your file:

            ###########################################################
            #
            # Add Cloudflare DDNS settings to default config.gateway.json
            #
            if [ ! -f /var/lib/unifi/sites/default/config.gateway.json]; then
            cat > /var/lib/unifi/sites/default/config.gateway.json <<_EOF
            {
            "service": {
            "dns": {
            "dynamic": {
            "interface": {
            "eth0": {
            "service": {
            "custom-cloudflare": {
            "host-name": [
            "sub.domain.com"
            ],
            "login": "email@gmail.com",
            "options": [
            "zone=domain.com"
            ],
            "password": "API_KEY_GOES_HERE",
            "protocol": "cloudflare",
            "server": "www.cloudflare.com"
            }
            }
            }
            }
            }
            }
            }
            }
            _EOF
            echo "Cloudflare DDNS added to default config.gateway.json";
            fi

            Reply

            1. Ahem.. You are right. It should go under the site directory. Your script looks fine to me. Testing will tell the truth.

              Reply

            2. Oh, one more thing. Don’t put it at the end! If there is no dns-name metadata my script will exit on line 342. Put your addition ahead of the Let’s Encrypt stuff on line 336 for example (or any line before that).

              Reply

            3. I too have cloudflare…and I doubt my Ip will be changing … I’d like to get this setup. Having said that, I am a little unclear on how to add this to my VM….can anyone help?

              Reply

              1. So, I think I figured out where to add JC Connell’s Cloudflare script. I created it this way, because I like Petri’s script and it works perfectly, the only thing I wanted to change was the Cloudflare update. So, if anyone can foresee an issues…please let me know. My concern is at this point, is that GCP will pull Petri’s script and execute it, then my Cloudflare DNS update will happen. So, I need ot make sure the timing is correct.

                In my VM instance, I created a custom meta data called: startup-script. and I added these content to the text box:

                #! /bin/sh
                ###########################################################
                #
                # Add Cloudflare DDNS settings to default config.gateway.json
                #
                if [ ! -f /var/lib/unifi/sites/default/config.gateway.json]; then
                cat > /var/lib/unifi/sites/default/config.gateway.json <<_EOF
                {
                "service": {
                "dns": {
                "dynamic": {
                "interface": {
                "eth0": {
                "service": {
                "custom-cloudflare": {
                "host-name": [
                "sub.domain.com"
                ],
                "login": "email@email.com",
                "options": [
                "zone=domain.com"
                ],
                "password": "GLOBAL_API_KEY_GOES_HERE",
                "protocol": "cloudflare",
                "server": "www.cloudflare.com"
                }
                }
                }
                }
                }
                }
                }
                }
                _EOF
                echo "Cloudflare DDNS added to default config.gateway.json";
                fi

                Reply

                1. I ran the startup script test using the command:
                  sudo google_metadata_script_runner –script-type startup –debug
                  And the script was throwing an error on line 6 (if [ ! -f /var/lib/unifi/sites/default/config.gateway.json]; then)

                  So, I just removed that line and the script was happy and it created the /var/lib/unifi/sites/default/config.gateway.json file with the correct contents.

                  However, its seems like my setup is not executing the /var/lib/unifi/sites/default/config.gateway.json file…can’t find any errors as to why… I can check cloudflare and the IP has not been updated.

                  Any ideas?

                  Reply

                  1. IIRC you need to Force Provision the USG for it to pick up the config.gateway.json.

                    Reply

                    1. Thanks Petri!
                      I found this article, and was able to text the JSN file that was created, but with the website mentioned in the article and with the CLI. Both tested perfect.

                      However, I don’t have a USG…just a couple of APs right now. So, I am unable to follow the rest of the article about Force Provisioning the USG. 🙁

                    2. Then you are out of luck, I’m afraid. The config.GATEWAY.json refers to the usG. It is additional configuration for the USG not available in the GUI.

  8. Wow, great call. Thanks for all your help on this.

    I’m slowly moving my homelab to the cloud. This has been a fun and useful learning experience.

    Reply

  9. Great job Petri! It really helpme, I create 2 vcpu + 7.5 gb Memory +30gb storage, i need to manage about 150 unifi AP, there is no problem with “The script will create a swap file to accommodate the controller.”?, Is the script limiting the memory capacity?

    Reply

    1. No, the script won’t. It will create a swap file if there is less than 2GB memory. You have more so there won’t be any swapping. UniFi defaults however do limit the controller memory to 1GB. If you log on via the SSH button in GCP Console and edit system.properties file you can increase the initial (xms) and maximum (xmx) memory size of the controller. See Controller Tuning

      Reply

      1. ok, thanks, I will try that, under your experience, how many ap can you manage with 1gb memory(or users)?

        Reply

        1. I haven’t done any real testing. Dozens anyway. Ubiquiti quotes 30 devices for 1GB Cloud Key, but I don’t know whether it is memory or processor bound. I haven’t seen the load go up noticeably. Using the GUI is sluggish, though, and periodic MongoDB chores do eat up the processor causing some device updates to go unnoticed. If you can live with those you should be fine.

          Reply

      2. Hello Petri, I add about 65 ap on the google cloud controller, but im experimenting some issues on some ap, some users report wlan unestable, appears and dissapear or some ap’s become heartbeat missed and then disconected, after that conecten again I improve controller to 3 gb memory on system.properties,, do you have any idea? could you help me? (i will pay you for your service)

        Reply

        1. The WLAN works even without a connection to the controller. The missed heartbeats just mean a gap in history stats at most. Only if you run a guest network with hotspot you need continuous controller connection. There is some other problem with your network.

          I am abroad this week so I can’t help you right now. You should ask on the Ubiquiti wireless forum. There you get prompt replies for free.

          Of course you could use a slightly larger VM for 65 APs.

          Reply

            1. That is plenty for a hundreds of APs. The controller performance can’t be the issue here. Do you monitor the uplink from your network to the Internet? If the uplink is congested that could cause the symptoms you are seeing. I can’t see how I could help you further. You should post of on the UniFi forums or contact Ubiquiti support.

              Reply

  10. Thanks for this post Petri, very helpful.

    One question. I assume the ddns-url metadata field is unnecessary if you are manually pointing your hostname to the static IP through your DNS provider?

    Reply

  11. Petri, this was extremely helpful and easy to setup with your instructions. Thanks so much for your work.

    I’m up and running but I just have one question. Did you do something to redirect requests to your controller via hostname automatically to 8443 for management or do you have to enter https://yourdomain.com:8443 each time you want to access your cloud controller?

    Reply

    1. The script installs Lighttpd which will direct everything from port 80 to 8443. To use it enter the plain yourdomain.com in your browser, no https at the start or 8443 at the end. Catching all the possible combinations would have complicated the script far too much.

      Reply

  12. Kiitos paljon!
    Works like a charm, now I don’t have to worry about a power outage ruining my Raspberry PI controller.

    Reply

  13. What’s the controller upgrade process like once a controller has been up and running via this script?

    Reply

    1. Unattended upgrades will run every night and keep both the Linux and the UniFi controller up to date. You won’t have to do anything. It won’t do major Linux upgrades however. Debian “Buster” is expected next summer but “Stretch” will be supported for some time to come.

      Reply

      1. So the UniFi controller will auto update to the latest stable release that’s available each night? Is there anyway to stop this behavior so that the controller is only upgraded on demand?

        Reply

        1. Yes. This setup was intended for users without any Linux or cloud skills. I decided that it is safer to upgrade everything automatically than to leave unmanaged VMs running on the Internet. For the target audience the Linux command line is a real barrier.

          Linux savvy users can create and tailor their setups as they see fit. You can use my script as a starting point. If you want to revert to the default behavior with only Linux security upgrades being automatically installed you should just remove /etc/apt/apt.conf.d/51unattended-upgrades-unifi. If you want to disable all automation apply command systemctl disable unattended-upgrades.

          Reply

          1. Thank you, that was very helpful. I don’t mind messing with the Linux CLI at all. Your script just makes the initial setup quite hands off which is nice.

            Reply

          2. Petri,
            can’t thank you enough for this easy setup. Based on your comment here, if I delete /etc/apt/apt.conf.d/51unattended-upgrades-unifi via SSH, I could update the Unifi controller to a version of my choosing and still enjoy your automated linux updates and other fixes applied during the nightly scheduled call?

            Thanks again for this work,

            Reply

  14. What’s the best way to troubleshoot the autobackups not being sent to the Google storage bucket? It was working when the controller was first setup as I have one file shown in the storage bucket under the autobackup/ directory but none of the subsequent auto backups that are shown in the controller have been copied to the storage bucket.

    Reply

    1. What does
      sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://your-bucket-name-here
      produce?

      If that works check /etc/systemd/system/unifi-backup.service to see if the bucket name is correct.

      Reply

      1. I get the following output:

        CommandException: arg (/var/lib/unifi/backup) does not name a directory, bucket, or bucket subdir.

        Reply

        1. For me, the solution to the problem was to prepend the daily executed command with an absolute path to `sudo`.

          So this
          `ExecStart=/usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://NAME-OF-THE-BUCKET`
          becomes this
          `ExecStart=/usr/bin/sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://NAME-OF-THE-BUCKET`

          Reply

          1. Puzzling, since systemd runs the services as root. There shouldn’t be any difference in using sudo. There shouldn’t even be any other user accounts on the VM except for root. Makes no sense to me.
            If this were more widespread there would be many more complaints, so I have to assume there is something fishy in your setup.

            Reply

            1. So this is an issue after all. I just haven’t been able to reproduce it. I’ll add sudo to the script, it won’t do any harm to anyone. It is just me trying to keep privileges as low as possible, an old (but good) habit. Thank you for reporting!

              Reply

      2. I get the following when I run that command with sudo:

        Building synchronization state…
        Starting synchronization…
        Removing gs://ovretail-unifi-backup/5.7.23.unf
        AccessDeniedException: 403 Insufficient OAuth2 scope to perform this operation.
        Acceptable scopes: https://www.googleapis.com/auth/cloud-platform

        Reply

        1. My bad, I didn’t include the sudo. I’ll edit the answer above in case someone else tries it.

          Alright, the rsync works but your VM isn’t authorized. This is progress. Did you allow Read Write access to Storage for the VM?

          Another, later discovery was that some users had to enable the storage API. I added this update to the post:

          UPDATE: Some readers have reported that they needed to enable the storage API in APIs & Services > Dashboard > Enable APIs & Services > search for Google Cloud Storage JSON API > click Google Cloud Storage JSON API tile > click Enable button.

          Reply

          1. I confirmed the API was already enabled. In Cloud API access scopes I had Storage set to Read only. I set it to Read Write and started the VM backup. I still get the same output as I showed above.

            AccessDeniedException: 403 Insufficient OAuth2 scope to perform this operation.
            Acceptable scopes: https://www.googleapis.com/auth/cloud-platform

            Reply

            1. Good, you found the cause. You have corrected it, but nothing in the Cloud World happens instantly. Everything is queued and cached. I suggest you wait til tomorrow.

              If it doesn’t work by tomorrow you should spin up another VM with the proper permissions. Don’t work too hard trying to resurrect broken VMs. Creating a fresh one is the Cloud Way. If you want to learn something new you can detach the current virtual disk and attach it to the new VM so you won’t have to install anything. Although running the script is probably easier and faster.

              Reply

      3. What is the best way to check the bucket name through the browser of the VM instance? Thanks

        Reply

        1. In Compute Engine console click on the VM name and scroll down to Custom metadata.

          Reply

          1. Thank you for the fast reply. My bucket name looks right there. If I run the script manually it syncs with the bucket but otherwise it doesn’t appear to be. I know there’s a config also somewhere in the unifi files. I can’t remember how or where to check this.

            Reply

            1. If you SSH into the VM, then you can look at /etc/systemd/system/unifi-backup.service and /etc/systemd/system/unifi-backup.timer

              Reply

              1. I couldn’t find any issues with the bucket name anywhere. So I created a whole new vm and bucket and carefully followed the directions making sure access scopes was set to read / write, etc. However I’m still not getting a bucket sync. Any help would be appreciated.

                Reply

                1. Can you see the backups in /var/lib/unifi/backup ?
                  What happens if you issue command sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://your-bucket-name-here ? Any error message?
                  IIRC someone once had issues with the bucket and it was solved by using the https url for the bucket in /etc/systemd/system/unifi-backup.service.

                  Reply

                  1. No errors. And the command goes through with no hitches. Verified with the bucket and it synced up. Kinda strange because seems like all permissions are in place it’s just not actually doing this automatically.

                    I’ll try adding the whole url, and will see what happens!

                    Reply

                    1. Still no luck tried the https url and it looks like that won’t even run. Even tried someone else’s suggestion to make a direct path to sudo. In the mean time I’ve just made it my habit to ssh into the VM and manually run the command to sync it. It’s odd.

                    2. You are systematically doing something differently from all of us others, but I can’t tell what it is.
                      You don’t need to sync it by hand. You can download the backup through the web user interface and it is the same thing. Unless you are worried about losing log data, it suffices if you download a backup after configuration changes.

                    3. 100% percent sure it’s just something I’m doing or done. No it’s nothing that specific I just sync it because I am a back-up junkie just having access to the latest backups always makes me feel better. So I think I’ll just stick to making a habit of downloading one manually after a big change once I’m all finished with everything.

      4. Hi Petri,

        I also have troubles with the offline backups to the GCP bucket.
        I doublechecked everything: bucket name, meta data, API storage rights set to RW.
        The only way to get the backups to the bucket is by issuing the “gsutil rsync” command you provided above.
        PS: I’ve setup the backup frequency to 1 hour to be able to test.

        This is the ouput I get from “/etc/systemd/system/unifi-backup.service”

        cat /etc/systemd/system/unifi-backup.service
        [Unit]
        Description=Daily backup to eerdekens-unifi-backup service
        After=network-online.target
        Wants=network-online.target
        [Service]
        Type=oneshot

        Any help is greatly appreciated!
        Steven

        Reply

        1. Replying to myself: it’s fixed now, my bad. I entered “Brussels” instead of “Europe/Brussels” as timezone in the metadata field.
          I assume this prevented the daily rsync script from running.

          Reply

          1. Hmm.. The time zone shouldn’t have anything to do with the rsync task. I’ll look into this whenever I have some cycles to spare. I haven’t actually tried using invalid timezone values. I have crowdsourced trying out all imaginable errors to my helpful assistants around the internet 😁

            Reply

  15. I’m curious if there are some special settings for the Guest Portal access. I noticed today that guests could connect to the guest wifi but they were not automatically redirected to the guest portal. I’ve added the hostname of the controller to the pre-authorization list. Any ideas why they might not be automatically redirected?

    Reply

    1. None that I know of. Over here in Finland Guest Portals are rarely used, though. The guest portal redirection is layer 3 anyways, so running the controller elsewhere shouldn’t make a difference.

      Check first that your GCP VPC Firewall allows traffic to the guest portal ports 8880 and 8843. You can check it with the browser: http://your.controller.dns:8880 and https://your.controller.dns:8843 both should produce some result.

      What are your settings in UniFi Controller Settings > Guest Policies ? You could also ask this in the UniFi Wireless forum, where there are many more users who could help you.

      Reply

  16. Thank you! For some reason, i had small problem with java (or maybe with apt). Unifi daemon did not start because java was not found. When entering java –version, system reported openjdk java fine. But for some reason this was not enough for Unifi.

    I had to ssh in and run sudo apt-get install default-jre -y

    Apt wanted to install several packages (like, +30). After everything was installed and rebooted, i was able to log in to Unifi

    Reply

    1. Was this just recently? With UniFi 5.8.24? I haven’t rolled a new VM with it yet. My script doesn’t install Java by itself. It should get installed as a prerequisite for UniFi Controller. Apparently someone has goofed upstream. Thanks for reporting this, though.

      Reply

    2. I confirmed this. I will try to get a response from Ubiquiti whether they will fix their package or should I write a workaround. I added an update at the top of the article about the situation.

      Reply

      1. Wow, quite the response time! Yes, this was this morning. Thanks for all effort you put in, will be big help to everyone who wants install Unifi to Google VM

        Reply

        1. I decided to solve this by installing JRE8 manually before UniFi Controller. The script is now updated (and some old scruff was polished as well).

          Reply

  17. I followed these steps but the webpage never seems to load, I just get “This page isn’t working”. I just ran the script this morning and not sure if I’m doing something wrong.. how do I troubleshoot the installation?

    Reply

    1. This morning there was a red warning at the top of the article about a problem. It was reported yesterday by GJ and I wrote a workaround today. It has been up for an hour now. Your easiest solution would be to delete the VM you created and spin up a new one. You should still have the backup on hand so it won’t take you but 10 minutes.

      Reply

    2. I do see an error in script..
      unifi-debian unifidb-repair.sh[631]: bash: /var/log/unifi/gcp-unifi.log: Permission Denied

      Should I worry about this? The above seems to be the only error I’m getting.

      Reply

      1. But no web page? That is something to worry about.
        The permission denied you are seeing is just about logging, nothing to worry about. It is odd though, since the systemd unit is run as root, so something is very wrong there. If the installation is hosed then I still recommend setting up a new one.
        If you still get this with the new VM please add a comment.

        Reply

  18. Any idea why I am unable to get my controller to enable cloud access? When i attempt to enable the following error is received “There was an error saving the cloud access changes. No cloud license is assigned.”

    I can access the controller directly without any issues so i guess it’s not that big of a deal, however this was never an issue on AWS, RasPi or Windows hosted controllers.

    Reply

    1. I never figured out. I had it happen on Cloud Keys and other local servers, too. Occasianally disabling and enabling it a couple of times did the trick – or was the trick waiting for a few days in frustration, I never knew. Trying different accounts didn’t help IIRC. After I switched over to remote controllers I no longer bothered with Cloud Access. Now I can manage the network from anywhere with a direct https connection instead of screen sharing magic. Downloading backups works, less broken or stuck connections, less trouble overall. I haven’t missed Cloud Access at all.

      Cloud Access is very useful for a local server, I admit. It’s a luxury to login from home instead of driving to the office to reboot a misbehaving AP. If you want to troubleshoot it you should ask on UniFi forums

      Reply

    1. For some reason the Ubiquity downloads were unavailable. Testing those links now everything appears to be OK so perhaps it was some transient error? Have you tried again? And I mean deleting that VM and creating a new one today.
      No, you weren’t doing anything wrong and there is nothing you or I could do about Ubiquity downloads repository or GCP network connecting to it.

      Reply

    2. On a second thought: the NTPd error just before could indicate there is something wrong with the network. The VM did load the startup-script from my bucket, so it is not completely isolated. My script doesn’t set up NTP because Google does it for us. It uses the virtualization host as the NTP server so it should be closer than the Storage bucket. I can’t say. Just give it some more time.

      Perhaps you should switch to some other zone within your region or to another region altogether. I just tested this in us-central1-c and it went smoothly.

      Reply

  20. How would you setup a email notification In Unifi hosted on GCP for down device’s?

    Reply

    1. You need to set up authenticated SMTP to some server. You can use your ISP’s SMTP or it could even be Google’s. There’s a paragraph on this under Maintenance and troubleshooting.

      Reply

  21. Thanks, I have this up and running it’s been really useful. Just been working on sending emails, signing up to Sendgrid via GCP appears to offer more than their normal free plan in terms of number of emails you can send and although the GCP instructions talk about using postfix, the Unifi Controller will allow us to specify the port for sending email. So as soon as sendgrid is configured and we can create an API key and use port 2525 which isn’t blocked to send emails. 🙂

    Reply

    2. +1 for me, signed up for Sendgrid and used smtp.sendgrid.com / 465 / SSL with the auth credentials that Sendgrid prompted me to use from my control panel in the SMTP server config of the UniFi controller. No need for the bits in the Sendgrid guide on the GCP marketplace for Postfix.

      Reply

  22. Any way to force update to 5.8.28?
    Installed today but got an issue when importing the backup as my PC already has 5.8.28 and the VM now has 5.8.24.

    Reply

    1. You need to log on to the VM through the Google Cloud Console SSH button. Then download the version of your choice using curl. Install the package via sudo dpkg -i unifi_sysvinit_all.deb

      Reply

      1. Hi, I’ve rebooted the VM but still running 5.8.24 and it didn’t upgrade to 5.8.28.

        Is that correct – you mentioned the VM would auto upgrade to stable releases so I assume 5.8.28 isn’t considered stable currently.

        Reply

        1. After a release is promoted from stable candidate to stable it still takes about a week before it is made available in the repository. Most often it appears on a Monday, but that is not a firm rule. When the release is in the repository unattended-upgrade will install it after 04 AM automatically. No reboot is required. See UniFi Controller Releases

          Reply

          1. Thanks so much for your reply, makes perfect sense to me. I’m in no rush to upgrade, just more wonding to make sure my install is working as it should (auto upgrade etc).

            You’ve done a great job putting all this information together, really appreciate it all.

            Reply

      2. Thank you! Figured our myself after reading on the forums that it takes one week for the new release to be added to the repo. Used wget and dpkg but the fun thing was that I could do this from my mobile using the Google cloud app which has built in SSH. That was an epic experience 🙂

        Thank you very much for publishing this solution and keeping it updated. It’s working perfect for me now 🙂

        Reply

  23. Petri, what if i need to upgrade the controller software? How do i proceed?

    Thank you!

    Reply

    1. You have two alternatives. The easy one is to wait for a few days for the latest release to hit the repos. Unattended-upgrade will install it for you overnight. The other one involves Unix command line and here is a copy of what I answered Balazs Szabo above:
      You need to log on to the VM through the Google Cloud Console SSH button. Then download the version of your choice using curl. Install the package via sudo dpkg -i unifi_sysvinit_all.deb

      Reply

  24. Petri – you must have put a lot of time and effort into this so thank you very much for that.

    This is the first time I’ve tried Google and your instructions were excellent. I had a controller up and running in no time and I’m pretty impressed by the performance considering it’s free. I’m now using the instance as my test controller having disabled the auto-upgrade job and done a manual upgrade to one of the Beta versions. Working great 🙂

    Thanks again..

    Reply

    1. I wrote the script and the article because when I suggested people to use cloud based controllers I got two objections: “I don’t know how to use cloud services” and “I don’t know how to use Linux command line”. The first one can be learned in minutes and the second one can be completely circumvented with the script. My idea was that user never needs to log on. Your post reveals that you are not in the intended target group 🙂

      You can use the script as a starting point, though. If you look at GCP Unifi Controller startup script explained you can see what the script does and at the end there is a list of files which were modified during installation. That post is not quite up to date with the current version, but close enough. I have a couple of improvements in the works and then I’ll update it.

      Reply

      1. No. You may want to look what I answered Rob on 2018/11/21. Rob wanted to install a legacy version. You can do the same to install a beta version. There is no way to auto-update the beta, though. You need to download the new, remove the old and install the new package of your choice.

        Reply

  25. First thing is thank you for all your work with this script and tutorial!

    Next thing is I wonder how the new billing setup on google cloud will affect the tutorial and then free teir.

    From the email.
    “Hello Google Compute Engine Customer,

    We are writing to let you know that on October 1, 2018, Google Compute Engine instances with predefined machine types will be billed by their individual resource types (i.e., vCPU, Memory) instead of by their machine type (e.g., n1-standard-2).

    What do I need to know?
    In order to provide additional savings and simplicity, starting October 1, 2018, predefined machine types will be reported as individual vCPUs and memory SKUs. This means that you will receive sustained use discounts for your instances based on your use of the individual vCPUs and GB of memory instead of being limited to instances of the same machine type.

    What do I need to do?
    The changes will take effect automatically and will result in either the same or lower effective bill. No action is required on your part.”

    Reply

    1. I don’t KNOW, but if you trust Google then the last paragraph says the charges won’t increase. Since zero charge can’t decrease either we’ll have to assume it will stay free.
      I hadn’t seen that notice, thanks for the heads-up. [Edit: I received the same email a couple of hours later.]

      Reply

  26. I followed the instructions, but the webpage won’t load. It just says Unable to Connect. I can ping the IP address, but I cannot access 8843 through the browser. I made sure all the ports were allowed in the firewall rules, but it acts like it basically ignores them all. If I use the Open Port Check Tool, all the ports are closed. #Frustrated!

    Can anyone help me please? Thank you.

    Reply

    1. Can you access port 80? It should redirect to port 8443 immediately but you can see the redirect in your browsers developer console. Lighttpd is listening on 80 and doing the redirect. Another debugging option is to look at Serial Port 1 (console) to see the output. The link is available on the VM details page.

      Usually it is easier to delete the VM and start over. You have made a small error at some point. Hopefully you won’t make the same error twice. On popular omission is to not assign the correct network tag to the VM. The network tag ties the firewall rules and the VM together and it would appear as you describe. Watch the video once more and make notes of the steps.

      Reply

    2. Hi. Thanks for your article. However when I created it it says “Bad Request
      This combination of host and port requires TLS.”?

      Reply

  27. I already have my controller running on GCP. But I haven’t been successful in getting a cert applied. Do you have a version that checks to see if a stage is already done and skips those steps? I don’t want to mess up my install. Thank you!

    Reply

    1. My script checks for most of the steps so it won’t retry them on every boot. It can’t detect what you have done manually since there are so many ways to skin a cat. I didn’t intend my script to be run on top of a manual installation so I don’t recommend it. If the cert is the only problem why not try this script by Steve Jenkins. It does pretty much the same as mine in a single-shot fashion.

      Reply

  28. Thank you so much for pulling this together Petri, I’m sick of having cloud keys failing so was thrilled to find this tutorial and amazed to have a controller running 45mins after signing up for a GCP account. You’re a legend!

    Reply

      1. Thanks! Although I have one small problem; I made a typo when entering the value for my storage container bucket so backups aren’t transferring. Is there any way to correct this (eg editing the VM instance) or do i need to set up a new instance and run the script again?

        Reply

        1. You can just edit the metadata and reboot by stopping and starting the VM.

          Reply

  29. Hello Petri!
    Thank you very much for your hard work!
    A couple of questions: can I use the LTS version of the controller? Will your automatic update script override it with latest stable?
    And am I obliged to use DDNS? If Google gives you a static IP can’t I just use that? Or is it needed for Let’s Encrypt?

    Thanks!

    Reply

    1. You need to edit the script. After you have edited it to install the LTS version you can either save it in your own bucket or copy-paste it to the startup-script meta-data field in the GCP console. Unattended-upgrades will keep all installed packages up to date. (I have never figured out how to install LTS via apt-get.)

      Another untested solution is to run the script as it is. Then log on and uninstall unifi package and manually install your choice. To prevent unattended upgrades from updating the controller you remove line "c=ubiquiti"; from /etc/apt/apt.conf.d/51unattended-upgrades-unifi

      No. You only need to fill in the meta-data fields you want. If you don’t add a bucket your backups won’t be copied. Without timezone GMT is used. If you have your own DNS records you don’t need ddns. If you don’t include dns-name you won’t get a certificate. Certificates can only be issued to DNS names, not to bare IP addresses.

      Reply

  30. If anyone is confused on the dynamic dns portion, use the reserved static IP address (reserved in step 3) as your destination address. This is the name on freedns.afraid.org. The name on other sites may vary. Continue as Petri shows (in his video) using the direct URL in the value field of the ddns-url key and the subdomain.domain in the value field of the dns-name key field.

    Reply

  31. Dear Petri,
    We want to use certbot .. after using your script.
    I did not use the ddns services ..
    What will be the root map for the certification of the ssl .. (var/www/html) .. in your script we have no ‘home-dir’ as like /srv/htdocs/[domainname]
    BTW .. the main page is still running with ssl error, nice script and video !
    Regards,
    Eric

    Reply

    1. I’m afraid I don’t get what you are referring to with “root map for certification”. Current Let’s Encrypt certificates are in /etc/letsencrypt/live/yourdomain/ (actually there are symlinks to the certs). The script uses certbot’s –standalone plugin which will start its own web server for validation. That’s why I have to shut down Lighttpd for renewals.

      What kind of SSL error you are getting? I’ve never seen one or I would have fixed it 🙂 I occasionally check the setup with SSL Labs test which comes out clean. (I have OpenSSL 1.0 which doesn’t include TLS 1.3 so it gives me a B rating. An upgrade is on my to-do list…)

      Reply

      1. The next error was coming up after the command:
        #certbot certonly –webroot -w /var/www/html/ -d wifi[domainName]

        Failed authorization procedure. wifi.[domainName] (http-01): urn:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching https://wifi.[domainName]:8443: Invalid port in redirect target.
        Only ports 80 and 443 are supported, not 8443

        I will change 8443 to 443 for generating the key.

        Reply

        1. This article is about the script I wrote. I am trying to help users who have problems with it. If you want to do it on your own you should read https://certbot.eff.org/docs/using.html#getting-certificates-and-choosing-plugins at least.

          If you need a script to fetch the cert then take a look at Steve Jenkin’s script. It is quite close to mine but will only fetch the initial certificate. You need to set up the update cycle and import the renewed cert to the Java key store.

          Reply

          1. Dear Petri,

            At this moment I had made the correct ssl for port 443.
            When I check this on the url name: https://wifi.[domain_name]/manage/account/login?redirect=%2Fmanage
            But of course I get the error that the page was not forewarded correct.
            It has to go to port 8443.
            What will in combination with the lighttpd webserver the best way to do.
            A standard redirect to port 8443 .. will give the situation that the site is not secured.
            ssl cert. generation for port 8443 is not allowed .. can you explain how you fixed this with the ‘ddns solution’ in your script.
            From their I can hopefully make from their to the correct forwarding from port 443 to 8443, what unifi needs.

            Reply

            1. Redirections won’t help with a cert error. My guess is that you haven’t imported the cert to the Java keystore as per Steve Jenkin’s script.

              If you need help with Lighttpd configuration and redirects then you should look at Lighttpd documentation, forums and mailing lists.

              Reply

    1. Yes, it can be done, but my script is GCP specific. It would be quite easy to adapt: how to access the metadata and the rsync command are the only specifics that come to my mind.

      Reply

      1. Let me first say, thanks for the GCP script.
        It was super, super easy to setup!!

        Is making a Vultr script on your bucket list?

        Reply

        1. No, I am not going to expand this. I wrote my script as a proof of concept that you actually can install UniFi controller in a cloud VM without any Linux skills. Many part time admins are not comfortable with remote VMs or Linux command line. I chose GCP because you can run the controller there for free. Anyone can fork my script from GitHub and modify it for Azure, AWS or whatever they desire. You could do it for Vultr! The only CGP specific parts are accessing the metadata about the VM (very similar in other system) and rsyncing the backups to a storage bucket (simple as well).

          Reply

  33. Thanks so much for the script and this amazing guide!

    One question: does the unifi controller software updates by itself? Or do i have to use apt update to get the latest version?

    Thanks

    Reply

    1. Yes, it will. Not at once though, but after Ubiquiti uploads the latest release to the repositories. It is typically about a week from the announcement. The idea is that the user never needs to log in to the VM. Everything is done within UniFi Controller or GCP Console.

      Reply

  35. Is there no risk to leaving all these ports open to all ip addresses? I am working on a script that can limit what devices can talk to the server via iptables and specific ports that also auto updates iptables when the dynamic ip changes.

    Also has built in functionality to allow additional ports to be opened up for your dynamic “admin” ips.

    Reply

    1. Yes, there are risks, but reasonable in my opinion. When a controller adopts devices they exchange credentials, which are used for future device to device communication. The biggest risk lies in the management interface where the user can choose the password. That’s why I stressed the importance of a good password. I also added Fail2Ban to the mix to make password guessing harder. Only the necessary ports are open in the GCP firewall. For example SSH is not and you would also need to take extra steps to allow remote logins to the VM.

      Typically the UniFi devices are behind a NAT device with a dynamic IP. Trying to track this automatically would introduce more points of failure with questionable benefits (since the devices are strongly authenticated). Limiting the IPs where the user can reach the management interface is even more tricky. Most probably the users would be locked out without skills to open the lock by logging to the VM and editing IPTables manually.

      You always need to choose a sweet spot between security and usability. I targeted my script to average, small scale users, with limited technical skills. Yes, you can harden your controller, but it will require more maintenance. If you require high security you should keep the server in your own data center. If you have satellite offices connect them by static VPNs so all connections are controlled and secure. For remote administration you should set up dynamic certificate-based VPN to the controller.

      Reply

      1. Well said! Thank you for this script btw.. it works awesome! I wasn’t aware of how strong the security was between the devices and the controller.

        Reply

        1. Do not use the pastebin from above as it will leave you unable to access your server on port 22.

          I have update the below script to take into account how Google Cloud handles SSH access. This script allows you to block all access to Google Cloud Compute Unifi Controller except by the dynamic dns names defined by you in the script. It will run as often as you want via crontab and polls the dynamic IP address and updates the firewall on the Unifi Controller if a new IP address is detected.

          https://pastebin.com/VA6T0hKj

          Reply

  36. Hi,

    I just restarted the vm but in the console i see this:
    unifi-controller login: Sep 28 00:53:41 unifi-controller google_metadata_script_runner[778]: AccessDeniedException: 403 518175677229-compute@developer.gserviceaccount.com does not have storage.objects.list access to petri-unifi.
    Sep 28 00:53:41 unifi-controller startup-script: WARNING Could not download gs://petri-unifi/startup.sh using gsutil. Command ‘[‘gsutil’, ‘cp’, u’gs://petri-unifi/startup.sh’, ‘/startup-jZMNIO/tmpJaC8dm’]’ returned non-zero exit status 1.
    Sep 28 00:53:41 unifi-controller startup-script: INFO No startup scripts found in metadata.
    Sep 28 00:53:41 unifi-controller startup-script: INFO Finished running startup scripts.

    Can i just ignore this? Or do i have to restart again?
    Thanks

    Reply

    1. It is my fault, I’m sorry. I did receive some e-mails from Google saying I need to upgrade, but I couldn’t find any hint in the GCP console about this upgrade. So I decided to ignore the e-mails. Apparently they were for real. Now I have “upgraded” my account and everything should be smooth again.

      This incident didn’t affect any controllers already running. The running controllers don’t need the script after the first boot unless you change the metadata. You can even remove the startup-script-url row from the metadata. You just couldn’t create new controllers while the script was not available. If you had your controller already set up you can just ignore the error. Or you can restart to verify the problem is gone, if you want to help.

      Reply

  37. Awesome work but for the life of me I couldn’t connect. Console log shows:

    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Starting startup scripts.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Found startup-script in metadata.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO startup-script: /startup-J506OP/tmpdAqmV6: line 1: gs://petri-unifi/startup.sh: No such file or directory
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO startup-script: Return code 127.
    Oct 4 01:13:37 wiz-unifi-controller startup-script: INFO Finished running startup scripts.

    Which I assumes means I have a lovely Debian VM without your script running

    Reply

    1. I’m afraid your assumption is correct. Although the script wouldn’t continue running, it would only install what was required and quit. In your case it didn’t run at all.

      I couldn’t reproduce this. I tried with my own account, but then it could be some obscure permissions issue. I got a friend to spin one up on her account and it succeeded as well. Could you please delete the VM and try again? Please be careful with extra characters that may be produced by copy-paste. I can’t see any on the third line of your console output, but it might be the cause. Or Google had a hiccup on their system. As you can read from the comments, many have succeeded with the first try. I believe even more don’t report it in a comment.

      Reply

        1. Works for me™ Your mileage may vary.. Both upgrading an existing controller and creating a new one. Usually new Controller releases aren’t the problem. The problem lies in the compatibility with prerequisite software. If only Ubiquiti would make their software compatible with current Java and MongoDB versions.

          Reply

    2. Replying to an extremely old comment here for the sake of anyone Google-ing this error—the startup script only worked for me on us-east1 region when I set the startup-script-url metadata to the actual https:// endpoint of the script instead of the gs:// endpoint. The equivalent to gs://petri-unifi/startup.sh in Google’s HTTP Storage API is https://storage.googleapis.com/petri-unifi/startup.sh

      Reply

  38. OK, so what needs to be done to allow public key SSH? 🙂

    I’m able to use the SSH button in the cloud console, and I can use ‘gcloud conpute ssh [unifi-instance-name]’. But when I try to ssh from a host a home, it fails. I’ve allowed port 22 on the VCP Network Firewall rules. And I can see that the traffic that matches that rule is allowed and passes to the host.

    Is there anything else in the VM that needs to be changed to allow SSH? (I’ve added my keys in the instance already).

    Thanks and awesome job on this!!

    Reply

    1. This isn’t about UniFi Controller at all but GCP VM administration. I wrote the script so people would never need to log on to the VM.

      GCP creates a new, temporary user account when you log on through the Console. When you log out the account is removed with the keys you installed. You need to create a new, static user with adduser and install the keys for that user. If you want to use sudo for the new user, you need to create a new file in /etc/sudoers.d with
      username ALL=(ALL:ALL) NOPASSWD:ALL.

      Reply

  39. Petri, thank you for sharing this with the community, it’s absolutely brilliant.

    I’m all up and running although as I don’t have a domain name to use at the moment I left the SSL config key out. I’ll probably just access the controller via unifi.ubnt.com so can you think of any reason to apply a DNS name / SSL cert (incase there’s something I’m missing).

    Also can you see any issues with turning auto-backups to weekly in the UniFi controller?

    Thank you again for sharing!
    Dan

    Reply

    1. You can use it without a domain name. You can get a domain name for free, though. I personally had far too much trouble with WebRTC that the cloud service is using. I find it easier to use the controller directly over https. Keep your password long and complex. Anyone can try to guess it over http.

      Your free bucket allowance is 5GB and there is about the same space available on the VM. As long as you don’t fill either one (they are synced) with the backups you should be good to go. I like to keep 26 weekly backups (or 6 months worth) but they are quite small. The network is small so there isn’t that much statistics to gather.

      Reply

      1. I found an old domain name I had for testing, added the key for the certificate, shutdown and restarted the VM and I’m in business, SSL all added – thank you again!

        Reply

  40. Thanks!

    Notes: In my case, on Firewall create, there are separate fields for TCP and UDP, had to paste the port list to each separately (tcp: “8443,8080,8880,8843,6789,443,80,22” and udp: “3478”.

    Oddly enough, when editing the same rule, there is only one field: “Specified ports”, containing “tcp:8443,8080,8880,8843,6789,443,80,22; udp:3478” 🙂

    other than that it seems to be working very well. How much can I scale this (if I pay)? Should I set an administrative limit every such-and-such many sites/devices/clients?

    Reply

    1. The user interface has changed – thanks for the heads up! I’ll update the instructions, but I am not going to shoot the whole video again. Google keeps making these small changes and they never announce them.

      There is very little information on scaling. Take a look at UniFi – Server Hardware & Database Management. It basically says that you should keep monitoring the CPU & RAM usage. They are mostly affected by the number of APs and other network devices unless you run a captive portal as well. In that case the number of client devices starts to matter as well. The good thing about cloud computing is that you can easily move to a bigger instance as needed. You don’t have to buy spare capacity that you expect to need, you can always use exactly the optimal size. Oh, if you are going to pay for the service use the nearest Google data center to keep the latency down.

      Reply

  41. Hi, I’m new to GCP and of using your script. A couple of questions:
    1. Do I run your script from the GCP command line? (I’m on a Mac and had a bunch of “command not found” when trying to run the script in Terminal.

    2. Do I setup the DDNS after the controller has been created by the script? (How do I know what the IP address of the VM is otherwise?)

    Thanks for all your work on this.

    Reply

    1. 1) You add a startup-script-url metadata field in the virtual machine config. That way the the VM will run the script automatically at boot. Watch the video one more time.
      2) If you want to set the DDNS statically then you can set it up when you have reserved the IP address. You don’t need to add the dns metadata field in that case.

      Reply

  42. I’m getting some possible errors when running the script. In particular, a few “is configured multiple times” and “systemctl not found”. I’m running the script from my home directory. I started it, then cancelled to run as root. Screenshot is on Dropbox and linked.

    Reply

    1. You are apparently running the script manually on your local machine. My script is intended to be run automatically at boot on the remote virtual machine. That’s why it doesn’t work. Did you watch the video one more time as I suggested? There is no command line involved.

      Reply

  43. I haven’t rewatched the video yet. I promise I will. I setup my VM following the command line info posted on https://www.instructables.com/id/Unifi-Controller-on-Google-Cloud-Platform-GCP/ (he’s how I found your web page). When I follow his last command,
    gcloud compute instances create unifi-controller \
    –machine-type f1-micro \
    –image-family ubuntu-1604-lts \
    –image-project ubuntu-os-cloud \
    –boot-disk-type pd-standard \
    –boot-disk-size 25GB \
    –zone us-central1-c \
    –description “Unifi Controller” \
    –scopes=default,storage-rw \
    –tags unifi-server \
    –address unifi-external \
    –metadata=startup-script-url=gs://petri-unifi/startup.sh,timezone=US/Central,dns-name=your.domain.com,bucket=some-unique-name

    it completes at command line without error. However, when I login to the new VM (I setup the DDNS and it works) the certificate didn’t load (he says the same thing on a question I posted on that page). Any idea what I might do to get the cert to load correctly? Also, does the script run every time the VM is called from a web browser? It’s OK if it does I just want to know for knowledge sake.

    Reply

    1. You are asking me to help with MallocArray’s instructions? I haven’t ever tried them. I trust him though, but you should ask him for help.

      My script creates a log at /var/log/unifi/gcp-unifi.log Any hints there?

      Does the DDNS name resolve from any computer on the Internet? (Try from work, library, your cell phone using mobile data, neighbor’s..) It must resolve from Let’s Encrypt’s servers in some data center.

      The script runs on every boot in case you change some of the metadata. With no changes it won’t do much anything. You can remove the startup-script-url from the metadata if you don’t want to run it every time. (I trust you did change the metadata on the last line of the command: dns-name and bucket, didn’t you?)

      Reply

      1. Hi Petri,

        No, not asking for help with MallocArray’s instructions. I worked through his command line stuff pretty easily. Just wanted to show you his command automatically calls your script. (I watched the section of your video a couple times to make sure the metadata was exactly the same. It was.)

        I can run the VM from every Mac and iOS device. So, that works. The DDNS setting is operational. I’ll have a look at the log, and post back if I find some useful info. Thank you so very much. You are appreciated!!

        Reply

        1. The log only lists this:
          CertBot run on Mon Nov 5 16:54:48 CST 2018
          Certificate files missing

          This would have been about the time I finished with everything on 11/5/2018.

          Reply

          1. I just created a new VM using my instructions. It got a certificate just fine. So we can sum this up: there are two sets of instructions. One that works (mine) and one that appears to have problems. Is there some reason why you won’t try my instructions? It will only take you ten minutes to create a new VM. You can even transfer the static IP to the new VM so you don’t need to tinker with DDNS.

            I also noted that MallocArrays’s instructions are for Ubuntu 16.04 LTS. I guess you are using Ubuntu, too. I have only tested mine on Debian. Ubuntu is based on Debian, but they do differ. I can only support you if you follow my instructions to the T (or you have some really exceptionally good reason not to).

            Reply

    2. Watched the video again. I’m reasonably certain I’ve got everything configured properly. My outstanding problem seems to be the certificate. Please have a look at the web page I listed above. You’ll see the fellow who admins that page said he had deployed a new VM in the last couple of weeks and the certificate wasn’t working for him, either.

      Reply

      1. Hi Petri,

        I’ve been in IT for quite a few years, and command line is not something I’m shy about. The only reason I haven’t followed your instructions to a T is because I was confused about what the script does & doesn’t do. (For whatever reason, I did not realize I had to start with 1. Preliminaries. I was under the impression the script did everything for me. My bad.) Am I correct that I have to follow the step-by-step as outlined in both the video and the verbal post here (beginning with preliminaries)?

        I was also going to try adding CertBot to my existing VM but wasn’t sure of the choice for I’m using (Apache, Nginx, Haproxy, Plesk or None of the above).

        The Ubuntu VM I created uses 25GB, so I’m thinking I’d have to destroy that or register a new Google Cloud account in order to use your instructions. Does that seem correct?

        Reply

        1. I made the video to give you the big picture and help decide if you will be able to make it through. Creating a virtual Linux machine in some cloud service sounds intimidating if you have never done it. When actually doing the steps it is easier to have them written down instead of pausing the video. Essentially they are the same thing, but there is more detail in the text.

          If you just created your GCP account then you should have received a couple of hundred dollars worth of credit. You can run several VMs side by side for the money. Even if you don’t, it won’t cost you but a few dollars. You can only attach the IP to one VM at a time, though.

          Reply

          1. Ultimately, I stopped the Ubuntu VM, changed the static IP it was using to None, then deleted the VM. I followed your directions. The only problem was when I got to the Networking section, it didn’t see the original reserved static IP. I released that, created a new one and attached that to the new VM. After I created the new VM and having changed the DDNS IP, I connected to the new VM & YES, a certificate was attached. Thanks! Now, I’m trying to figure out how to get the APs which are still connected to the proper sites in the VM but show as Disconnected probably because they were setup and adopted on the local Unifi Controller. I’m guessing until I’m back onsite with those APs I won’t be able to get them on the new cloud controller.

            Reply

            1. You followed instructions and succeeded. Good! There might be a lesson in there 🙂

              Did you also go through step 5 to the end? You should first set the Controller Hostname/IP on the GCP controller and then on the old controller. That’s what will make the devices connect to the new controller.

              You probably had the IP in a different region. Static IPs are tied to a region, you can’t move them. Makes perfect sense if you think about it from the routing perspective.

              Reply

  44. There’s no question there was a lesson in there and the main reason I went thru all of this was to learn. Success is a great motivator!

    I would like to make one suggestion. I think a preamble explaining to very newbies such as myself, that one HAS to go through the steps to make this all work is important. I never think I’m so unique I’m the first to have the ideas or questions I have. So an explanation that the steps need to be used and THEN the script kicks in at the end would have helped me. If this is the very first time someone didn’t understand (me) I will chalk it up to a new experience.

    Thank you for a great tutorial, but just as grateful you are willing to respond to help anyone who asks questions. So, I have one more: In the reply process, I give my email. Is there anyway to get an alert via that email you have responded? Another of my naive expectations. I thought I’d get an email after your responses, but those didn’t come. I didn’t see your first response til the early afternoon yesterday. Regardless, you are a prince! Very big thank yous for teaching me something new!!

    Reply

    1. Thanks for your input, but I try to minimize edits to old posts. Let’s see if I get more complaints. I also feel I would belittle readers, if I wrote a cookbook and started by explaining that you really need to take all the steps. You can’t just jump to the end and expect to find a roasted turkey in your oven! 😛

      This site is built around WordPress. WordPress used to notify commenters, but apparently the function has been removed. The email address is not verified by any means so it would be very easy to bully someone by generating thousands of comments around the Internet with the victim’s email. It is sad the Internet has become the environment it is today. I have been using the Internet since the early 80’s, when you really could email any textbook author or a professor in any top university and expect a reply.

      Reply

  45. I’m having a tough time getting this to work. I got it to work initially once, but then I deleted the VM and started over. Since then, just can’t get anything working anymore.

    Looking at the console, here is what I find.

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 26.)

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype

    Nov 12 06:00:12 unifi-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:

    [ 27.127267] random: crng init done

    [ 27.131931] random: 7 urandom warning(s) missed due to ratelimiting

    Reply

      2. It seems I’m still getting an issue. It doesn’t look like the script is running.

        Looks like it finds the script and downloads it

        “`
        Nov 13 01:08:49 unifi-controller startup-script: INFO Downloading url from https://storage.googleapis.com/petri-unifi/startup.sh to /startup-CjurIU/tmpgH4bCM using authentication token.
        “`

        “`
        unifi-controller login: Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to stop mongodb.service: Unit mongodb.service not loaded.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to disable unit: No such file or directory

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 168: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/filter.d/unifi-controller.conf: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 172: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/jail.d/unifi-controller.conf: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: /startup-CjurIU/tmpgH4bCM: 180: /startup-CjurIU/tmpgH4bCM: cannot create /etc/fail2ban/jail.d/unifi-controller.local: Directory nonexistent

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to reload-or-restart fail2ban.service: Unit fail2ban.service not found.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Failed to start dbus.service: Unit dbus.service not found.

        Nov 13 01:08:54 unifi-controller startup-script: INFO startup-script-url: Trying to start dbus

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Failed to start dbus.service: Unit dbus.service not found.

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Failed to create bus connection: No such file or directory
        “`

        “`
        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Backups to snunifistorage set up

        Nov 13 01:09:09 unifi-controller startup-script: INFO startup-script-url: Return code 0.

        Nov 13 01:09:09 unifi-controller startup-script: INFO Finished running startup scripts.
        “`

        This is everything I have with startup-script

        Reply

        1. Which region/zone are you using? I just tested in us-east-1b and it works. It could be a discrepancy between zones or regions. The suggestion to use ‘dpkg –configure -a’ tells me it is the same GRUB bug.

          Reply

          1. us-east1-b is what its sitting on.

            I will delete the VM and start over again, see if it changes anything.

            Reply

          2. Just wanted to let you know it works now!

            I wonder if it was loading a cached version of the script then?

            Reply

    1. The script will do it for you. You just enter the dns-name metadata. If the DNS name matches with the public IP, the script will acquire a certificate. If you want to download it, it is in /etc/letsencrypt/live/

      Reply

  47. Hi, installed perfectly but the lets encrypt cert is saying its invalid with the reason below

    This CA Root certificate is not trusted. To enable trust,
    install this certificate in the Trusted Root Certification
    Authorities store.

    Reply

    1. What OS are you using? (Win/macOS/Linux/iOS/Android/..) Which browser?

      Is the certificate really issued by “Let’s Encrypt Authority X3” and the root is “DST Root CA X3”?

      Reply

      1. Hi Petri,

        cert says issued by unifi, im using windows and have tried all browsers with no luck?

        Certificate
        General Details Certification Path
        Show:
        X

        Field
        Version
        Serial number
        Signature algorithm
        Signature hash algorithm
        Issuer
        Valid from
        Valid to
        n S. ihi^rt_________________________
        Value
        V3
        5be2c937
        sha256RSA
        sha256
        UniFi, UniFi, ubnt.com, San Jo.
        07 November 2018 11:15:03
        04 November 2028 11:15:03
        I IniFi I IniFi i ihnf rnm ?an In
        V3

        Reply

        1. So it is not the Let’s Encrypt certificate that is not trusted, but the default UniFi certificate. The script couldn’t acquire a Let’s Encrypt certificate. The most probable cause is that Let’s Encrypt server cannot connect to your instance. It could be a firewall issue. My instructions tell to allow TCP to ports 443 and 80 on the VPC for this reason. It could be a DNS issue. Does the dns-name resolve universally to the instance IP address? Universally means that from anywhere, not only on your computer. You should try it at home, work, library or on neighbour’s computer.

          Reply

            1. Just to verify: You get the bad certificate warning if you just type the DNS name into you browser address bar? Like your.server.dns.

              If you want to debug this further you need to SSH into the VM command line. There is a SSH button in the GCP Console. You can view the script log with command
              sudo more /var/log/unifi/gcp-unifi.log
              Other logs that may help are:
              sudo more /var/log/letsencrypt/letsencrypt.log
              sudo more /var/log/unifi/server.log

              The other way is to spin up a new VM and transfer the IP address and the backup of your UniFi Controller to it. Keeping your fingers crossed that you’ll do something slightly differently this time. If it is a DNS problem it won’t help, though. While you are at the command line you should test how the VM sees the DNS with command
              getent hosts your.server.dns

              Reply

    1. I am not able to reproduce this. Neither with a new controller or an upgraded one. Did you use my script for your setup?

      Joseph McParland in Willie Howe’s YouTube comments suggested the problem stems from the Windows CRLF combination. My script is pure Unix style LF. Thanks for the heads-up, though!

      Reply

  49. Hi Petri,
    great video and instructions. Three concerns/questions:
    1. When I type xxx.duckdns.org nothing pops up in my browser (xxx would be the name of my DDNS name)
    2. Can I edit the values once the VM is created?
    3. Can I delete the entire project and start over? What does this do to my Lets Encrypt Cert?

    Thanks

    Reply

    1. 1) There is something wrong with your DNS. You should check to see where does your DNS name point to. There are some web tools or local command line tools like nslookup and dig.
      2) Yes. The new metadata will take effect when you stop/start your VM.
      3) Yes. Your cert will be deleted with the VM. LE will issue you a new one. However, LE will issue only five certs per week to all nn.duckdns.org servers together.

      Reply

  50. Hi Petri,
    I”m not sure if I got this to work. I’m not entirely clear on DDNS vs DNS. Right now, I only have a duckdns domain. Let’s say it’s called test.duckns.org Am I also required to have a DNS domain e.g test.net? Right now, my duckdns points to my public IP (at my house).
    Also, I’m starting from zero on this project. Do I need an old Unifi controller setup to proceed? I don’t have one. I was hoping to not have to buy a Cloud Key to get this to work
    Thanks

    Reply

    1. The difference between dDNS and DNS is that dDNS can be updated dynamically without editing a config file or restarting a server. dDNS is used for dynamic IPs that change frequently or even occasionally. In case of GCP VMs the IP won’t change, but dDNS names are available for free. My idea was that anyone could deploy this solution without spending a nickle.
      No, you don’t need test.net, test.duckdns.org will do fine.
      You need another DNS name that points to the IP of your controller in GCP.
      Have you read the section “How to Set Up a New Network with a GCP Controller” above? In my opinion option A is the easiest (if you have a laptop) – that’s why I put it first.

      Reply

  51. Petri, amazing script. question for you. I would like to spin up an older version of the controller software, 5.6.39 or .40 because I have some legacy devices that have fallen out of support by the newer controller versions, how easy is it to modify your script to pull a specific version (this could be a useful metadata key, if it is there to pull that version perhaps?) ubnt will EOL their devices every few years and lots of people cant afford a complete upgrade for that reason…

    cheers!

    Reply

    1. You can do it, but it requires Linux command line. After you have spun up the VM, SSH into it from the GCP Console. Remove the latest UniFi package and install the one you fancy. You need to edit Unattended-Upgrades configuration and remove the line with "c=ubiquiti"; Otherwise the controller will be automatically upgraded to the latest one on the next run. The commands you are going to need:
      sudo apt-get remove unifi
      sudo nano /etc/apt/apt.conf.d/51unattended-upgrades-unifi
      curl -o installer.deb https://dl.ubnt.com/unifi/5.6.40/unifi_sysvinit_all.deb
      sudo apt install ./installer.deb
      rm installer.deb
      sudo systemctl enable unifi
      sudo systemctl start unifi
      This is from the top of my head, not tested. Should be close, though.

      No, I’m not going to make this a metadata option. It would be a potential source for far too many errors. Anyways, you need to log on to the VM to update the controller. My target were users who are never going to SSH to the VM.

      Reply

      1. you sir are fantastic, thank you very much for the prompt response and a very very very useful script. I will give this a shot, thank you!

        Reply

  52. Hi Petri,
    I set up my Google VM per your instructions. Just waiting for a cert from let’s encrypt for duckdns. I do have a question: I use OpenVPN TCP443 on my router and duckdns refers to my IP address. WIll there be a conflict if both Open VPN and Google VM use port 443?

    Reply

    1. You need two DNS names: One for your home router and another one that points to the external IP of the GCP VM.

      Reply

      1. Sorry for my confusion, let me see if I understand:
        I need one DNS for my router (xxxx.duckdns.org) which I currently use for OpenVPN on my router. I should not use this one when I configure my GCP VM, correct?

        I need another DNS (yyyy.duckdns.org) for my GCP VM, and this does NOT need to point to my router. This one is used only for GCP VM, poining ot the external IP. correct?

        Lastly, If my DDNS is yyyy.duckdns.org, is that also my DNS entry for the GCP VM? This is where I get lost. Thanks

        Reply

        1. Yes. You need two names for two addresses thousands of miles apart. The yyyy.duckdns.org should point to the external IP of the GCP VM and you use that name to access your UniFi controller.

          Reply

          1. Ok,
            Thanks. The only problem is it’s not working. I must have done something wrong. When I type yyyy(not real name).duckdns.org, nothing shows up in my browser.

            Reply

              1. Does ping yyyy.duckdns.org work or can you only ping ip.ad.re.ss ?

                If pinging the name doesn’t work then you need to figure out where the name points to. On a Mac you use dig yyyy.duckdns.org and on a Windows box use nslookup yyyy.duckdns.org from the command line.

                Reply

                1. When I ping yyyy.duckdns.org (not the IP address) I get a valid return: (Note, the IP address below is fake so I can post it, as is the DDNS name). The actual IP address IS the assigned IP by my GCP VM

                  Microsoft Windows [Version 10.0.17134.407]
                  (c) 2018 Microsoft Corporation. All rights reserved.

                  C:\Users\Bud>ping yyyy.duckdns.org

                  Pinging yyyy.duckdns.org [25.263.251.45] with 32 bytes of data:
                  Reply from 25.263.251.45: bytes=32 time=37ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=24ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=24ms TTL=55
                  Reply from 25.263.251.45: bytes=32 time=23ms TTL=55

                  Ping statistics for 25.263.251.45:
                  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
                  Approximate round trip times in milli-seconds:
                  Minimum = 23ms, Maximum = 37ms, Average = 27ms

                  C:\Users\Bud>

                  Reply

                  1. Microsoft Windows [Version 10.0.17134.407]
                    (c) 2018 Microsoft Corporation. All rights reserved.

                    C:\Users\Bud>nslookup yyyy.duckdns.org
                    Server: resolver1.opendns.com
                    Address: 208.67.222.222

                    Non-authoritative answer:
                    Name: yyyy.duckdns.org
                    Address: 25.263.251.45

                    C:\Users\Bud>

                    Reply

                    1. If the name resolves to the external IP of your VM then stop and start the VM. If it doesn’t solve the problem delete the VM do it over. It will take 10 minutes instead of days of troubleshooting.

                    2. Deleted VM and started all over again. Still batting zero. Not sure whatI’m doing wrong.

                      When I type yyyy.duckdns.org into the browser, I get nowhere.

                      This site can’t be reached yyyy.duckdns.org took too long to respond.
                      Try:

                      Checking the connection
                      Checking the proxy and the firewall
                      Running Windows Network Diagnostics
                      ERR_CONNECTION_TIMED_OUT

                    3. What happens if you type the external ip.ad.re.ss into your browser?

                      If you want to debug this further: Create a new VM and when it finishes open it in the GCP console and click on the Serial port 1 (console) link. There you can see the story evolve. You need to click the Refresh button to see more. The lines containing INFO startup-script-url are generated by my script. Don’t worry about the lines where debconf or dpkg whine about there being no terminal.

                  2. The same thing happens when I type in the external IP address (This site can’t be reached). However, if I ping the external IP or my DNS name (yyyy.duckdns.org) I get a valid ping return.

                    Not sure why the browser will not let me in. I’ll try establishing another VM and report back.

                    Reply

                    1. Double check your firewall rules and that you have typed the network tag exactly the same way in the firewall settings and the VM settings. Sounds like the firewall is keeping you out. It has kept Let’s Encrypt out as well so there will be no certificate. Once you have the tags and rules checked you should create a new VM.

                    2. I did see these entries on the start-up script. Normal?

                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Dialog
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: (TERM is not set, so the dialog frontend is not usable.)
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Readline
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: unable to initialize frontend: Readline
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: (Can’t locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, line 21.)
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: debconf: falling back to frontend: Teletype
                      Dec 5 17:36:31 unif-controller startup-script: INFO startup-script-url: dpkg-preconfigure: unable to re-open stdin:

                    3. Yes. Debian package installer is wondering why there is no terminal. There is no terminal because it is a headless server.

                      Did you doublecheck the firewall rules and tags?

                    4. allow-unifi
                      Description
                      Unifi Firewall Rule
                      Logs
                      Off
                      view
                      Network
                      default
                      Priority
                      1000
                      Direction
                      Ingress
                      Action on match
                      Allow
                      Targets
                      Target tags
                      unifi
                      Source filters
                      IP ranges
                      0.0.0.0/0
                      Protocols and ports
                      tcp:8443
                      tcp:8080
                      tcp:8880
                      tcp:8843
                      tcp:6789
                      tcp:443
                      tcp:80
                      udp:3478
                      Enforcement
                      Enabled

                    5. I “may” have found the problem. My external IP is on
                      us-east1 but my VM is on us-east1-b.

                      Is that what’s causing the problem?

                    6. No. The IP is tied to us-east1 region and your VM is in zone b. You can’t attach an IP across regions.

                      Do you have the network tag “unifi” also on your VM?

                    8. Ok, I deleted everything and started over. Now, when I type in yyyy.duckdns.org I’m taken to the following:

                      Placeholder page
                      The owner of this web site has not put up any web pages yet. Please come back later.
                      You should replace this page with your own web pages as soon as possible.
                      Unless you changed its configuration, your new server is configured as follows:
                      Configuration files can be found in /etc/lighttpd. Please read /etc/lighttpd/conf-available/README file.
                      The DocumentRoot, which is the directory under which all your HTML files should exist, is set to /var/www/html.
                      CGI scripts are looked for in /usr/www/cgi-bin, which is where Debian packages will place their scripts. You can enable cgi module by using command “lighty-enable-mod cgi”.
                      Log files are placed in /var/log/lighttpd, and will be rotated weekly. The frequency of rotation can be easily changed by editing /etc/logrotate.d/lighttpd.
                      The default directory index is index.html, meaning that requests for a directory /foo/bar/ will give the contents of the file /var/www/foo/bar/index.html if it exists (assuming that /var/www is your DocumentRoot).
                      You can enable user directories by using command “lighty-enable-mod userdir”

                    9. You need to have exactly the same network tag on the firewall rule and the VM.

                      Apparently you did something differently this time. You got connected to Lighttpd default page. What does https://yyyy.duckdns.org:8443 yield?

                    11. Good! When you got your DNS name properly configured and matched the network tags.

                      I can’t say why Lighttpd isn’t redirecting you. There is nothing you could have done to cause this. There are no choices in the script for it. It is a glitch, because I have never heard anyone else running into it and I can’t reproduce it either. My suggestion is to create yet another fresh VM now that you know all the steps.

                    12. That was it!!!!! I deleted everything and started from zero.
                      yyyy.duckdns.org now goes to the unifi controller page.

                      Thanks Petri.
                      I am getting a message on my VM :Instance “yyyy-controller” is overutilized. Consider switching to the machine type: g1-small (1 vCPU, 1.7 GB memory).

                      Is this normal?

                      Thanks again.

                    13. Yeah, that is normal. Google wants to sell you a bigger VM. I admit f1-micro is sluggish but I can live with that.

  53. Need some help here –

    I am getting a certbot error –

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO startup-script-url: /usr/local/sbin/certbotrun.sh: 7: [: x192.64.119.136: unexpected operator

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO startup-script-url: Return code 0.

    Dec 1 21:04:09 unificloudcontroller startup-script: INFO Finished running startup scripts.

    Any thoughts?

    Reply

    1. Sorry for the delay. My ISP goofed big time and stranded my servers. A total blackout for a few days and of course when I wasn’t there.

      I couldn’t reproduce this, so my advice is to delete the VM and create a new one. The error you posted is real, though. It is not from CertBot but a line from my script. It looks like the text in the script was corrupted somehow, that’s why I was concerned. In this case it appears to be a random bit error, although those are really rare.

      Reply

  54. Petri,
    Question: I do not have a Unifi profile to import yet. However, I wanted to put a username and password on the Unifi splash page for added security (in case someone guessed my DNS). I set one up. Can I still import my Unifi profile into the VM? Is there any way to reset it from zero and start all over again?

    Thanks.

    Reply

    1. Hi Petri,
      just curious if there’s a solution to reset the Unifi profile and start from zero. As I mentioned, I set up a username and password to prevent someone from accessing my Unifi Controler (not yet set up) on the VM.
      Thanks

      Reply

  55. Hello Petri, here again with some questions, I bougth a Domain name for unifi controller and try to add to unifi controller with only this metadata

    startup-script-url gs://petri-unifi/startup.sh
    timezone America/Mexico City
    dns-name mydomain.com.mx (provided by godaddy)
    bucket unifi-respaldo

    after installation mydomain.com.mx dont work, i only acces by static ip to the controller, if i point from mydomain.com.mx to static-ip it works loading static ip. Do i need to add ddns-url?

    Reply

    1. No, dDNS is only for dynamic DNS. Go to your GoDaddy console and add an A record for the domain name (mydomain.com.mx) with the static IP address. You probably need to stop/start the VM to acquire a https certificate.

      Reply

  56. I had an issue with the backup that I used so I purged the Unifi install and then re-started the server to run your install script. Will the Let’s Encrypt cert be reused or reapplied? It has been a few days and I still get a cert warning when connecting to the controller.

    Reply

    1. I don’t know what you mean by “purging the Unifi install”. If you logged on to the VM then all bets are off – I can’t know what you did. You should delete the whole VM and create a new one. The new VM will acquire a new certificate from LE.

      Reply

      1. I used the backup created with the export site wizard. That backup has issues with account usernames and passwords I found out after I had imported it into the VM. So I SSHed into the server and ran “sudo apt-get purge unifi” as I did not know how else to clear a controller that I could not sign into. I then stopped and started the server so that your script would run again. I then used one of my full backups and was able to log in fine. In your notes it says that the LE part of the script will keep trying until it adds the cert but I am thinking that the cert might still be on my VM and just needs to be applied again. I am not a linux guy so I cannot read what all your script is doing.

        Reply

        1. An exported site is not a full controller backup. It only contains a single site. That’s why I have instructed to download a full backup of the controller. You should try to first run the setup wizard on a new controller and then importing your site – not restoring.

          My script won’t work after you purged the UniFi package. The way UniFi Controller is written requires that the certificate has to imported into the Java environment after acquisition. Purging the controller will also delete the Java keystore. My script will just see that there already is a certificate in the LE directory so it assumes it has been already imported. It will acquire a new certificate and import it in three months. You could import the certificate yourself. Look for instructions on importing your own certificate to the controller.

          Reply

          1. Waiting 3 months will be fine if I am not able to figure out how to manually import it. Thanks for your responses and the work on your script.

            Reply

  57. Hello!

    I get a renewal error from certbot (real domainname changed)

    Attempting to renew cert (unifi.domain.tld) from /etc/letsencrypt/renewal/unifi.domain.tld.conf produced an unexpected error: Problem binding to port 80: Could not bind to IPv4 or IPv6.. Skipping.
    All renewal attempts failed. The following certs could not be renewed:
    /etc/letsencrypt/live/unifi.domain.tld/fullchain.pem (failure)

    Looks like lighttpd is blocking the port.

    I tried “sudo systemctl stop lighttpd” then a manual “sudo certbot renew” followed by “sudo systemctl start lighttpd” which worked.

    Any idea why the renewal script doesn’t work out of the box?

    Hyvää Joulua!
    //Anders

    Reply

  58. Not to pile on the above issue, but just wanted to report an issue after a recent certificate renewal:

    My cerificate indicates that it’s valid until 1/25/19 and has the correct FQDN in place. I’ve verified that other Let’s Encrypt certificates I’m employing are still being reported as valid – it only seems to be a problem with my GCP-hosted Unifi controllers running this script.

    I’ll have to look into this more over the weekend, but if there’s anything I can provide to assist I’m more than happy to do so.

    Regards and happy holidays
    -Dave

    Reply

    1. Thanks Dave and Anders for reporting this. I have run mine for a couple of years now and this has never come up. The renewals have worked like a clock. I will need to look into this. I will update this post as I resolve this.

      Och god jul till Anders och Dave!

      SOLVED: I added explicit scripts to stop and restart Lighttpd for renewals. You will get the changes if you stop/start your VM and have the startup-script-url meta-data pointing to my bucket.

      Reply

      1. Just checked in and it appears to be coming up valid now – thanks a ton, Petri!

        Reply

  59. I was playing around with my controller, and it seems that fail2ban is not working, I tried logging in with fake accounts like 15 times and I can still access the webpage. I have tried rebooting the server and that did not help.

    I checked Console port
    “[ [0;32m OK [0m] Started Fail2Ban service.”

    Reply

    1. In Regards to the above post, I Deleted and made a new VM, Fail2Ban is working again. Thanks for all your work!.

      Reply

  60. Thanks! Great work, got it up and running the second try.
    Have no idea what I did differently the second time… Now I just have to install my Ubiquiti gear and then I’m ready to roll. 🙂

    One thing in your video was faulty though and had to be corrected by myself. The center of the Universe is obviously in Sweden so I changed the timezone to Europe/Stockholm instead 😉

    Reply

  61. Hi Petri,
    I have my VM set up per your instructions and the VM is up and running (very nice thanks to you). I do have a question, however; How does the VM know how to point to my home IP address? I understand the DDNS on the VM always points to the VM, but I’m unsure how the VM points to my home IP (WAN).

    Thoughts?

    Reply

    1. The way it works is that the controller passively waits for the devices to make contact. That’s why the port 8080 was opened in the firewall. So you need to tell the devices to contact your VM. This is done in the the Settings > Controller > Controller Hostname/IP and also override the Inform Host. Changing this in the old controller directs the requests to the new cloud-based controller.

      Reply

      1. Thanks. I assume the devices speak to the controller over an encrypted connection. Correct?

        Reply

        1. Ubiquiti hasn’t really documented this. The connection is over http so it is not TLS. However the authentication is strong. When you adopt a device they exchange credentials. That’s why you can’t readopt a device without forgetting it first or manually resetting it. The same goes for the controller. It won’t accept connections from other devices it hasn’t itself adopted.

          Reply

          1. That’s interesting. So all the data the devices send to the VM Controller are sent in the clear (unencrypted). Shouldn’t we be concerned about this? It seems to me anyone sniffing packets on my connection could capture the data sent to the VM. Not sure why Ubiquiti doesn’t utilize port 443 for encryption. If they did, this would be a VERY secure system. Strange. (Maybe I’m just being paranoid).

            Reply

            1. Petri, you’re right. Not much info on how the AP’s communicate routine data with the controller i.e IP, Clients, Bandwidth, frequency, etc. It appears the adoption of AP’s by the controller is done via SSH. However, not clear if the routine GUI data transmitted by AP’s, switches, and Security Gateway is via SSH (although it would seem it would be).
              Here’s all I could find on how the AP’s talk to the controller.

              https://help.ubnt.com/hc/en-us/articles/204976094-UniFi-Communication-Protocol-Between-Controller-and-UAP

              Reply

  62. Great job setting this up. As a Linux and GCP user, I could have gone through all of this myself, but why reinvent the wheel? I did run into a problem with the certificate, however.

    The automatic attempt to set up the certificate failed with:
    certbot.errors.FailedChallenges: Failed authorization procedure. ddns.net (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the
    client to verify the domain :: Fetching http://.ddns.net/.well-known/acme-challenge/cMPyZMoPEdV3BtYY2nKg4CLxh1JKVvPVNzI8NXUAd04: Connection reset by peer

    I then ran your certbot script manually:
    root@unifi:/var/log/letsencrypt# /usr/local/sbin/certbotrun.sh
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator standalone, Installer None
    Obtaining a new certificate
    Performing the following challenges:
    http-01 challenge for coventry-unifi.ddns.net
    Waiting for verification…
    Cleaning up challenges
    Importing keystore /tmp/tmp.4AY2AuoRuJ to /var/lib/unifi/keystore…
    Warning:
    The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using “keytool -importkeystore -srckeystore /var/lib/unif
    i/keystore -destkeystore /var/lib/unifi/keystore -deststoretype pkcs12”.
    Unable to import the certificate into keystore

    Any ideas?

    Reply

    1. The first, automatic attempt failed because the dns-name meta-data didn’t resolve to the external IP of the virtual machine or a firewall blocked the connection to port 80. If you look at the link in the error message the ddns starts with a dot (or did you edit that?)

      The second, manual attempt first warns about the age of the code in UniFi Controller. I wish Ubiquiti would update the code to support current Java, MongoDB, keystore etc. versions. Then there is the “Unable to import the certificate into keystore” message. I have seen it occasionally, but it still did import the cert and https worked. My script would write “KeyTool import failed” if it really fails. Did you try to connect?

      Since you are Linux savvy, you can manually run /etc/letsencrypt/renewal-hooks/deploy/unifi and see where it fails. It is very straightforward script.

      Reply

      1. Thanks for the quick reply. The leading dot was where this content system stripped out my angle-bracketed dummy name. So I’m still confused as to why that didn’t work correctly.

        I just checked, and the controller is indeed using the new cert now, so you are absolutely correct regarding the erroneous error message. I suspect that when I was working on this last night, Chrome was caching the self-signed cert as well.

        Reply

  63. Thank you for the great write-up. Everything seemed working well so I moved two sites over. Now I’m getting an IPS alert every 2 mins from my IPS that I can’t seem to figure out. The error is only coming from one of the two sites. Do you have any thoughts on this? Thank-you

    ERROR:
    IPS Alert 3: Generic Protocol Command Decode. Signature ET INFO Session Traversal Utilities for NAT (STUN Binding Request obsolete rfc 3489 CHANGE-REQUEST attribute change IP flag false change port flag false). From: [my-original-controller-IP]:47887, to: [google-instace.IP]:3478, protocol: UDP

    Reply

  64. Thanks again for the write up.
    I had sent you a note yesterday. Just an FYI, it was the IPS settings = INFO causing the STUN Alerts

    Reply

  67. Amazing writeup! I’m running into a peculiar issue now:

    I’ve had this running succesfully for a few months, and all of a sudden can no longer connect to my controller from my laptop. Interestingly, I CAN using mobile data. Same by hot spotting my laptop with mobile data.

    Did something mess up in the GCP settings? I haven’t touched them.

    Reply

    1. Sounds like Fail2Ban has banned your IP. With mobile data you are using a different IP. The ban should disappear in an hour.

      Reply

  69. Hi Petri,

    This worked amazingly! Thank you. Will this work for multisite as well or will more than one site push me over the limits per month of the free tier?

    Reply

    1. The instance size limits the performance. It won’t go over any limit by itself. The inform messages the devices send won’t tax the server. Occasionally the controller will do summary calculations and then it will be unresponsive. The backup routine definitely locks it up for a while, but it happens in the middle of the night. As long as you can live with the sluggish performance you should be OK. The capacity is quite comparable to the 1st gen CloudKey and Ubiquiti promises it can handle up to 30 devices. After that you might want to upgrade the VM. The number of sites don’t matter.

      Reply

  70. So you can run this controller 24/7 on a micro sized VM and only pay $7 a month? This $7 goes to the static public IP and not for the VM and the network traffic from the various access points? I have a Unifi controller on AWS and it’s getting expensive. I am looking for cheaper alternatives like Digital Ocean, Linode, and GCP.

    Reply

    1. No, it won’t cost you $7. The IP address is not charged for when it is attached to a running VM – and the first micro sized VM in an U.S. region is free. So far I have paid for few dozen cents occasionally when I have tested changes.

      Reply

  72. Hi Petri,

    Thank you very much for this write-up. I’ve been running my controller like a dream for 2 months now. I had lots of stability issues running my controller on a Raspberry Pi and I love that I no longer have to worry about that!

    I’m interested in setting up a Pi-Hole DNS server on GCP. Could this cause issues with my Unifi controller if I set it up on the same VM instance as my Unifi controller?

    Here’s the guide I’m considering following: https://github.com/rajannpatel/Pi-Hole-PiVPN-on-Google-Compute-Engine-Free-Tier-with-Full-Tunnel-and-Split-Tunnel-OpenVPN-Configs

    Thanks!

    Reply

    1. I don’t see any reason why not. Just remove my /etc/lighttpd/conf-enabled/10-unifi-redirect.conf which will redirect all http requests to https:8443. Pi-Hole uses Lighttpd as one of its components. My Let’s Encrypt solution will disable Lighttpd for the duration of the certification renewal every three months. It usually takes about a dozen seconds. Can you live with that?

      You just need to merge the instructions from both sources. You also need to leave ports 80 and 443 open in the VPC if you want to use Let’s Encrypt certificates.

      Reply

    2. Petri – thanks so much for this!

      My contribution: some instructions to hook this up to stackdriver logging. The Unifi Controller does a decent job of parsing the logs, but if you also want to access them from the Cloud Console, you need to do a few things. The painful part that should save you some work – a regex fluentd format to ingest the Unifi server logs. The rest is pretty straightforward. I did this all manually at first, then created a few lines that could be inserted into your deployment script. This builds on another contributor’s work who added the StackDriver/GCP logging agent (thanks!!).

      Once this is done, I get the following sort of output in the Logs Viewer:

      jsonPayload: {
      message: “[event] Guest[fc:18:3c:3d:XX:XX] disconnected from “MyGuestWiFiSSID” (10s connected, 0.00 bytes, last AP[80:2a:a8:02:XX:XX])”
      operation: “inform-3226”
      labels: “event”
      }
      resource: {
      type: “gce_instance”
      labels: {
      zone: “us-west1-b”
      instance_id: “222###############”
      project_id: “unifi-controller-######”
      }
      }
      timestamp: “2020-07-01T03:11:32.607Z”
      severity: “INFO”

      I added the following lines just after the section that sets up the swap file.

      ###########################################################
      # Enable StackDriver Logging
      #
      #
      if [ ! -f /usr/lib/logging-agent/install-logging-agent.sh ]; then
      echo “Installing StackDriver agent”
      if [ ! -d /usr/lib/logging-agent ]; then
      mkdir -p /usr/lib/logging-agent
      echo “Created /usr/lib/logging-agent”
      fi
      curl -Lfs -o /usr/lib/logging-agent/install-logging-agent.sh https://dl.google.com/cloudagents/install-logging-agent.sh
      bash /usr/lib/logging-agent/install-logging-agent.sh
      echo “StackDriver agent installed”
      fi

      ###########################################################
      # Add fluentd config to ingest Unifi server logs
      #
      #
      #
      if [ ! -f /etc/google-fluentd/config.d/unifi-server.conf ]; then
      cat > /etc/google-fluentd/config.d/unifi-server.conf << _EOF

      type tail
      path /var/log/unifi/server.log
      pos_file /var/lib/google-fluentd/pos/unifi-server.pos
      tag unifi-server
      format /^\[(?[^\]]*)\] <(?[^>]*)> (?[^ ]*) *(?[^ ]*) *- (?.*)/
      time_format %FT%H:%M:%S,%N

      _EOF
      echo “Added Unifi-Server to Fluentd Configuration”
      systemctl restart google-fluentd
      fi

      Reply

      1. Thank you for this. It looks really good. I’ll need to test this for some time, but I will incorporate this into the script.

        Reply

      2. One correction – when I pasted the script, two tags didn’t come through – either the browser is grabbing something that looks like an HTML tag, or the comment tool is hiding the text. (Or another real possibility… user error 🙂 )

        In any case, two blank lines in the configuration file should not be blank. They should have a “source” and “end source” tag like this:

        .
        If the above two lines are blank, the same problem is happening.

        In any case, I made my fluentd configuration file publicly viewable here
        https://storage.cloud.google.com/gbray-unifi-scripts/unifi-server.conf

        Enjoy!

        Reply

  73. Great instructions and script, Petri!

    I wasn’t seeing a backup of the storage and put a cron job in /etc/cron.daily and before I realized it I copied a file as “backup”.

    Did I just kill my automated backup, and if so, can you give me the script so I can fix my error.

    Thanks

    Reply

    1. I don’t quite get what the problem is. If you are worried that you overwrote a backup script in cron.daily, you didn’t. My backup runs from systemd and is called unify-backup.service. If you really want to manually fix something then there are links to the script at the end of the post (above these comments). My suggestion is always to download a Unifi backup, delete the VM and create a fresh controller. It takes about 15 minutes. Troubleshooting will take hours.

      Reply

  74. Hi Petri,

    I just wanted to thank you for taking the time to create the script, instructions, and video on how to host the UniFi Controller on GCP. I followed your instructions and everything went as smooth as silk and it’s working great.

    Reply

  75. I rebooted mine to see if it would upgrade to 5.10.12 but no dice, does it require manual intervention, or should I take a backup, blow it away and spin up a new one ? and cheers, awesome script

    Reply

    1. The new version will hit the repository in about a week. It is just good news, since a few releases have been withdrawn and reverting back to a functional controller cannot be automated. Just wait for a few days, let the hotheads test drive it first… Oh, there’s no need to reboot, unattended-upgrades service will install the upgrade the night after it comes available.

      Reply

      2. Petri,
        How long should we wait to update to latest Unifi?
        Any idea if newest software is ok?
        Thanks

        Reply

        1. If you mean the controller version 5.10.12 then it usually appears about a week after it has been announced. Quite often it has appeared on a Monday, but that is not a rule. You don’t need to do anything, it will be installed automatically. If you have a session open then you will need to reauthenticate.

          Reply

  76. Hey, Petri!!
    thanks again for the guide!

    I ran this setup about 6 months ago and everything went great!
    but I tried to access my controller yesterday and my unifi account is offline.
    i try to access the controller from my custom URL you showed me how to create and it said the website is unavailable.
    any recomendations of what do do about this?
    If i go to the actual IP of the controller i can access it but not from the unifi controller.

    Reply

    1. Do you mean you can’t access it through unifi.ubnt.com ? That is meant as a backdoor to controllers behind a NAT firewall. It based on WebRTC remote desktop connections and is flaky at best. I struggled with that until I started using GCP and AWS based controllers. There is no need to go through the Ubiquiti service since you can always access your controller directly by its DNS name. I don’t even bother to set up the “Cloud Access” on controllers anymore. Once in a while Ubiquiti announces that they have fixed some problems and improved it, but I really don’t need it any more.

      If you mean you can’t access your controller by its DNS name but the numeric IP address works, then you need to fix your DNS name to resolve to the IP address.

      Reply

  77. Hi Petri,

    Thanks for creating this script. I had it all setup and running great. Today I connected to the cloud instance through SSH and made some updates to Java. I then issued a “sudo reboot” command and when the system rebooted it was no longer using the Let’s Encrypt certificate while it then at the same time also had upgraded to the new 5.10 firmware. Is this expected behavior and is there a different process I should follow?

    Thanks.

    Reply

    1. No, that is not expected. I have no idea what happened when you made manual updates. I wrote the setup so the user never needs to log on. Probably the Java certificate keystore has been overwritten. I would suggest to create a new VM and restore the latest backup at the time it was still working. You could try to run /etc/letsencrypt/renewal-hooks/deploy/unifi manually (as root) to see if it can fix the Java keystore.

      Reply

      1. Thanks for the suggestion. I tried the renewal-hook and got the error that it was unable to import certificates into the keystore. I will restore a backup into a new VM.

        Reply

        1. Try to log on. The Java keystore thing is a mystery to me. Occasionally it just works even though it just said it didn’t.

          Reply

  78. Hi Petri, thanks again for sharing this great job.

    I’m using your script since april 2018. Everyting PERFECT. Now, I’m stuck in 5.9.29.
    Everytime I login I see “Controller software update 5.10.17 is now available” in top left corner.

    startup.sh script is ejecuted at startup:
    Feb 15 21:57:57 unifi-controller startup-script: INFO startup-script-url: Return code 0.
    Feb 15 21:57:57 unifi-controller startup-script: INFO Finished running startup scripts.

    I tryed deleting “/usr/share/misc/apt-upgraded-1” and running “sudo ./startup.sh” but system is still in 5.9.29

    Content of “/etc/apt/apt.conf.d/51unattended-upgrades-unifi” is:
    Acquire::AllowReleaseInfoChanges “true”;
    Unattended-Upgrade::Origins-Pattern {
    “o=Debian,a=stable”;
    “c=ubiquiti”;
    };
    Unattended-Upgrade::Remove-Unused-Dependencies “true”;
    Unattended-Upgrade::Automatic-Reboot “true”;

    I don’t know where else to check if autoupdates is working… any clue? xD

    Thanks again!

    Reply

    1. If you want to see how Unattended Upgrades is working the logs are in /var/log/unattended-upgrades.

      I think I know what is the cause. When Ubiquiti changed their domain name to ui.com they also inadvertly moved their repository behind TLS, which is just silly. The packages are signed with PGP, there is no need for TLS. Standard APT doesn’t know how to download over https. I changed my script to install yet another package, but it only applied to new installations. Now the latest version will apply the package to all installations. You just need to Stop/Start the VM in GCP Console and it will upgrade overnight.

      Reply

      1. Thnaks Petri!
        I check /etc/systemd/system/timers.target.wants/apt-daily-upgrade.timer and it was also ok, That make me think autoupdate works right… I ran “apt show unifi” and discover 5.9.29 was avaible version.

        apt-get update displayed “E: The method driver /usr/lib/apt/methods/https could not be found.
        N: Is the package apt-transport-https installed?”

        I ran “apt-get install apt-transport-https” and now current unifi version is avaible.
        Next autoupdate will work fine.

        Thank you!

        Reply

  80. Hi,
    I’m having trouble getting the lets encrypt certificate. The VM is set up correctly, but it defaults to the self signed ubiquiti certificate (vice lets encrypt). I’ve deleted and rebuilt the VM as well as start and stop VM. Still no Lets Encrypt certificate. Any ideas?

    Reply

    1. There are two requirements: First, the dns-name metadata must globally resolve to the external IP of you VM. Let’s Encrypt will use that DNS name to resolve the IP from their datacenter wherever that is. Does the DNS name you provided resolve from your cell phone, work laptop, library PC etc? The second requirement is that the VM must be reachable from the aforementioned datacenter to ports 80 and 443. Quite often users forget to attach the correct network tag to their new VM.

      If you are Linux savvy, you can press on the SSH button in the GCP console and use command sudo more /var/log/letsencrypt/letsencrypt.log to see what the logs tell you.

      Reply

  81. Hmmmm,
    stat of /var/log/letsencrypt/letsencrypt.log failed: No such file or directory

    Reply

    1. That means that CertBot hasn’t run ever. Did you supply a dns-name when you created your VM? Does is resolve globally? Which DNS provider are you using?

      Reply

  82. User Operator Error (Doh!). I should have read more closely what you were telling me. I did not update my DDNS VM External IP address. It is now updated and resolves properly.

    However, if I type in my DDNS address (with no port entry) it does not connect. But if I type in my DDNS:8443, it does connect and has the correct SSL. Any ideas?

    Reply

  83. You are correct Sir. All is well. Thanks. Great write-up. This will be fun to use.

    Reply

    1. Petri,
      Many thanks for this write-up. Is it possible to run the lets encrypt script on a Linux based router? It would be cool to have my own Lets Encrypt Certificate for my house, that automatically renews. I’m a Noob with Linux, but can follow instructions well.

      Thoughts?

      Thanks

      Reply

      1. You don’t need the script. Just install CertBot and run it manually. Google for instructions on the EFF.org site. Do note, that the Let’s Encrypt servers need to reach the Linux box inside your home firewall. You need to set up port forwarding for ports 80 and 443 and a resolving DNS name. If you don’t have a static IP you need some kind of dynamic DNS to track the changing IP. All this is left as an exercise for the reader.

        Reply

  84. Hi there.

    I’ve gone through the setup twice now to make sure that I’m not missing anything, but something isn’t working. It seems to be partially working, because I get redirected to the https version of the url, etc. when I try to connect, but then it just spins for a long time and then never connects. I’m not sure what I’m doing wrong, and I don’t know how to troubleshoot what might be going wrong.

    Any thoughts on how I can troubleshoot why I can’t get the controller to come up in the browser? It just keeps telling me that my site took too long to respond.

    Thanks

    Nick

    Reply

    1. p.s. I noticed that even though I get redirected, there is no certificate set up either. And I control my own domain, so it’s not a DDNS issue.

      Reply

    2. It works for me 😁 I just tested by creating a new test controller. I do it once in a while since my script depends on so many other packages and services that change frequently.

      Your problem isn’t caused by Fail2Ban since you can connect to port 80 (to be redirected). My guess is it is the firewall rules. Do you have them set up correctly in the VPC Network section? What is the target tag you used there? Have you applied exactly the same network tag to the VM? (Not the rule name!) An error there would cause the symptoms you are experiencing. If you find the mistake, don’t try fix it, but create a new VM without the flaw.

      Reply

      1. Hi there.

        Yes, I did everything twice. I did everything exactly as you did, including the names. The tag is “unifi”, and that’s what I applied, and I coped the settings from this page. 🙂

        I guess I’ll try a third time and see what happens.

        Thanks. I really appreciate your reply. I hope I can figure it out.

        Reply

        1. I just checked, and everthing looks correct. I did however find this message when looking at the VM in the console, so it looks like something is causing the instance to work too hard and that must be why it’s timing out.

          “This instance is overutilized. Consider switching to the machine type: g1-small (1 vCPU, 1.7 GB memory). Learn more”

          But I’ll try again

          Reply

          1. Third time is the charm, isn’t it. Don’t worry about the warning, all my controllers show it. Consider it as the sales pitch at McDonald’s: “Do you want to super size your meal?”

            Reply

            1. Ok, Thanks.

              I tried a third time, but still no luck, I can’t figure out what I’m doing wrong, but it just keeps timing out. I triple checked everything, and I can’t find any mistakes. Maybe I’ll try another Region for everything.

              Reply

              1. No luck in a new region either. I feel like there is some kind of network issue, but for the life of me I can’t figure out what. 🙁

                Reply

                1. No, it is not the region. You keep doing some mistake and you are blind to it. I would suggest you get someone to do it for you. S/he doesn’t have to be a computer genius, just as long as s/he is not afraid of computers. Let her/him do it with the instructions above without your help. Many (most?) have succeeded, you just had bad luck.

                  Reply

                  1. Yeah, it must be something simple I’m missing. But I’ve done it 5 times now, and I check every single step.

                    Oh well. I’ll figure it out. Thanks.

                    Reply

        1. Also I get this:

          ~$ curl https://unifi.619spirits.com:8443/
          curl: (60) SSL certificate problem: unable to get local issuer certificate
          More details here: https://curl.haxx.se/docs/sslcerts.html
          curl performs SSL certificate verification by default, using a “bundle”
          of Certificate Authority (CA) public keys (CA certs). If the default
          bundle file isn’t adequate, you can specify an alternate file
          using the –cacert option.
          If this HTTPS server uses a certificate signed by a CA represented in
          the bundle, the certificate verification probably failed due to a
          problem with the certificate (it might be expired, or the name might
          not match the domain name in the URL).
          If you’d like to turn off curl’s verification of the certificate, use
          the -k (or –insecure) option.

          Reply

          1. I don’t know wether this will amuse you, but your controller appears to be working great. I can connect to the wizard page just fine. No certificate errors. It has a beautiful LE certificate valid until June 8th. Probably all your prior installations also worked just fine. There is something wrong with the OS/browser you are testing with. Try another computer.

            Reply

            1. Ugh! I think I just figured that out.

              I can get to it from my phone, but now from my Chromebook for some reason, which is where I do most of my work from.

              Thanks for your patience! Sorry to have bothered you for nothing.

              Reply

              1. Ok, now the plot thickens. So it turns out that I cannot get to the VM from inside my own network. But I can get to it from my phone, which is using cellular data, and from home, but not from my work network. Very strange!

                Sorry for all of the drama, but it’s clearly an issue with Cox internet here at the buisness.

                Reply

  85. Petri,

    I followed your procedure but now when i go to my domain it takes me to my local routers login screen, edgemax login screen
    How do i fix that?

    Reply

  86. Hello, Thanks for putting this guide together! I’ve had my controller running now for couple weeks and yesterday it stopped. I logged in to GCP and i have the recommendation to “increase perf, This instance has had high CPU and memory utilization recently. Consider switching to the machine type: g1-small (1 vCPU, 1.7 GB memory)

    What could be causing this error and how can i get my controller running again?

    Couple of days ago, i tried to download a setting snapshot from the controller and it wasn’t working… then i realized it was probably downloading the file to my instance. I tried about 4 times before realizing my mistake. Could that be the issue and if so, where can i go to to delete the files to free up space?

    Thanks for any advice!

    Reply

    1. What do you mean stopped? Do you mean it is in stopped state in the GCE console? In that case you can start it from there. I’ve never seen a controller to stop itself in that way, though.
      If you mean the controller is not responding, then the only supported remedy is to stop and start the VM from the GCE console.
      Have you had the browser window open all the time? In that case the browser may have been upset by the upgrade earlier this week (to 5.10.19). Close the browser and open it again.

      The suggestion to upgrade is reasonable. The micro VM is sluggish, but it is free. Google is only loosing money as long as you keep using it.

      If you downloaded the backup from Settings > Maintenance > Backup > Download Backup then it should be downloaded to your workstation, not left on the server.

      Reply

      1. Thank you!!!! a Start/Stop got it working again. I was worried the new updates caused to the controller to become too large/processor intensive for the free tier.

        Thanks again for the quick response!

        Reply

    1. I’m sorry but my script doesn’t support this. You can use my script as a starting point, though. After the script finishes you can log in via SSH (there is a button in the console). You can use sudo apt-get remove unifi to remove the installed UniFi package and install the one you fancy. Remove line "c=ubiquiti"; from /etc/apt/apt.conf.d/51unattended-upgrades-unifi or your installation will be automatically upgraded the following night. You need to keep your UniFi package updated manually.

      Reply

        1. sudo nano /etc/apt/apt.conf.d/51unattended-upgrades-unifi
          You need to use sudo for installs, uninstalls and configuration edits.

          Reply

  88. First of all thanks for writing this guide and setting up the video. It was very easy to follow and setup and I have been running this smoothly except for one exception which I will point out since December.

    Now for the exception: It appears that the dynamic DNS piece only runs when the server restarts (or is first started). Can you update it so that it automatically runs every so often? If this is already the case could it be that dynoDNS resets the DNS values BEFORE the VM does?

    Reply

    1. The DDNS service should keep the same IP address until reset. The VM will keep its IP address forever if you have set up a static IP. Even if you are using a non-static IP it will remain until the next boot. That’s why my script only updates it at the boot. All other scripts I have looked at (for home servers etc.) monitor the external IP address and reset the DDNS service only if the external IP changes. In most cases it will be never or a very long time.

      I am curious: What IP does your DDNS service reset to?

      Reply

  89. I have 3 custom domains hosted on no-ip.com pointing to a local NAS web server. I have those configured in Unifi to update via all.dnsomatic.com.

    I also have a subdomain we’ll call it unifi.domain.com I have configured in the GCP VM. So I guess what might be happening is the Unifi update to all.dnsomatic.com is also updating the controller’s DNS entry and making it my local ISP assigned IP rather than the one hosted on GCP.

    I guess I answered my own question haha. In order to avoid this behavior I would probably need to specify each domain separately in the Unifi configuration so that it does not blanket update them all via the Unifi controller if that makes sense.

    Reply

  90. Petri,
    I have used your setup/configuration since last year October 2018. It has been running great! It is at 5.10.19_11646 as well.

    I always use GCP console and click SSH to the VM to check things around.

    My fail2ban.log is big and it seems a lot of try out there on the internet (almost every few mins I see an IP tried to SSH with an invalid ID) .

    Do you know which setting I need to disable this?

    Please advise,
    Martin.

    Reply

    1. Disable what? Fail2Ban? Logging? Hackers?
      By default GCP has disabled password logins altogether so you don’t need to worry about scripts trying different passwords. Every time you click on the SSH button a new key is generated and the key is valid for the duration of the session.
      You may regard the login attempts as burglars trying which car doors are open. There is no way you can prevent it. However, as long as you keep your car doors locked there is no harm done if someone tries the handle.
      If you want to rotate Fail2Ban logs more frequently edit /etc/logrotate.d/fail2ban

      Reply

      1. Petri,
        I am trying to see if Port 22 can be blocked facing the internet since inform don’t use that port to monitor the unifi system. That way less I/O being used with those hackers trying on a minute basis.

        Reply

        1. Don’t block port 22 or you won’t be able to use the SSH button! I don’t know if Google has published the address ranges used by the Console. You could allow only those and hope they won’t be adding more addresses. If they do and your future Console is on one you won’t be able to log on.

          Reply

  91. Thank you Petri for the write out! Just set up the controller on GCP. Everything is running flawlessly.

    For people use Cloudflare to manage their domain, you may encounter “STUN communication failed” alert, remove proxy and use DNS only would help solving the problem. I spent a full hour to figure this out…

    Reply

    1. Yeah, I would expect reverse proxying causing trouble for the Controller. Thanks for reporting this. Hopefully someone will avoid this trap.

      Reply

  92. When I log into the controller, i get a notification that 5.10.20 is available.
    Isnt the controller suppose to auto-update to latest version?

    Reply

    1. The autoupdate runs at night after the publication. In the worst case you need to wait 23h59m for the update. Mine is already upgraded.

      Reply

  93. Hi,
    Thank you so much for this! Did it with my personal UniFi two days ago and am doing it for our company UniFi today.

    Everything worked perfectly at home where I have Cloudflare DNS and just added an A-record unifi.xxx.xx that pointed to the public IP of the VM.

    Here at our company we don’t use Cloudflare – but I again just added an A-record unifi.yyy.yy that pointed to the public IP of the VM.

    But Let’s Encrypt doesn’t work! When looking at the logs I get:
    simon@unifi:~$ sudo more /var/log/letsencrypt/letsencrypt.log
    2019-03-25 14:43:16,545:DEBUG:certbot.main:certbot version: 0.28.0
    2019-03-25 14:43:16,546:DEBUG:certbot.main:Arguments: [‘-q’]
    2019-03-25 14:43:16,547:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryP
    oint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
    2019-03-25 14:43:16,557:DEBUG:certbot.log:Root logging level set at 30
    2019-03-25 14:43:16,558:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
    2019-03-25 14:43:16,560:DEBUG:certbot.renewal:no renewal failures

    Any idea what’s wrong?

    Reply

    1. Does the DNS name resolve globally (instead of just in the company internal network)? Try nslookup of dig from your home.
      That log looks innocent enough, no errors. It looks like Let’s encrypt is happy. Are there certs in /etc/letsencrypt/your.domain.dns/live ?

      Reply

      1. Doing an nslookup looks fine (it says “Non-authoritative answer: Name: unifi.xxx.xx Address: IP-ADDRESS.

        In /etc/letsencrypt there is only a file and a folder:
        file: cli.ini
        folder: renewal-hooks

        Reply

        1. What happens if you type the DNS name into your browser? Do you get the UniFi Controller after a certificate warning?
          For some reason the script didn’t get a certificate. If the external IP doesn’t match the DNS name then it won’t even try. Let’s Encrypt will only provide five certificates per week for a given domain. This could be an issue if you use some popular DDNS domain or if you keep creating VMs for some reason.
          I would suggest to delete the VM and create a new one (unless you are going over the limit of five).

          Reply

          1. Yes, when typing the domain in a browser it continues to the UniFi controller after the certificate warning.
            I created an A record on my DNS server – is that the correct way of doing it?
            I will try to delete and create a new one later. Thanks!

            Reply

            1. Is your VM behind a reverse proxy, NAT or such thing? The VM compares its external IP address to the resolved DNS name IP. If they don’t match (i.e. the DNS name doesn’t point directly to the external IP address) it won’t even try. You can delete this test from /usr/local/sbin/certbotrun.sh if necessary.

              Reply

  94. Hi Petri

    Thanks a lot for your efforts, it works like a charm!

    Have you considered a supplementary tool to install UniFi video as well? – or expand the current script with a flag to include the UniFi video bits…

    Again, thanks for sharing this.

    /Martin

    Reply

    1. No, I don’t have videocams so I wouldn’t be using it myself. I wouldn’t notice problems or have any means of testing.

      You can do it, though! Feel free to fork off a branch on GitHub and add some functionality. You’ll learn a bunch and make new friends.

      Or you can just log on to your VM and install the video controller manually. If my memory serves me right you need to edit the firewall rules a bit. You can even uninstall unifi package if you just want to use the base system as a starting point.

      Reply

  95. Hi Petri,
    Everything working great except I am getting the STUN triangle warning on the device page. I know I have to port fwd 3478. However, what device do I fwd port 3478 to?
    I tried to fwd 3478 (from my home router) to my switch on my LAN 192.168.XXX.XXX. However, That doesn’t do the trick. I also tried to fwd the port using my ddns address, no Joy. Also tried to fwd 3478 using the IP of my Google VM. Still no joy. Any ideas?

    Thanks

    Reply

    1. No, you don’t need any port forwarding. All your devices should have the GCE VM as their inform address so they all report directly there. You can check the current settings by SSHing to the IP of one of your APs and issuing command info. Look for the (last) line “status”.

      Reply

        1. There are a few explanations:
          – Do you have double-NAT? This could happen if you have an USG and your modem is routing as well. This can be remedied by configuring the modem to bridge instead of routing.
          – Is port udp:3478 certainly open in your VPC firewall? (Not tcp:3478)

          I have seen this occasionally, but it has been transient. Fortunately most of the functionality doesn’t require STUN. You’ll miss opening terminal sessions from the controller and the ability to blink the LEDs (locate function) from the controller. See Troubleshooting STUN Communication Errors

          You may want to verify the settings by SSHing to an AP and using command more /etc/persistent/cfg/mgmt. Check inform URL mgmt.servers.1.url and stun url.

          Reply

          1. Hi Petri,
            here’s what the SSH output looks like:
            mgmt.servers.1.url=http://mywebsite.duckdns.org:8080/inform
            stun_url=stun://mywebsite.duckdsns.org/

            Any ideas?

            Reply

            1. That is what I expected. I’ve never seen a stun url mismatch. But there may come the first time.

              Have you double checked the GCP VPC firewall rules? Do you have a routing modem? Is there any chance of double-NAT? Is your modem bridged? Some ISPs do NAT for their clients. It is known as carrier-grade NAT (CGN) or large scale NAT (LSN).

              This problem is specific to your GCE VM or your network. I can’t reproduce it and there are no other reports. There are limits to what I can do from here.

              Reply

              1. Hmmmm,
                GCP VPC firewall rules allow UDP 3478. I have an Arris DOCSIS 3.1 Modem connected to the Cox domain. My modem is connected to an ac router. Not sure what gives.

                Reply

                1. Is your modem bridging or routing? Does you ISP use CGN? My bet is on double-NAT somewhere along the line. But you can just ignore it and live with it.

                  Reply

          2. Hey Petri and Frank,

            Were you able to fix your STUN issue?

            I am having the same issue with my controller hosted on Google Cloud.

            Here is what I have tried so far:

            1. Changed STUN Port in controller in GCP ( but I changed it back) and uncommented ports per this article on page 14.
            https://community.ui.com/questions/STUN-Communication-Failed/226fb3e1-26ac-43d0-b265-714d9e45f161?page=14
            portal.http.port=8880
            portal.https.port=8843
            unifi.db.port=27117
            unifi.http.port=8080
            unifi.https.port=8443
            unifi.stun.port=3478
            unifi.throughput.port=6789
            2. Then I tried to enable dhcp option 43 on my router by following the instructions here https://serversideup.net/automatic-controller-assignment-unifi-dhcp-option-43-mikrotik-routers/ to convert

            Convert the controller’s public IP to HEX and add 0104 to the front and putting the below in the lan interface settings of my router running openwrt.
            list dhcp_option ‘43,01:04:22:79:5A:2A’
            3. Controller version 6.5.54.0 one of the UAP-AC_LR firmware version: 5.43.51.12773

            4. truncated Output of cat /etc/persistent/cfg/mgmt:
            mgmt.servers.1.url=http://373houseap.club:8080/inform
            stun_url=stun://373houseap.club/
            mgmt_url=https://373houseap.club:8443/manage/site/default

            5. Verified that egress and ingress for 3478 is allowed in firewall rules in GCP
            6. Debian version is 9.13 , stretch

            Not sure if I should set DHCP option 15 in my router.

            Do you have any other troubleshooting suggestions?

            Reply

  96. is ther any way to set up a vpn for internet or other stuff on the same vm since i doint want to pay for a anotehr vm as unfi controlor

    Reply

    1. Yes, you can certainly set up the VM as you like. Note that the micro-sized VM is struggling as a UniFi controller already, as you can see from the sluggish performance. You can add some lightweight service or some intermittent loads as you see fit. If you need a bigger VM you will need to pay for it anyway.

      Reply

      1. Petri,
        The micro-sized VM seems to work fine for me. Do you think with future Unifi upgrades, the micro-sized VM will no longer adequately support the controller?

        Reply

        1. Ubiquiti has sold so many original Cloud Keys they can’t let the controller grow bigger than that. The micro VM is performance wise close to the CK (but the disk is better).

          Reply

  97. Hi Petri, thanks much for the guide, worked really nice for me. I had a issues with the script because i copied the url but didn’t see the blank space “gs: //”, maybe it’s a good idea delete the blank space this for avoid future problems for people who’s follow your guide?

    Reply

    1. I couldn’t spot anywhere a space after the gs: Where did you find it?

      Reply

      1. Google traductor generate that blank space, i used that for translate this website (to spanish). I just realized, I’m sorry for the misunderstanding. Regards

        Reply

  98. Hi, It worked like a charm- great work, however i skipped the dynamic dns part and did not enter those two lines of key values of dns name and dns url when setting it up , thinking I can take care of it after it’s all setup. Now the question is there a way to do after having the controller setup and up and running or do i have to delete the VM and start again?

    Reply

    1. You can change the metadata anytime. Then just stop/start the VM to run the startup-script.

      Reply

  99. Hi Petri,

    Great work, thanks for this. ‘Step 3. Set Up the Virtual Network’ does not currently include tcp 22, is that an error?

    Reply

    1. The controller is safer if you don’t allow SSH from the Internet. I wrote my script for people who don’t want to type Linux shell commands.
      You can access the command line from the GCP Console. If you want to allow SSH access from the outside you also need to set up a permanent account on the VM (and allow password auth if desired).

      Reply

      1. Okay thanks, was reading some of the comments and saw a reference to port 22 so I was asking.

        Reply

  100. I setup everything like you show on your video and guess what :

    IT’S WORKING PERFECTLY

    A big thanks you from Montreal…

    Robin

    Reply

  101. Petri,
    for some reason my controller is stuck on “UniFi Controller is starting up…please wait a moment” I restarted the VM and it’s still stuck on this message. Any idea whats wrong?

    Reply

    2. Hard to say from here. The fastest solution is usually to deploy a new VM with the same IP and restore the latest backup from the bucket. If there is something wrong with the VM it will usually give more trouble as time passes.

      Reply

        1. It depends. If you are recovering from an operator error then go to Settings > Maintenance > Restore. If you are recovering from a system error then it is best to create a new VM. On the first page of the setup wizard there is an option to restore a backup. There is no point in trying to resurrect a broken controller when you can create a new one in 10 minutes flat. Download the latest backup from Google Cloud Console at Storage > Browser.

          Reply

    3. Hi all I had this same issue and just started and stopped the VM and it now works again. I noticed a message on the google console about that there was high cpu usage detected though. @Metis any ideas what could cause this?

      Reply

      1. Google wants to upsell you a bigger VM. It could be that the CPU spikes for some database compression tasks, but I haven’t seen it ever. Usually it sits around 10-15%.

        Reply

  102. Just want to leave a thank you for the great guide. I set mine up earlier this week and it’s working flawlessly!

    Reply

  103. Hi Petri,

    Google recently sent out an email about the JSON API called “New endpoint for the Google Cloud Storage JSON API”. In it the following is stated:

    ——
    What do I need to do?
    If your production or test code doesn’t check for endpoint-specific details, no action is required on your part.

    If your production or test code checks for endpoint-specific details, you will need to modify them before June 20, 2019 as follows:

    If your code checks that the ‘baseUrl’ or ‘rootUrl’ fields in the JSON API Discovery document point to http://www.googleapis.com, you will need to modify those checks to allow either storage.googleapis.com or http://www.googleapis.com. Note that the oauth2 scopes fields in the Discovery document will not change and will continue to point to http://www.googleapis.com.
    If your code checks that the ‘selfLink’ field in bucket or object metadata points to http://www.googleapis.com, you will need to modify that check to allow either storage.googleapis.com or http://www.googleapis.com.
    If you access Cloud Storage through a firewall, you will need to ensure that requests to storage.googleapis.com are allowed by your firewall rules.
    ——

    Is there something that needs to be done. Can you comment? Thanks for your great script and feedback!

    Reply

    1. No action is needed. The change is at a very low level and my script uses a high level command written by Google. If the high level command needs some changes then Google will take care of it.

      Reply

  104. Is it possible to change dns to the VM without reinstalling it?
    It seems the free .my.to has gone down and I would like to switch to another one.

    kind regards

    Reply

    1. Yes. The metadata is applied at boot, so change the metadata and stop/start the VM from the GCP console.

      Reply

  105. Many thanks Petri, I have been using the GCP hosted UniFi controller for a few weeks now by using your script (previous controller was on AWS).

    Up until very recently I was using a subdomain from afraid.org but I’ve since created a subdomain from my own hosted domain, altered the metadata field to reflect the changes & all is working perfectly upon starting the VM again, LE cert is valid etc etc.

    Very happy with this controller solution made super easy by you 🙂

    Reply

  106. Hi Petri,

    Thank you again for a great tutorial.
    My controller upgraded last night to 5.10.24 and now I’m getting flood by this error on two of my AP: “Message: AP[xx:xx:xx:xx:xx:xx] /usr/sbin/dnsmasq exited with code 512 and restarted by inittab”

    Can I restore to previous backup from within the controller interface? settings:maintenance:retore:choose file?
    Or do I need to do the whole installation all over again?
    Also how can I disable the automatic update of the controller?

    Thank you very much.

    Reply

    1. Hmm, mine is running 5.10.24 just fine, but YMMV of course. There is no easy way to downgrade. My suggestion is to create a new VM and restore the latest backup there. The new install will be 5.10.24 as well.
      If you really need to go back you’ll need to login via SSH in the GCP console. Uninstall by sudo apt-get remove unifi and then install your favorite edition manually. To prevent the automatic upgrade next night you’ll need to edit or remove /etc/apt/apt.conf.d/51unattended-upgrades-unifi.

      Reply

      1. Thank you Petri. I think I’ll wait for the next update to resolve this. Actually the controller is giving warnings and not errors. But the warnings are generated every few seconds. I have switched out the option to send this warnings by email otherwise my inbox would not survive.
        I’m not the only one though, there are a couple of threads on the internet about this in different forums.
        Is there a line I can comment out in 51unattended-upgrades-unifi so it won’t upgrade and I can just un-comment the line when I want to upgrade?
        I’m sorry I’m not much of a coder

        Reply

        1. The line with “c=ubiquiti” is the one that keeps UniFi controller up to date.

          Reply

  107. Petri, a question for you sir. I love how everything is working, however if I have started to outgrow the free tier from google and wanted to say upgrade the VM, would it be a simple case of increasing the ram / cpu ? would any changes have to be made to your script with regards to its swap file usage etc ? I have started to add more and more devices and I worry the spec might being to have issue coping ?

    Reply

    1. Yes, it will work just fine. If you give your VM 2GB or more memory the script won’t create a swap file. That is the only difference the VM size will cause.

      Reply

      1. fantastic! have you seen any limits with the micro VM in terms of clients / adopted devices ? when you start to notice performance related issues ? just want to make sure thats whats happening, I see some devices missing heartbeats and coming straight back, with no change in uptime

        Reply

        1. I don’t know effect of load or the load components for the controller. It does get sluggish, it will lock up occasionally during backups and database compression, miss some heartbeats, but nothing really serious. I’ve recommended the same 30 device limit that Ubiquiti gives for the first gen Cloud Keys. The performance is similar so I’d expect similar capacity.
          Ubiquiti has published some papers on high volume controllers. You may want to read those. They have factored the device count, captive portal, different authentication schemes and so forth.

          Reply

          1. Just seeing this now. I wasn’t aware of the 30 device limit. I have 43 devices (20 are wireless dimmers) and everything seems to be working fine. Am I asking for trouble with this many devices?

            Reply

            1. No, you are good. The devices only send their status once a minute. In your case more than a second apart on the average. You may miss a few once a day when the backups are run and some database compression occurs. If you run a captive portal with plenty of users then it is another matter altogether. See How to Tune the Controller for High Number of UniFi Devices for more info.

              Reply

  108. Hi there! First of all thank you for taking the time to write this up.
    I’ve got everything up and running smoothly, or so it appears. The only quirk is when trying to test fail2ban. I used a VPN and I was able to enter a fake username/password into the login screen numerous times without triggering an IP ban.

    Is there a way I can check if fail2ban is working? I’m a bit novice with linux/ssh but I can see that there is a unifi-controller.conf in the fail2ban/jail.d directory, but not sure if the jail is active or not.

    Also it does not appear that the backups are going to the storage folder at all. How can I check the backups are being stored correctly?

    Any help would be appreciated, I’d like to avoid re-doing the VM but if I have to, do I just need to backup my existing config from within the controller and then reload it on a fresh VM? Will my devices automatically be picked up?

    Thank you in advance.

    Reply

    1. Also, I can see in the fail2ban the logs are showing “Found: ##IP##” but the ban does not trigger.

      Reply

      1. The easiest way to test is to try to log in with wrong credentials three times. After that you’ll still get the login page (it comes from your browser’s cache) but the log in just hangs. Normally you’d get a failure message. After an hour it will open again. I haven’t tested this lately. I follow my logs and I haven’t seen any scripts trying to log in to my controllers.

        Reply

      1. I am having the same trouble with backups not completing on their own. I browse the bucket and there was nothing there. I logged into the controller box on GCP and ran the command I found in the service file. It failed when I was an unprivileged user but if I sodo it works from inside the box:

        sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://my_bucket_name

        The logs do show it initiating near the 1am time:

        ug 4 01:18:47 unifi-controller-main systemd[1]: Starting Daily backup to org_tom_unifi_backups service…
        Aug 4 01:18:53 unifi-controller-main gsutil[10601]: Building synchronization state…
        Aug 4 01:18:55 unifi-controller-main gsutil[10601]: Starting synchronization…
        Aug 4 01:18:55 unifi-controller-main systemd[1]: Started Daily backup to org_tom_unifi_backups service.
        Aug 4 01:18:55 unifi-controller-main systemd[1]: unifi-backup.timer: Adding 10min 16.622564s random time.
        Aug 4 01:18:55 unifi-controller-main systemd[1]: unifi-backup.timer: Adding 18min 29.025547s random time.

        I am unsure how to make this work. It is one of the most important things for me as the thing that drew me to run this in the cloud was consequences I had from a power outage at home. All of the sudden, I care *a lot* about backups. 🙂

        Thanks in advance!

        Reply

        1. Systemd runs the units as root.
          Apparently you have found the unit files in /etc/systemd/system and they are OK. Have you checked /var/lib/unifi/backup/autobackup that the backups are created?
          I can’t tell.. Are you certain there is no autobackup folder in the storage bucket? The periodic backups are inside.
          (Yes, backups are golden!)

          Reply

  109. Petri-
    Installed this last year and was working great until today, something went belly up, couldn’t access the software by IP or DNS. Decided to delete the VM and start over, same issue, Safari can’t open the page “https://[GCP-IP]:8443” because Safari can’t establish a secure connection to the server “[GCP=IP]”

    I’ve checked the serial logs and looks like Unifi is up on the VM. Anything else I can look at directly? Appears that Lets Encrypt did its thing correctly from the logs too.

    unifi-controller login: Jun 16 18:07:07 unifi-controller unifi.init[677]: Starting Ubiquiti UniFi Controller: unifi.
    Jun 16 18:07:07 unifi-controller systemd[1]: Started unifi.
    Jun 16 18:07:19 unifi-controller startup-script: INFO startup-script-url: OKDynamic DNS accessed

    Reply

    1. That sounds like a certificate issue. Try accessing your controller with Chrome. Chrome lets you bypass the bad certificate warning. Then you can click on the lock icon in the address bar to see what’s wrong with the certificate.
      The .my.to domains I used in the video are not available any longer. The owner decided to withdraw them. It could be your computer has kept the IP address in memory, but Let’s Encrypt renewal process can’t work.

      Reply

      1. I tried Chrome, ERR_CONNECTION_RESET error. I just created a new VM, again. Here are the fresh logs, does seem cert related. I am using DuckDNS

        Jun 16 18:43:33 unifi-controller startup-script: INFO startup-script-url: Saving debug log to /var/log/letsencrypt/letsencrypt.log
        Jun 16 18:43:36 unifi-controller startup-script: INFO startup-script-url: Plugins selected: Authenticator standalone, Installer None
        Jun 16 18:43:36 unifi-controller startup-script: INFO startup-script-url: Registering without email!
        Jun 16 18:43:37 unifi-controller startup-script: INFO startup-script-url: Obtaining a new certificate
        Jun 16 18:43:38 unifi-controller startup-script: INFO startup-script-url: Performing the following challenges:
        Jun 16 18:43:38 unifi-controller startup-script: INFO startup-script-url: http-01 challenge for [redacted].duckdns.org
        Jun 16 18:43:38 unifi-controller startup-script: INFO startup-script-url: Waiting for verification…
        Jun 16 18:43:41 unifi-controller startup-script: INFO startup-script-url: Cleaning up challenges
        Jun 16 18:43:45 unifi-controller systemd[1]: Starting Lighttpd Daemon…
        Jun 16 18:43:45 unifi-controller systemd[1]: Started Lighttpd Daemon.
        Jun 16 18:43:45 unifi-controller startup-script: INFO startup-script-url: unable to write ‘random state’
        Jun 16 18:43:47 unifi-controller startup-script: INFO startup-script-url: Importing keystore /tmp/tmp.fk0W8Gzfsk to /var/lib/unifi/keystore…
        Jun 16 18:43:47 unifi-controller startup-script: INFO startup-script-url: Warning:
        Jun 16 18:43:47 unifi-controller startup-script: INFO startup-script-url: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using “keytool -importkeystore -srckeystore /var/lib/unifi/keystore -destkeystore /var/lib/unifi/keystore -deststoretype pkcs12”.
        Jun 16 18:43:47 unifi-controller systemd[1]: Stopping unifi…
        Jun 16 18:43:52 unifi-controller unifi.init[26905]: Stopping Ubiquiti UniFi Controller: unifi.
        Jun 16 18:43:52 unifi-controller systemd[1]: Stopped unifi.
        Jun 16 18:43:57 unifi-controller startup-script: INFO startup-script-url: Unable to import the certificate into keystore

        Reply

      2. I just created a completely new GCP project, new IP, and tried using afraid DNS, same exact issue. Unable to import cert into keystore. Any ideas?

        Reply

        1. The “unable to import cert to keystone” is expected. It still works. I haven’t found out what causes the error. UniFi controller is using really old Java mechanisms, which could cause it. The ERR_CONNECTION_RESET is a different matter. It has nothing to do with the certificates.
          I’ll spin up a new test instance tomorrow. It is getting late over here.

          Reply

          1. Ok, this is sort of figured out but not confirmed. I plugged in a new AP and it seems this is when GCP decided to stop hosting me. The new AP I plugged in, seems to have some Cat that is not punched down correctly and maybe created a feedback loop that GCP then disabled me, but I wasn’t seeing that anywhere. The only other possibility would be a duplicate IP, but it was a brand new device, so double that. Rebuilding my network last night using local software, I found the wiring issue. Unplugged the device and today my GCP instance is working. Sorry for the spam on this. Now I just need to find out how something was wired wrong in my house.

            Reply

            1. I’m glad you found the culprit. I couldn’t reproduce the problem.

              Reply

  110. Hi and thank you for all the work on this setup. I’ve had everything up and running for over 7 months and no issue. I have noticed in the past month that when I go to connect I get the following:

    “UniFi Controller is starting up…”

    I can stop the VM and check the logs and don’t see any reason why this it at a start up area. I can stop the VM and start it backup and it will work for about a week or so and stop again. This is just for my personal use with 3 AP and switch. Any help is greatly helpful

    Reply

    1. No idea, I haven’t seen it or received any other reports. I suggest you create a new VM, assign the same IP address to it and restore the latest backup. Report back if the problem still persists.

      Reply

  111. Any reason why i keep getting permission denied when i try to access /usr/lib/unifi/data .

    Reply

    1. When you connect by SSH through the GCP Console you are not root. You need to use sudo for elevated privileges. If you need a root shell use sudo su -

      Reply

      1. Thanks ! Now i have one more issue. It seems when I enable GuestMode that creates the portal it re-directs the guest machhine to the IP address of the controller and not the DNS hostname. I checked the cert and i see its valid only for the dns hostname and not the public ip address. I have tried to updated the controller settings to use the DNS hostname and checked the box to overide but sitll didnt help

        Reply

        1. Over here we don’t tend to use captive portals. They provide little value and are expensive to maintain (like you just learned). I don’t have much experience with them – I’ve played with them, but that’s not real life experience.
          No, you can’t get a certificate for a bare IP address.
          I checked and there is a “Redirect using hostname” option in Settings > Guest Control > Guest Policies. Is that what you have tried? If that doesn’t work I’d ask on Ubiquiti forums. There is plenty of expertise there. (Ubiquiti just messed up their forum platform, but that’s an another matter)

          Reply

  112. Hi!
    I got an email from google regarding a change they will do in September.
    Is this something that will affect the controller setup?

    Text from email:
    ”
    We’re writing to let you know about a change to the behavior of HTTP(S) Load Balancers that is being rolled out gradually, beginning September 30, 2019. We are making this change to standardize HTTP(S) load balancer behavior across HTTP/1.1 and more modern protocols, including HTTP/2 and QUIC.

    What do I need to know?
    After September 30, HTTP(S) Load Balancers will convert HTTP/1.1 header names to lowercase in the request and response directions; header values will not be affected.

    As header names are case-insensitive, this change will not affect clients and servers that follow the HTTP/1.1 specification (including all popular web browsers and open source servers). Similarly, as HTTP/2 and QUIC protocols already require lowercase header names, traffic arriving at load balancers over these protocols will not be affected. However, we recommend testing projects that use custom clients or servers prior to the rollout to ensure minimal impact.
    ”

    Regards,
    Mats

    Reply

    1. Have you set up a load balancer in front of you controller? In theory you could run multiple controllers if you run a separate MongoDB instance and configure the controllers to use it instead of a local one. That is a pretty complicated setup. My script just sets up a single controller without a load balancer so you don’t need to worry about this change.

      Reply

      1. Hi Petri,
        How many sites can I connect to a single Controller? I have my main house, a second house, and my work all with Ubiquiti equipment. A total of 12 Access points at all three locations. (three separate sites: Home 4 Ap’s, Second House 2 Ap’s, Work 6 Ap’s) I currently run my main house and the second home on the same controller (2 sites). Can the GCP handle all three sites with one controller?

        Reply

        1. AFAIK the number of sites doesn’t matter, it is only the number of devices. Ubiquiti says the first CloudKey supports up to 30 devices. The free tier VM is of similar capacity, so 12 devices should fit well.

          Reply

  113. Thank you for the answer!

    I have just followed your setup and have one controller.
    In the mail from google it says:
    “Your project(s) listed below have used HTTP(S) Load Balancing in the past 60 days and may be affected by this change:”

    So I have used load balancing… but don’t know why.

    Reply

    1. Hmm.. I don’t know. But anyways, the change is only in the character case of the headers. I can’t imagine how it could affect the controller.

      Reply

  114. Many thanks for this, up and running quickly and all looks good. But looks like the free GCP will end in 12 months?

    Reply

    1. Your free $300 credit will expire after 12 months. One micro sized VM and 5GB of bucket storage are perpetually free AFAIK. See GCP Free tier and please report if you find some changes to this.

      Reply

      1. Thanks Petri. All was working fine this morning at 6:30am (UK). But by 7am all 3 of my APs were disconnected. Using PuTTY I cannot access the APs as the password is not accepted (standard), but then at about 8am all three APs came back online (but still unable to access via PuTTY).

        I wonder if security has been compromised and someone has accessed the APs?

        Tony

        Reply

        1. Some hiccup somewhere, but a deliberate break-in wouldn’t be at the top of my list of suspects. You probably have a firewall (USG maybe) which denies connection attempts from the Internet. UniFi controller is passive. The devices connect to the controller to report activity and get the latest configuration. The controller can’t connect to the devices because of the firewall. The attack vector would be to break into the controller, change the configuration for the devices and wait for the devices to pick up the malicious config. There is very little to gain so I haven’t heard of such an attack.
          To SSH into the devices you need to use the credentials in Settings > Site > Device Authentication.

          Reply

          1. Thanks Petri, as you say a hiccup. I’ve applied new AP credentials via Device Authentication, and all looking good..

            Reply

  115. Petri, I am noticing with my controller that it periodically goes offline so to speak. I get websocket errors and then my browser can’t connect to it. after about 5/10mins it comes back and all is fine, this happens a few times a day, not when backups are happening according to the schedule. do you think this is database compact / maintenance and the VM is a temp overloaded a little ? I haven’t checked in GCP console when its happening, but not sure what to look for to tie it down. I have exactly 30 devices reporting into the controller. do you think this would be signs that I am reaching the limit. no captive portal or anything running just basic stuff

    Reply

    1. It could be. The 30 device limit is just my hunch based on the 1st gen Cloud Key performance. Do you see any spikes in the GCP CE VM Details Monitoring graphs? There are graphs for CPU, network and disk utilization. If you see simultaneous spikes on several graphs you could try to figure what’s going on.
      Do the graphs in UniFi Controller have breaks at these times? That would mean that the devices can’t reach the controller either.
      Another guess could be that the VM hosts are starved because Google is oversubscribing them for non-paying customers. That wouldn’t show up as your CPU utilization. You could run a next bigger instance for a few days (it won’t cost more than a dollar) and see if the problem goes away.
      Please report back if you discover something.

      Reply

      2. noticed a significant performance increase when I upped the VM power, obviously to be expected, but lots of small issues with the controller have gone away. I would suspect the device limit should be around 15 almost. I noticed issues a while back, but did not actually count my devices until now, increasing them over time.

        I would have adoption errors with devices, they would fail the adopt and upgrade as I expect that puts more strain on the controller. some devices would go offline until their switch port was PoE cycled etc, then I would have the issues I described earlier with the web interface going offline. I checked the logs and cant see any gaps in the graphs so I think the controller was staying online but something was going with the Web UI. I changed the type to n1-standard-1 (1 vCPU, 3.75 GB memory), I still have the full allotment of credits from the account creation so its eating into that a little, I will run it for a week or so and see how I get on.

        Thanks for your time Petri

        Reply

  116. Hi Petri.

    I have an email billing alert that I have exceeded 50% of the monthly budget.. Looking at the GCP interface I see I am in Zone us-east4-c, so I’m not sure this is one of the freebee zones? There is an overutilized warning and a recommendation to switch to a g1-small.

    Your advice appreciated..

    Tony

    Reply

    1. If I have understood correctly all the U.S. zones are included (except for one that is for government use only). The over utilization warning is normal. You should look into your billing statement to see what is incurring charges.

      Reply

  117. I wonder if you can help Petri?I turned off both of my WiFi networks to get my son off his Xbox. I come back to my wired PC and no internet access. I can turn the WiFi back on with the Unifi Android app, but still nothing, no WiFi and no internet access. I don’t even seem to be able to ping my router! Any ideas?
    Tony

    Reply

    1. No idea. You should ask on the Ubiquiti forums. They will need more details though, like:
      How can the Android device connect when there is no Wi-Fi?
      Does your PC Ethernet show link light at the PC end or at the switch end?
      Does your PC get an IP address?
      If you have turned the Wi-Fi back on, can you see it on a fresh device, which hasn’t ever connected to the network?

      Reply

      1. I can only use mobile data to access your website and Unifi app etc… I’ll check the forums…

        Thanks, Tony

        Reply

        1. I’m still wondering what happened. How did you “turn off” the networks? There is a “Enable” checkbox for each wireless network just for this purpose.

          Reply

  118. Petri,

    Thanks for this. It works a treat for both of my sites.

    Going to upgrade to two USGs so it’ll be interesting to see them in there too.

    Super job.

    Neil

    Reply

  119. Hey Petri,

    Thanks for putting this together. I’ve been using it through google cloud for a little over a month. A few days ago I started having issues, it may be something you’re unable to help with but thought I’d check here first.

    I have had trouble logging in, the controller hasn’t accepted my password. After 3-4 failed log in attempts I am unable to load the site at all. Chrome returns a Site Cannot Be Reached error saying the IP took too long.

    After stopping the instance and starting it again, I was able to access the login screen. Same issue though, after 3-4 failed attempts I am unable to access the site at all. On the latest restart I am unable to access the login screen at all. The logs return the following:

    Any thoughts?

    unifi-controller login: Jul 9 21:21:32 unifi-controller dbus[464]: [system] Activating via systemd: service name=’org.freedesktop.timedate1′ unit=’dbus-org.freedesktop.timedate1.service’
    Jul 9 21:21:32 unifi-controller systemd[1]: Starting Time & Date Service…
    Jul 9 21:21:32 unifi-controller dbus[464]: [system] Successfully activated service ‘org.freedesktop.timedate1’
    Jul 9 21:21:32 unifi-controller systemd[1]: Started Time & Date Service.
    Jul 9 21:21:32 unifi-controller startup-script: INFO startup-script-url: Localtime set to US/Central
    Jul 9 21:21:32 unifi-controller systemd[1]: Stopping System Logging Service…
    Jul 9 21:21:32 unifi-controller systemd[1]: Stopped System Logging Service.
    Jul 9 21:21:32 unifi-controller systemd[1]: Starting System Logging Service…
    Jul 9 21:21:32 unifi-controller systemd[1]: Started System Logging Service.
    Jul 9 21:21:32 unifi-controller startup-script: INFO startup-script-url: Return code 0.
    Jul 9 21:21:32 unifi-controller startup-script: INFO Finished running startup scripts.
    Jul 9 21:21:32 unifi-controller systemd[1]: Started Google Compute Engine Startup Scripts.
    Jul 9 21:21:44 unifi-controller unifi.init[651]: Starting Ubiquiti UniFi Controller: unifi.
    Jul 9 21:21:44 unifi-controller systemd[1]: Started unifi.
    Jul 9 21:21:44 unifi-controller systemd[1]: Reached target Multi-User System.
    Jul 9 21:21:44 unifi-controller systemd[1]: Reached target Graphical Interface.
    Jul 9 21:21:44 unifi-controller systemd[1]: Starting Update UTMP about System Runlevel Changes…
    Jul 9 21:21:44 unifi-controller systemd[1]: Started Update UTMP about System Runlevel Changes.
    Jul 9 21:21:44 unifi-controller systemd[1]: Startup finished in 1.702s (kernel) + 22.736s (userspace) = 24.438s.
    Jul 9 21:36:55 unifi-controller systemd[1]: Starting Cleanup of Temporary Directories…
    Jul 9 21:36:56 unifi-controller systemd[1]: Started Cleanup of Temporary Directories.
    Jul 9 21:48:09 unifi-controller dhclient[587]: DHCPREQUEST of 10.128.0.2 on eth0 to 169.254.169.254 port 67
    Jul 9 21:48:09 unifi-controller dhclient[587]: DHCPACK of 10.128.0.2 from 169.254.169.254
    Jul 9 21:48:09 unifi-controller systemd[1]: Stopping System Logging Service…
    Jul 9 21:48:09 unifi-controller systemd[1]: Stopped System Logging Service.
    Jul 9 21:48:09 unifi-controller systemd[1]: Starting System Logging Service…
    Jul 9 21:48:09 unifi-controller dhclient[587]: bound to 10.128.0.2 — renewal in 1596 seconds.
    Jul 9 21:48:09 unifi-controller systemd[1]: Started System Logging Service.

    Reply

    1. After three failed login attempts Fail2Ban will ban your IP address for an hour. That could be one explanation. To see if your IP is banned you need to log in via SSH through the GCP Console and issue command sudo zgrep Ban fail2ban.log* You can find out your external IP address from https://www.whatismyip.com or some other service.
      I can’t say why the three first attempts fail. Like always, it is usually not worthwhile to debug a VM problem. Just create a new VM with the same IP address and restore your latest backup from Google Storage. Of course, if the problem is within the UniFi database, it will carry over with the backup. In that case, create another VM and restore an older backup. That’s what they are for.

      Reply

  120. Petri, just want to say, had my first GCP controller die today, not sure why it just stopped responding, rebooted and it just sat at controller is loading (corrupt db?). anyway I blew it away and had a new one up with your script in about 45seconds, crazy easy! thank you! leads to me to wonder if theirs a way (not free) to have redundancy with this? a second controller that mirrors the first that you could just assign the external IP too in the event of an issue ? I restored one of the backups on a docker unifi I spun up quick to make sure everything was still there, but a DNS flip to point my hostname to another IP can take like 24ish hours, so quite a bit of time to loose control of multiple sites

    Reply

    1. Ubiquiti doesn’t provide any ready made high availability option. Each controller hosts its own MongoDB. Ubiquiti has documented how to use a separate MongoDB instance instead. You could build your own HA controller with two controllers using the same MongoDB database. This wouldn’t protect against database failures. You would need to use two database servers for redundancy. You could run them on the same computers as the controllers, so you would need only two VMs. Perhaps you can find someone who has already done all this.
      However, I suspect that some controller bug screwed up your data inside the database. I have a lot of faith in MongoDB. The only remedy against such bugs are backups. A HA system won’t help, since the data inside is corrupt.

      Reply

  121. You’re an absolute legend, thanks for this – got me up and running in <1 hour, including getting my GCP account re-configured, DNS provisioned, etc. Very happy! Thank you…

    Reply

  122. Just for clarification, in the metadata do you put your bucket name with or without “gs://” in the field? Thanks!

    Reply

    1. Just like in the table in the post and on the video, just the plain bucket name without gs://. The startup-script-url requires the gs:// but it is mandated by Google – I can’t change that. I find it useless to require users to type boilerplate text. It can only cause errors, I can’t see any advantages.

      Reply

  123. Is it normal for Google to halt the VM when it comes to the end of the first free year? My understanding is that $300 free for a year is part one of the deal, the other is we can use the smallest VM for free forever, but I checked my conroller yesterday and found it was offline. Went to the console to check and had to agree to any charges before I could restart the VM. Looking again the costings page is still $0 so I don’t think I’m due any charges.

    Reply

    1. Yes. Google will send you emails about needing to “upgrade” your account. I guess they just want to make sure you know you might be charged. Your first micro VM in the U.S. zones is still free. You could have upgraded your account beforehand so it hadn’t stopped.

      Reply

  124. Hi,

    I got this email from google:

    Hello Google Cloud Customer,

    We’re writing to let you know about upcoming changes to our pricing for external IP addresses for Virtual Machines (VM) and Cloud Network Address Translation (NAT) that will take effect January 1, 2020.

    Cloud NAT is our network address translation service that allows your applications to access the internet while keeping your deployments private, promoting the enterprise security best practice of using private VMs.

    What do I need to know?
    First, we’re increasing the price for Google Compute Engine (GCE) VMs that use external IP addresses. Beginning January 1, 2020, a standard GCE instance using an external IP address will cost an additional $0.004/hr and a preemptible GCE instance using an external IP address will cost an additional $0.002/hr.

    Second, we’re reducing the price for Cloud NAT. Beginning January 1, 2020, Cloud NAT gateway will cost $0.0014/hr per GCE instance up to a maximum of $0.044/hr and Cloud NAT data processing charges will be fixed at $0.045/GB for all regions.

    Below is a summary of the pricing changes:

    Today January 1, 2020
    External IP – Standard VM $0 $0.004/hr
    External IP – Preemptible VM $0 $0.002/hr
    Unused External Static IP $0.01/hr $0.01/hr (unchanged)
    Cloud NAT Gateway Depends on region $0.0014/VM/hr
    $0.044/hr maximum
    for all regions
    Cloud NAT Data Processing Depends on region $0.045/GB for all regions
    We will fully discount any external IP usage for the first 3 months to help you quantify the impact of these pricing changes. Please take note of the following dates:

    January 1, 2020: Although your invoice will show your calculated external IP-related charges, these will be fully discounted and you will not need to pay these.
    April 1, 2020: You will need to pay for any incurred external IP-related charges shown on your invoice.
    What do I need to do?
    The new pricing and discounts will be automatically applied to your accounts from January 1, 2020; no action is required on your part. However, we recommend migrating to private VMs for better security.

    We are committed to providing the best enterprise security tools to our customers, including Cloud IAP (Identity Aware Proxy) for administering private GCE instances from the internet, as well as Cloud NAT for secure access to public hosts. For information on how to migrate to private VMs, please see the below guides:

    For private VMs: Building Internet Connectivity for Private VMs
    For Google Kubernetes Engine private clusters: Setting up a private cluster
    If you have any questions or require assistance, please contact Google Cloud Billing Support.

    Thank you for being a valued GCP customer.

    https://www.dropbox.com/s/i0ziws3gqt0vdj6/Schermopname%20%28241%29.png?dl=0

    Does this means we gonna have to pay for our external ip’s too?

    Reply

    1. Yes, I got it too. Looks like the free offer is over. My math says it will cost USD4/month or USD47/year. You would pay the price of the first gen Cloud Key in less than two years. The CK still sucks, IMO. I’ll still wait til Q1/2020 to see what the charge will be.

      Reply

            1. No, but apparently they will start charging for the public IP address (which you need to use it as a controller).

              Reply

              1. Hi Petri,

                Firstly, please continue your work, even if that means porting your script to a self-hosted VM. The combination of SSL, Fail2ban and backup options is amazing, I’d love to continue to have the same functionality under Ubuntu Server or whatever you suggest.

                Secondly, it seems the IP for the free tier will remain free, providing the correct association is maintained. A Google employee (a googler) has said:
                “Hi Folks,

                Googler here.

                You will not be charged for your free tier VM’s external IP address.

                From the docs:

                “Your Always Free f1-micro instance limit is by time, not by instance. Each month, eligible use of all of your f1-micro instances and associated external IP addresses are free until you have used a number of hours equal to the total hours in the current month. Usage calculations are combined across the supported regions.””

                https://www.reddit.com/r/googlecloud/comments/cswg9g/will_my_free_microvps_start_cost_money_2020/

                Reply

                1. Thank you for your input. The response you got appears to contradict some others. I hope yours in correct. At least it very clearly answers the question we are having.

                  Reply

      1. Perhaps Google will edit that section, I don’t know. All current users received an email about the change to come. I’m going to wait until Q1/2020 to see what the effect will be.

        Reply

        1. I messaged GCP support about it, this is the response I received:

          ________
          “Hello,

          Thank you for reaching Google Cloud Platform Billing Support. As I understood your concern, you want clarification regarding the received email for pricing changes on January 1, 2020 and wants to know if this changes will have you reach more than $300 every year. I will be more than happy to assist you on this matter.

          Please be advised that the update was to inform you that Static and ephemeral IP addresses in use on standard VM instances will be charged $0.004 per hour starting January 1st, 2020 [1]:

          This means that an external IP address will be no longer free, even if it is in use and you will have to pay ~2.90 USD per month for each IP(v4) address.”
          ______

          So yes it appears the free IP address with a micro instance is a thing of the past.

          Thanks

          Reply

          1. Thank you for confirming this. I added the NAT gateway ($0.0015/h) to the price so I ended up with $4/month.

            Reply

                  1. I haven’t tried how Google controls it. Perhaps you will need to create a new email address. You can use the same credit card for multiple accounts, that I do know.

                    Reply

  125. Hi Petri,
    Thanks for detailed updated instructions.
    I have two problems.
    1. In the controller Settings – Guest Control – Enable Guest Portal Message: CONTROLLER ON-LINE REQUIRED (but guest portal work)
    2. When I open in browser unifi.yyyyyy.org:8443 – all looks fine, cert valid. But when the user try to connect to Guest Portal unifi.yyyyyy.org:8843 (Use Secure portal – Enabled, Redirect using hostname https://unifi.yyyyyy.org – Enabled) – Chrome and Edge show cert error. But Mozilla just open Guest Portal.
    I`m try to check cert at https://www.geocerts.com/ssl-checker and seems that my unifi controller don`t have middle and root cert installed: “Certificate Chain Complete? A valid Root CA Certificate could not be located, the certificate will likely display browser warnings.”

    Reply

    1. 1) I have very limited experience with the guest portal as I haven’t seen anyone use it here in Finland. Guest portals are strongly disliked over here. Perhaps you should ask on Ubiquiti forum about the check whether the controller is on-line, how is it determined.
      2) I checked my controller with the GeoCerts checker and got the same result. Too bad SSLabs checker won’t test any other port than 443. However, when I check the cert in my browsers (clicking the lock icon) I can see a full path. I use Chrome, Safari and Firefox (in that order of preference). Do you see a difference with different browsers connecting to the controller vs. the guest portal?
      I am interested in this as this may reveal some error in my certificate handling. There is an error when importing the cert chain but I have dismissed it as superfluous as I haven’t seen any symptoms. ATM I am not certain wether the problem lies in the certificate chain or in GeoCerts’ checker.

      Reply

    1. AFAIK. That’s why I wrote my script for GCP.
      Personally I believe it was an attempt to gain market share for Google. A very expensive since they gathered all the tiny players who wouldn’t be paying anyways. Of course some of my script users have upgraded to a bigger VM, too. Still, I believe it wasn’t well thought out move. I was glad to make use of it, though. Google can afford it.

      Reply

  127. Hi Petri,

    Due to the new changes in GCP. What is the best method and or how to move the settings and etc to my controller at home.
    I will be running a VM with ubuntu installed.

    Thanks

    Reply

    1. Ian above received a note clearly stating that the free tier will still be free. I am waiting to see what the final verdict will be. Q1/2020 will still be free but we will see the cost accrual.

      You can run UniFi controller on a vanilla Ubuntu or Debian system. Ubiquiti has decent documentation for it. You wouldn’t need Fail2Ban on your home network nor swap or HAVEGEd on your own system. Getting a working certificate for SSL will be a struggle, but there is nothing I can do to help it with scripting. I wrote my script with the peculiarities of a cloud environment in mind and to secure a system facing the public Internet.

      Reply

  128. Hi Petri,

    I saw that there is a new version of the unifi controller available.
    I thought that the script would automatically upgrade the controller from 5.10.26.0 to 5.11.39 ?
    Thank you.

    Reply

    1. Mine just did. The upgrade will happen the night after Ubiquiti updates the repos. They first make it available for manual downloads to get more feedback. 5.11 turned out to be troublesome for many so Ubiquiti withheld the release for a while. I am certain you either wouldn’t want your working controller to auto upgrade to a defunct version.

      Reply

  129. I already have another VM besides the dedicated one for Unifi, I’m wondering if we could use GCP’s Cloud Identity and Access Management (IAM) to run the Unifi controller without a public IP to work around the static public IP charges that may come about?

    Reply

    1. I can’t think of a way to use IAM, but that’s just me. I would set up port forwarding on the VM with an IP and forward ports 8443, 8080 and whatever else you need to the UniFi controller.

      Reply

    1. The script were, but UniFi Controller isn’t. Java 8 and MongoDB are required, but not supported on Buster. There are workarounds, but I prefer not to kludge together something unsupported. Stretch will be supported until 2022 so I have decided to wait for true support. (Oh boy, if only Ubiquiti would move to current Java versions!)

      Reply

  132. Hi Petri!

    I am trying to establish some security here by only allowing traffic from fixed ip address to the controller. The question is about ports 80 and 443. Are these for certbot? Does letsencrypt need open access to the webserver? Lighttpd is only for redirecting, right?

    too many questions in one comment, I know…

    thank you for your support!

    Reply

    1. Yes, Let’s Encrypt uses either 80 or 443. Lighttpd is only for redirecting. Lighttpd is stopped during certification renewal when CertBot takes over those ports.

      Reply

  133. Seems my controller quit working 2 days ago, and I only realised today. Going into console it confirmed the free trial had ended, and only allowed the VM to be started after I emailed billing. Has anything changed that you’re aware of?

    Reply

    1. After your first year credit expires you need to confirm/update your account. I believe this is so you are made aware of the forthcoming charges. I dismissed my email notices and got my account locked up at the time. All it took was a few clicks in the billing section.

      Reply

      1. Yes I seem to have got it back up and running… Bit surprised there was no notification and they just stopped the VM! Question is… How do you estimate the cost? I’ll need to weigh against just migrating to a cloud key.

        Reply

        1. The notification email is probably in your spam folder.
          The first U.S. based micro sized VM is free. This apparently includes the new charge for the IP address as well. Something Google didn’t make clear at first and it caused a lot of confusion.

          Reply

          1. I couldn’t find the notification anywhere, but nevermind it for now.

            I’ve noticed that I’ve started to accumulate ‘real’ costs now for the traffic between US and UK. I’m monitoring and will see how it works out over a full month.

            Should I decide to migrate to a local cloud key, what’s the process? Do I just download a backup from the VM and restore that onto a local controller?

            Reply

            1. My network is in Finland but I haven’t been charged. What do carry over the Atlantic? My VM and backup bucket are in the same zone in the U.S. Inbound traffic is free. Outbound (from U.S. to U.K.) is charged after a threshold. That contains you browsing the controller and firmware upgrades from the controller to the devices. My traffic has been well below the threshold.
              Yes, you just restore the latest backup to the new controller (CloudKey, local PC, some other cloud VM). Then you change the inform address in the old controller to point to the new.

              Reply

              1. Not entirely sure, but it appears only 1gb of egress is free monthly, the rest at a charge. My average over the last several months is about 4gb. What does yours look like out of curiosity?

                Reply

                1. 1Gb should be plenty! I just went through one site’s billing information. One month this year there was 2Gb of transfers (in March) but I couldn’t get the details for but the last three months. Nobody knew anything specific had happened in March so it is a mystery. There were some small amounts of traffic to&from APAC for example, which I can’t explain. My guess is they are some exploit attempts.

                  Reply

                  1. I’ve no idea what can be using that kind of bandwidth. I have only A USG, two AP’s, and two switches. All of this hosts 40-50 network devices. From what you’ve said it’s only firmware updates and (presumably) accessing the controller via browser or app.

                    Any way to tell what’s using this?

                    Reply

                    1. I believe it would be wasted effort. For example, you may have an IP that was used for some other service earlier and b/c of some misconfiguration there are millions of devices that keep trying.
                      Create a new VM, grab a new IP address, point your DNS to the new IP, restore the latest backup and point your devices to the new inform host.

  135. Hi there,
    Thanks for your assistance with this.

    I got an e-mail from Google stating the following:
    E-mail Subject: “[Action Required]” Legacy GCE Metadata Server endpoints will be shut down on January 15, 2020″

    Hello Google Compute Engine Customer,

    We are writing to let you know that the v0.1 and v1beta1 endpoints of the Compute Engine Metadata Server will be shut down on January 15, 2020. From January 15, 2020, requests to the v0.1 and v1beta1 endpoints will no longer be supported, and may return HTTP 404 NOT FOUND responses.

    Our records show that you own projects that have requests to these endpoints. You will need to update your requests to the v1 endpoint before January 15, 2020….

    I’ve tried to read up on this but Google’s documentation does not seem to be compatible with my brain. Can you assist?

    Reply

    1. Hmm.. I haven’t seen that announcement. However, it only says that when the script reads metadata it has to provide a Metadata-Flavor header. My script has always included it as far as I can remember. Perhaps you have used some other script or made some personal experiments with the metadata? Anyways, the script only runs at boot so this change won’t affect running controllers. Even if the script would fail during boot the controller will still start (without any metadata changes, though).

      Reply

  136. Hi Petri,
    Thanks for all of the work you have put into this!
    I have followed all instructions and have a fully functioning cloud controller. It is a beautiful thing!
    I have one problem/question/request???
    I am not sure how to view the log files I believe they are located at /usr/lib/unifi/logs/
    I there any way that the log files for the controller could be exported to the bucket so that they can be reviewed?
    Could this be part of the script?

    Reply

    1. I wrote the script for people who don’t know Unix and really don’t care. They just want their Wi-Fi to work. They never look at their Windows events either. Copying the logs to a bucket would just waste my time, storage capacity and transfer bandwidth.

      If you are savvy enough you can log on using the SSH button in the console. Use sudo su - to get a root prompt to get to most of the UniFi stuff. If you really want you can duplicate and edit /etc/systemd/system/unifi-backup.service and /etc/systemd/system/unifi-backup.timer to copy the logs.

      Reply

  137. Hi Petri,
    Great news on the f-1 micro sized VM staying free (fingers crossed). I am curious on how the “Always Free” usage limits are calculated. What does 28 frontend instance hours per day mean and 1GB of egress per day mean?
    I like to log onto my unifi controller a few times a day from both my computer and the App (I’m sure this will subside once the newness of it all wears off).
    Am I at risk of busting the limits if all I do is log on a few times a day?

    Reply

    1. No worries. I’ve kept a browser window open for months without any significant traffic. The only times you cause significant traffic is when you download backups to your local computer (easy to track) and when your devices download firmware updates from the controller (by default manual operation as well). Each device downloads its firmware separately, so there is a risk of excess traffic if you have many devices.

      Reply

  138. Does anyone else get alerts each day saying their USG / APs were disconnected? I seem to be getting this. Its around 4am which is when I have my daily backup set up? Could the backups be taking up too much server resources leading to the devices missing their heartbeat?

    Reply

    1. I haven’t seen many of those, but yes, it could happen. The controller kind of freezes when it is compacting a database or creating a backup.

      Reply

  139. Prefacing this comment by saying a huge thanks for this guide/script, works great.

    I do have an issue though where if I reboot the controller my devices don’t reconnect. When I log into my USG for some reason the FQDN of my controller is resolving as 127.0.0.1.

    PING unifi (127.0.0.1) 56(84) bytes of data.
    64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.338 ms
    64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.254 ms

    To get my devices to reconnect I have to set-inform http://GoogleExternalIP:8080/inform and they all reconnect stright away. I use my own domain so not a shared DNS service.
    Any ideas?

    Reply

    1. set-inform isn’t covered in the instructions 🙂 The devices store the set-inform address and a backup value. If the primary host can’t be contacted they fall back to the backup – and never retry the primary. The solution is to issue set-inform twice to overwrite the backup value, too.

      Reply

  140. There might need to be some updates to the LetsEncrypt section. I’m running it today and getting an error

    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator standalone, Installer None
    Registering without email!
    An unexpected error occurred:
    The client lacks sufficient authorization :: Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details.
    Please see the logfiles in /var/log/letsencrypt for more details.

    I found this article that includes today as a brownout day.
    https://community.letsencrypt.org/t/installing-error/103874/4

    I’m not sure what ACME client would be, since it is just using certbot.

    Any thoughts?

    Reply

    1. Thanks for the heads up. I thought certbot should automatically switch to the new protocol. I’ll look into this and update this comment.

      Reply

      1. You haven’t followed my instructions if you are on Ubuntu 🙂 Debian repos are at certbot 0.28. Good to know an update will solve this. I’d hate to hardcode a specific endpoint version into the script.

        Reply

        1. Hah. No I’m not following the instructions to the letter. I’m adapting it for Ubuntu to get it running on Oracle.

          Apparently I should not assume that:
          1) Ubuntu repositories would install a current version of certbot
          2) Ubuntu and Debian would install the same (or close) versions

          Reply

  141. Petri,

    It seems that fail2ban isn’t working in my setup. I followed the instructions & the video and in my console logs, I can see that the fail2ban service has started

    Oct 17 19:24:23 unifi-controller systemd[1]: Starting Fail2Ban Service…
    Oct 17 19:24:23 unifi-controller fail2ban-client[28798]: 2019-10-17 19:24:23,365 fail2ban.server [28799]: INFO Starting Fail2ban v0.9.6
    Oct 17 19:24:23 unifi-controller fail2ban-client[28798]: 2019-10-17 19:24:23,366 fail2ban.server [28799]: INFO Starting in daemon mode
    Oct 17 19:24:23 unifi-controller systemd[1]: Started Fail2Ban Service.
    Oct 17 19:24:23 unifi-controller systemd[1]: Reloading.
    [ 183.775165] systemd[1]: apt-daily.timer: Adding 7h 53min 36.834348s random time.
    Oct 17 19:24:23 unifi-controller systemd[1]: certbot.timer: Adding 2h 35min 2.601576s random time.
    Oct 17 19:24:23 unifi-controller systemd[1]: apt-daily-upgrade.timer: Adding 26min 38.884724s random time.
    Oct 17 19:24:25 unifi-controller startup-script: INFO startup-script-url: Fail2Ban installed
    Oct 17 19:24:25 unifi-controller systemd[1]: Reloading Fail2Ban Service.
    Oct 17 19:24:26 unifi-controller systemd[1]: Reloaded Fail2Ban Service.

    Since then, I’ve tried to fail my own logon on a different network, and I still get the landing page even after the 3 bad attempts. When I run the sudo zgrep Ban fail2ban.log* command in the /var/log directory, I don’t get any returns for the blocked IP’s. Is there something I’m missing in the configuration or otherwise? Everything else is working as intended from the cert to the ddns.

    Reply

    1. You seem to have spotted something interesting. UniFi controller doesn’t log failed logins any longer. Fail2Ban runs just fine, but there is nothing in the log! You need to change the logging level in the Controller: Settings > Maintenance > Services > Log level > Mgmt > More > Apply. I have updated the blog post step 5 accordingly. Thank you!

      Reply

  142. Awesome, thank you so much Petri. That fixed it no problem.

    Now to figure out why my AP’s don’t migrate to the GCP controller, when the USG and switches migrated no problem. If I ssh to an AP, it says it’s unable to resolve my inform address, or any other address, I can ping my static GCP IP without issue. I’m assuming this is because the USG is no longer on the same controller providing DNS. I’m going to try and bring the USG back into the original controller and then use the unifi migration tool.

    Reply

    1. Even if the USG is not on the same controller, it is still providing DNS for the subnet. The APs get the DNS setup via DHCP. It is completely independent from the controller.
      How about if you use the IP address for the inform host? Will they connect? There is no added value in using a DNS name as long as you have the same IP address. You could change the inform host from the GCP controller after the migration, if you want.

      Reply

  143. Hi! I have had the controller running for over and year and just noticed that Google charged me $0.02 for the month of October and the changed was for:
    Unifi Controller studious-linker-219118 $0.34 -$0.31 — $0.03

    What do you think is causing me to be charged. It’s not a large amount I know, but just strange that all of the sudden I’m being charges.

    Thanks for you help!

    Reply

    1. I really can’t tell. One explanation could be a common script trying to exploit some PHP vulnerability and sending zillions on http requests. You controller would always send back the login page. All those login pages could eventually add up and go over the free egress amount. There is very little we can do about that. You could change your IP, but that could case an outage before DNS settles.
      The way my script is written those http requests will always be redirected to the UniFi controller and not logged.

      Reply

      1. I also got charged for 0,02 euro. Never had this before…
        So maybe you need to change something inside your script.

        Because i have nothing else running besides your basic script.

        Reply

        1. Actually the script is not running. The script only runs once at boot, checks the setup and exits.
          I am thinking if there is something I could do to help this. I’ll announce it here if I come up with something.

          Reply

  144. Hi also getting some charges. Storage PD Snapshot in US, Network Internet Standard Tier Egress from South Carolina.

    Each for .04 USD. Any ideas? I though some storage was free.

    Reply

    1. I am working on a solution. Apparently there are tons of robots downloading the login page causing the traffic. Please check back next week at latest, I hope.

      Reply

      1. Hi Petri any update on the script fix to stop the charges from Google? I can see a lot of Egress to Japan, Mumbia and other areas. I believe this is was causing the charges to show up.

        Reply

        1. Not yet. I have apparently found a bug in Fail2Ban, which needs to be fixed. In the meantime, you can disable Lighttpd altogether by logging in and issuing sudo systemctl stop lighttpd and sudo systemctl disable lighttpd. That means you will need to use the full https://my.ctrl.dns:8443 URL, but your browser should already know it.

          Reply

  145. Do you have an issue where the WiFi drops for a minute after the controller restarts at 4:00 AM?

    Reply

    1. No, that shouldn’t happen. The devices don’t depend on the controller. They can work indefinitely without one. You can run the controller on a laptop and only launch it when you want to change the configuration. The cause is something else.
      The controller doesn’t restart every night. Only when there are kernel patches or something alike that requires a reboot. Debian development has moved to Buster and later, so our Stretches see very few patches.

      Reply

      1. I restarted the controller and it didn’t have the WiFi disconnect so I think maybe the internet drops at 4 AM normally or the garage door opener is broken and disconnects from WiFi and then complains that it’s offline every morning.

        Reply

  146. Hi, Curious how long it typically takes for the script to install the latest version of the controller. Looks like I am on Version 11.50 but there is 12.22 on the site. Thanks.

    Reply

    1. My script won’t install any updates. My script only does the initial setup. After that, unattended-upgrades service takes care of updating. It runs every night and will install the latest version as soon as Ubiquiti releases it to the repository. This has usually taken about a week after the release. With such a big version bump this time they may want to gather more feedback from the early adopters. My script is not for them. I wrote it for people who don’t want to fiddle with their system.
      There is no easy way to downgrade a controller. You only want to upgrade when the upgrade has been proven to be solid.

      Reply

  147. Hi Petri,
    Apologies if this is already answered above; Ingot an email from Google stating “Action requested: Please take a moment to upgrade your account (expiring in 7 days).”

    It says if I don’t upgrade, my VM will stop working. If I do upgrade, do I get charged for the f-1micro?

    Very confusing information from Google.

    Thanks

    Reply

    1. That just means that you first year’s credits are gone. Google just wants to double check that you are ready to pay if you exceed the free tier. The first f1.micro is still free.
      You need to comply. I dismissed the notice and got locked out from my account at the time. No need to retry that.

      Reply

      1. Ok, I upgraded (hopefully it stays free).

        Question: I looked at my dashboard for billing, and it appears there is a small charge every month for “Cloud Key Management Service (KMS) ” It’s a different amount every month. What is it, do I need it, and can I turn it off?

        Thanks.

        Reply

        1. No idea. I haven’t seen that. Turn it off if you can find where. If something breaks, report here.

          Reply

          1. Do you have any more news on the 0.03 cent that we had to pay extra this month regarding the http traffic? You said you would come with a solution the latest by the end of this week…

            Reply

            1. I have found out that there are huge bot networks combing through GCP IP addresses looking for common web server vulnerabilities. They all get the login page, which will add up to the egress traffic. I have tested a remedy against the bots, but it requires changes in the setup. I haven’t yet found a safe way to implement the changes to existing controllers. It looks like you will need to create a new controller. Don’t do this yet, since I haven’t published the new version. I’ll let you know when I am confident with it. Keeping the controller functional is more important than a dime, after all.

              Reply

              2. Hello Petri,
                Thank you so much for your work!

                Just my 2 cents about the unwanted traffic discussion.

                I expanded your firewall rules with specific source IPs.
                For every site I add, I first add the IP. Since I cannot protect this controller with 2FA, I felt uncomfortable leaving it like it was. In a way we need a private cloud for the component.

                I have to leave port 80 open for Let’s Encrypt auto-renewal to work which visits the well_known folder, but at the root only the redirect exist. All the other ports only allow traffic from the different sites running equipment.

                It does mean I can only log into the controller from one of those IP’s, but with a VPN connection that is not that much of a trouble, even from phones/tablets.

                To keep sane, I had to change the composition of the firewall rules a bit in G-cloudconsole.
                -I have one rule for every site containing all the ports without 22.
                -I use the original letsencrypt rule port 80 since I cannot predict the origin of letsencrypt validation calls. (Side-note: validation via DNS could be possible if txt record creation can be automatically published in a DNS-zone.)
                -I have a separate SSH rule port 22 rule for when I need it, since SSH terminal started from G-cloudconsole can come from different IPs.

                Since I don’t add new Controller site every week or month, I accept the inconvenience of the firewall rules in exchange for not having unwanted log-in attempts. When I set it up, it was because of security, but nowadays I found an extra argument: no weird traffic that costs money.

                Curious to see what you have been doing, look forward to reading it.
                Possibly the case above can be of aid to some. (If it passed already; sorry for missing it.)

                Regards,
                Djo

                P.s. I read you FAIL2BAN remark at the top. Do you have a mailing list? Updates that break a security measure like this would be something I would like to know asap. I know you do this all for nothing, don’t get me wrong, I am very grateful. But if there are any existing ways to stay in the loop, let me know.

                Reply

                1. Hello and thank you for your input. I wrote the script and instructions for users who have zero experience with Linux or cloud services. Many users have reported success with very little IT skills in general. Apparently you could have done it all without my script. Setting up firewall rules for specific IP addresses and ports is way beyond my original scope. My intention was to make the setup as secure as possible but keep it as a turn key. That’s why there are automatic upgrades and Fail2Ban rules.

                  There is no mailing list at the moment. This Ubiquiti’s change of logging has been the first to actually break something. I put up the notification at the top so you don’t need to wade through all the comments to see if something important has turned up. I never expected this to become so popular. I’ll need to think about this.

                  Reply

          2. Hi Petri,
            somehow the Cloud Key Management Service (KMS) API was enabled on my GCP. I want to disbale it, however it says “If any resources were created by Cloud Key Management Service (KMS) API, they may be deleted soon after Cloud Key Management Service (KMS) API is disabled.
            All code that uses this project’s credentials to call Cloud Key Management Service (KMS) API will fail.”

            My concern is; does this have anything to do with my LetsEncrypt Cert? If not, I will disable. Thanks

            Reply

            1. I don’t have that enabled. No, it doesn’t have anything to do with Let’s Encrypt, either.

              Reply

  148. My controller is still on 5.11.46.0 and keeps telling me “Controller software update 5.11.50 is now available.”. I’ve done multiple reboots and it never seems to update. Any idea how I troubleshoot this?

    Reply

    2. It should have upgraded by now. If you want to troubleshoot it, login by SSH and look in the logs in /var/log/unattended-upgrades. You may want to try sudo apt-get update and sudo apt-get upgrade to see any error messages. However, my recommendation is to create a new VM, transfer the same IP address to it and restore the latest backup. It is a quick procedure and if a VM has misbehaved in the past it doesn’t bode well for the future.

      Reply

      1. 2019-11-22 04:29:24,323 INFO Initial blacklisted packages:
        2019-11-22 04:29:24,338 INFO Initial whitelisted packages:
        2019-11-22 04:29:24,338 INFO Starting unattended upgrades script
        2019-11-22 04:29:24,338 INFO Allowed origins are: [‘origin=Debian,codename=stretch,label=Debian-Security’]
        2019-11-22 04:29:26,680 INFO No packages found that can be upgraded unattended and no pending auto-removals

        Nothing here really. I did apt-get and it found the update and upgraded. I’ll keep an eye on it and if it does it again with the next release I’ll create a new instance. Cheers!

        Reply

        1. Yes, there is. Allowed origins should be ['origin=Debian,codename=stretch,label=Debian-Security', 'o=Debian,a=stable', 'c=ubiquiti'] according to my script. The last one is for UniFi updates. Please check if /etc/apt/apt.conf.d/51unattended-upgrades-unifi exists and what its content is.

          Reply

  149. Thanks Petri for your work and the script. In the past I set up unifi controller at aws.

    Reply

  150. Hi Petri,
    I’m seeing a charge for $0.22 (I know, nothing to cry about) for “Network Internet Egress from Americas to EMEA” It says 2.1GB egress was sent to EMEA from my GCP. WHat is IMEA and is there any way to stop it? Thanks.

    Reply

    1. EMEA is an acronym for Europe, Middle-East and Africa. I am working on a fix, but it takes some time to test each intermediate version. That’s why it is taking so long.

      Reply

  151. Hi Petri,
    You’ve helped before and we’re very grateful 🙂 We run 20 sites with 78 devices. We have it on a f1-micro (1 vCPU, 0.6 GB memory). We keep not being able to access it and have to restart it. Once it’s restarted it works fine for a while. Any idea why? Thanks!

    Reply

    1. The performance of f1-micro is on par with CloudKey gen1, which suffices for up to 30 devices says Ubiquiti. You should upgrade to next larger instance (for a fee). At least you can try it for a month and see if it solves the problem.

      Reply

    2. I dont want to speak for Petri, but according to him: “Ubiquiti quotes 30 devices for 1GB Cloud Key” and according to Petri, the f1-micro is similar to the cloud key. With 98 devices, you might need to bump up your VM. My $.02

      Reply

  154. Anyone else get this for the logs changing and modify the script??

    Hello Google Compute Engine Customer,

    We’re writing to let you know that Compute Engine activity logs, currently in Beta, will be shut down on September 30, 2020. From that date, new activity logs will no longer be emitted.

    Our records show that you own projects that may be reading or exporting activity logs. To continue monitoring activity on your Google Compute Engine instances, we recommend that you migrate from activity logs to Cloud Audit Logs.

    [… abridged …]

    Thank you for being a valued Compute Engine customer.

    Sincerely,
    The Google Compute Engine Team

    Reply

    1. The script or any of the software it install don’t use any beta services. I wasn’t even aware of any activity logs. No need to worry. Perhaps you have some day clicked on some link in the Console and Google has made a note of your interest in it. No action required. Thanks for the heads up, though.

      Reply

  155. is there a way to (self) modify the scrpt so that it doesnt auto update either the unifi devices or the controller itself? I prefer to take ownership over when the updates are made, ensuring adequate time for bugs to be ironed out etc

    Reply

    1. Most of the time people complain why their controller isn’t upgrading quickly enough 🙂
      You need to log on to the VM and edit /etc/apt/apt.conf.d/51unattended-upgrades-unifi. Remove the line with ubiquiti to stop upgrading the controller. If you don’t want any automatic updates you can stop and disable the unattended upgrades service: sudo systemctl stop unattended-upgrades and sudo systemctl disable unattended-upgrades.
      Unifi device auto-updates are controlled within the controller and it is disabled by default.

      Reply

  156. HI, Im new too this. I followed your instgructions to the T but when accessing my site via the webpage it comes up with the Unifi Security Gateway sign in page. Please help

    Reply

    1. You need to use the DNS name of the controller or the external IP address for access. Not the IP address of your USG (probably 192.168.0.1).

      Reply

  157. Hey, Thanks for the script. Everything is up and running well – however I am looking for where unifi puts its logs. I am havingg an issue uploading a 130kb JPG to use as a portal background. The only one I found is in /var/log/unifi/

    I am unsure if unifi controls its own installation or if you set custom paths for logging but there are no logs there. I would like to look at the logs so that I can see why the image upload fails.

    Or maybe you have run into this.

    On trying to upload JPG:
    The selected file is either broken or of unsupported type.

    Thank you much, do you have a donation link?

    Reply

    1. My controller log is at /var/log/unifi/server.log. Do you have any devices connected? Most of the server log is device reports. There should also be a gcp-unifi.log, if you are using Let’s Encrypt.

      I haven’t used any personal backgrounds. Perhaps there is something funny with the image. Try saving it with some other editor. Avoid any “progressive” JPEG or JPEG2000.

      No, no monetization at the moment. Not even Google Ads 🙂

      Reply

  158. Hi Petri,

    Firstly thank you for the guide. I’ve followed it through and everything is working successfully.

    I just thought I’d leave a note in case anyone encounters the same issue as I just did which vexed me for half a day.

    The GCP controller is set up to listen on port :8080 for devices when they are looking to connect – that’s the standard. Unbeknownst to me (or maybe I forgot…) my local controller was using 8088 instead, as 8080 happened to be in use when I installed it (not on a cloud key). This meant that when I pointed the inform host to my new GCP location, it was trying to connect to a non-existent :8088 location.

    This was resolved by stopping the local controller, and updating the system.properties file with port 8080. (After confirming that port was no longer in use, obviously).
    See here: https://help.ubnt.com/hc/en-us/articles/204910084-UniFi-Changing-Default-Ports-for-Controller-and-UAPs

    Once it was set back to 8080, I restarted the controller and once this was sent to the devices (automatically), they immediately connected to the GCP controller instance.

    Reply

  159. Hi Petri, many thanks for all your work on this. I was able to follow your instructions to set up the UniFi Controller on the Google Cloud platform the best part of a year ago, and this has worked seamlessly until a few days ago..

    Now I simply get “site cannot be reached” when trying to access the controller. I’ve restarted the VM, but still the same.

    Any pointers?

    Many thanks,

    Tony

    Reply

    1. Trying pinging the external IP address from your workstation. This test tells whether the IP connection works. If it doesn’t, then traceroute (or tracert on Win) may help figure out where it is blocked.
      Try resolving controller’s hostname from your workstation. There are multiple commands but nslookup my.ctrl.dns should return the external IP address. Two other possible commands are dig and host.
      If both ping and name resolution work, then the problem is in the http/https connection. Your browser has developer tools, which can show the network connection process. Lighttpd should return a 302 redirect to https://my.ctrl.dns:8443, which should give you the login page. Some port scanner tool can verify that you have TCP ports 80 and 8443 accessible.

      Reply

      1. Hi Petri. I’m afraid I haven’t made any progress with this. Everything is working fine with my network, but I still can’t access the controller. Pinging the external IP address is fine. I’ve tried a couple of tools to check 8443 is open; its not, but then again I haven’t been able to find any ports open using these tools.

        I tried to add a UniFi Security Gateway a month or so ago, but there was a default gateway clash with my ISP router. I was able to change the router gateway, but still couldn’t see the Security Gateway, and gave up in the end. I left it all in place, and perhaps this is a related issue?

        Reply

        1. Not sure if you had your problem resolved yet. I had the same problem, google stopped my VM after one year free trial expires. I fixed it by rebooting the VM. Good luck.

          Reply

          1. You need to reconfirm that you are still aware of the possible charges on your credit card. My announcement went into the spam folder and I was locked out, until I figured out what was the problem.

            Reply

        2. Toni,

          Have you resolved the issue? I have the same problem now. The controller was running wonderfully (thanks petri!), then all of a sudden, I cannot connect any more. “This site can’t be reached”. However, ping and trace return normal result. I even created a second VM on a different account, but with the same error. Any help would be appreciated.

          Thanks

          Reply

          1. I am not aware this is a widespread problem. I haven’t seen it myself. I just heard about another case, but it didn’t respond to ping either.
            If you want to troubleshoot it you can check the logs: /var/log/unifi/server.log and /var/log/syslog of course. Check the process table, too. You should have a few Unifi processes, a Java process and a MongoDB process running. systemctl status unifi will show something as well. You can also try to stop and start it.

            Reply

  160. Petri,
    does Lighttpd do anything other than automatically add the port number (8443) at end of URL?
    Isn’t there another way to to do that vice Lighttpd?
    Thanks

    Reply

    1. It adds the port number and changes the connection to https. The problem is that there are bots trying to exploit common WordPress, phpMyAdmin etc weaknesses. They all get the login screen, which is a few kilobytes. With enough bots it adds up.

      Reply

      1. Got it. So turning off Lighttpd will stop bots probing http URL from automatically getting to https login screen. Correct? Also, unlikely a bot will probe a HTTPS:// URL with :3448 at the end. Do I have that right? I wonder if there’s a way to drop bot requests after the VM receives like more than 5 requests per hour? Just a thought.

        Reply

  161. Petri,
    Thanks for the time and efforts you have spent on this!

    I have really enjoyed learning cloud stuffs even by using your automated scripts, etc.

    I mainly use this GCP as an off site DR in case my UCK RIP or gets corrupted. The new external IP costing $ (not a lot) but I don’t use it for Business like everyone do (majority of the population).

    It looks like I will be deleting my account by the end of March/2020. There is no free lunch forever and that is always true.

    Keep up the good works and best of lucks but I will keep on checking the Unifi forum and yours 🙂

    Reply

    1. No, the free offer still stands. The IP is included in the free tier. That’s how I have interpreted and someone (above) has specifically asked Google and got the same response. I am still waiting to see what happens in March, but don’t give it up, yet.

      Reply

  162. My email notifications stopped working recently anyone else suffering this issue?

    Reply

    2. There are so many variables there. The SMTP server administrators may have tightened their relaying policies. The account used may have expired. There might be some block between the SMTP provider and GCP. I am not aware of any changes to the UniFi Controller in this regard, but I don’t use email alerts myself.

      Reply

  163. Hi,

    Im still getting charged for 0,02 eurocent. I disabled lighttpd like you said 2 or 3 weeks ago. The total GB they chatged was 0,2GB (Network Internet Standard Tier Egress from South Carolina).

    Reply

  165. Im getting this on this month details:

    Credit- External IPs will not be charged until April 1, 2020. And here theres a credit.

    Compute Engine – Network Internet Standard Tier Egress from
    South Carolina 1 jan. – 31 jan. 0.198 gibibyte 0,02euro

    And the total of 200mb this month and again 0,02cent charged while i already disabled lighttpd 3 weeks ago.

    Reply

    1. Somehow you should find out the type of traffic. I noticed quite a lot of http-vulnerability exploits, which always were redirected to the login screen. Disabling Lighttpd should fix that. I am not seeing any other traffic.

      Reply

      2. And again 0.01 cent charged (70mb traffic) to North Carolina server. And its February 12 with Lighttpd disabled.

        Reply

  166. My invoice looked like this. I doubt care much about the 0.01 but what is the 27.77 SEK that’s free now but it’s supposed to be charged from April 1?

    Interval Usage Amount(SEK)
    Compute Engine External IP Charge on a Standard VM Jan 1 – Jan 31 742.42 Hours 27.77 SEK

    Compute Engine Network Internet Egress from Americas to
    China Jan 1 – Jan 31 0.002
    Gibibytes 0.01 SEK

    Credit External IPs will not be charged until April 1,
    2020
    Subtotal in SEK SEK 0.01
    VAT (25%) SEK 0.00
    Total in SEK SEK 0.01

    Reply

    1. I haven’t disabled Lighttpd yet though, but I assume that is the 0.01 part, not the 27.77?

      Reply

  168. Hello Petri or anyone,

    Do you have experience with disconnecting and then quickly reconnecting wifi clients?

    I have stumpled upon this for a few months now on 6 different tiny sites (prosumers).

    In means that the eventlog is filled with listings of a client every now and then disconnecting having used X MB and then in the same or next minute, reconnecting. Sometimes there is a “roams from here x to y in there”, but those have always been there and make sense because somebody physically moves around; but the quick disconnect-connect events do not make any sense. The users don’t seem the notice anything. One site does have performance issues, but I am suspecting slow DNS performance.

    The behaviour is new to me. The Unifi forums do refer to such incidents, but they date for years back. A more recent thread I could find is this one: https://community.ui.com/questions/iPhone-Constantly-Disconnects-and-Reconnects/0240acbd-6db7-4f52-86b9-e75ac7373948

    I can confirm iPhones are related to these events, but they seem to be in higher numbers; I also see the behaviour in some non-iPhone devices.

    In the last few months I have rotated several settings, even isolated a site to a different controller, but no difference. The eventlog keeps being filled with the disconnect/connect events.

    The devices in my case are Unifi AP AC Lites and Unifi AP AC LongRange. The AP number varies from 1 to 3 in these tiny sites.

    Before I start listing my configurations, I just wanted to check in if anyone has this similar behaviour and if this might be a firmware issue which cannot be fixed by making config changes.

    There are a lot of variables inside the controller-config, but I am beginning to suspect the cause is another common nominator.

    Looking forward to hear from you!

    Reply

  169. Hi Petri,
    Thanks alot for your guided steps and video and it works well on my first try setup as I am a noob on this.

    Am having an issue now. The DNS name have been changed on my side and I am not sure how I can edit that on the GCP So that the Let’s Encrypt will update on the new DNS name for the HTTPS.

    Hope you could assist.
    Thanks alot.

    Reply

    1. I think I found out how to edit the custom meta-data by editing it in the VM instances. I did a stop and start after I edit the dns-name. Please guide me if that is the right way to do.
      Thanks

      Reply

      1. Yes, that is right. You can edit the meta-data in the console and the stop/start the VM to run the script.

        Reply

  170. I have had this running for a couple days with auto backup on. However, the backups don’t seem to be moving to the storage bucket and the certificate still hasnt worked.

    Reply

    1. Double-check the bucket name and the DNS name in the meta-data. Also verify that the DNS name resolves to the external IP of your VM from some other computer.

      Reply

  171. One major issue I see is that you don’t give much focus on security. By default you allow open wide access to your controller from the internet, wide access to port 22 (ssh), wide access to port 3389 (RDP) and ICMP. There is no need to keep these enabled unless you have a specific use case and you are locking it down. You should include in your tutorial to disable the following firewall rules and tweak the unifi policy to only allow from specific IP address or a subnet. This will also cut down on the Fail2Ban login attempts/exploits from the internet.

    https://i.imgur.com/f7bGFD7.png

    Reply

    1. I disagree. I wrote the script and article for users who are not familiar with cloud services or Unix, or even networking. Great many people have no idea what is their public IP address and whether it is subject to change. They would just lock themselves out.

      Google’s default configuration will only allow SSH using public key authentication. The key and account are created only when you press the SSH button in the console and deleted afterwards. There is no way to log in at other times (root login is not allowed). I consider SSH safe with these settings. RDP and ICMP are also open by Google default rules. There is no service on the Linux VM listening on RDP so there is no access. Blocking ICMP would stop pings, which most users use to test connectivity, causing more confusion. Very little security is gained by disabling ICMP, that’s why it is open in Google’s default configuration (not mine).

      Yes, you can tighten the firewall rules as you see fit. Close also the UniFi ports you don’t use: captive portal comes to mind (ports 8880 and 8843), 6789 if you don’t need the speed test and so forth.

      Reply

  172. Just wanted to say thank you for this!

    Ive been using this for about 5 months now and its been working flawless!

    Reply

  173. First off, this is a GREAT resource you have created. Kudos.

    I have a question about *sizing* the Google Storage Bucket. If I have a 10GB disk on the server, how large of a Storage Bucket do I need? In other words, is Unifi snapshotting the *entire disk*, empty and used space both or just tarballing the configuration itself?

    Since buckets accessed frequently are not cheap I was hoping to find some guidance from those of you who’ve been running for a while. Thanks.

    Reply

    1. The bucket is only used for UniFi Controller backups, which contain only the MongoDB database. It doesn’t contain the entire disk. The size depends on the size of your network and the amount of history you want to keep. You can edit these in the controller: Settings > Backup. The local backup folder is rsynced with the bucket every night.

      Reply

  174. Hi Petri! I just set this up a few months ago because my Cloud Key kept failing and it works great. However, I have been getting the following charges:
    Compute Engine Micro Instance with burstable CPU running in Virginia: 700.934 Hours: $6.03
    Compute Engine Storage PD Capacity in Virginia: 13.992 Gibibyte-months: $0.62
    Compute Engine Network Internet Egress from Virginia to Americas: 0.376 Gibibytes: $0.05

    Any thoughts where I maybe misconfigured or why these charges would show?

    Thank you!

    Reply

    1. Move your setup to some other U.S. region. Virginia is not included in the free tier for some reason.

      Reply

      1. Thanks Petri! What would be the best way to move the setup? Create a new instance delete the old one?

        Reply

        1. Yes. Take a backup of the old one, first. You can’t use the same IP address so it may take some time before the DNS settles and you’ll get a new certificate.

          Reply

  175. Thank you so much for this project! I really appreciate the care that you have taken with it and with us newbies.

    This of course means that I am having a problem: I am unable to connect to the Unifi Controller.

    I am using a static IP address and have created an A Record and get a response when I ping unifi.myobscureddomain.com.

    I hope that you may understand the link between the problem and my Serial port 1 (console) output.

    Thank you in advance!

    First Restart after setup:
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    [[0;32m OK [0m] Started OpenBSD Secure Shell server.

    Apr 5 17:14:42 unifi systemd[1]: Started OpenBSD Secure Shell server.

    Apr 5 17:14:43 unifi google-clock-skew: INFO Starting Google Clock Skew daemon.

    Apr 5 17:14:43 unifi startup-script: INFO Starting startup scripts.

    Apr 5 17:14:43 unifi google-accounts: INFO Starting Google Accounts daemon.

    Apr 5 17:14:43 unifi google-networking: INFO Starting Google Networking daemon.

    Apr 5 17:14:43 unifi google-clock-skew: INFO Clock drift token has changed: 0.

    Apr 5 17:14:43 unifi startup-script: INFO Found startup-script-url in metadata.

    Apr 5 17:14:43 unifi startup-script: INFO Downloading url from https://storage.googleapis.com/petri-unifi/startup.sh to /startup-o1_uc669/tmpt2xwaood using authentication token.

    Apr 5 17:14:43 unifi network-setup: INFO Disabling IPv6 on Ethernet interface: [‘eth0’].

    Apr 5 17:14:44 unifi network-setup: INFO Calling Dhclient for IPv6 configuration on the Ethernet interfaces [‘eth0’].

    Apr 5 17:14:44 unifi dhclient[831]: Internet Systems Consortium DHCP Client 4.3.5

    [ 9.375229] google_network_daemon[803]: Internet Systems Consortium DHCP Client 4.3.5

    Apr 5 17:14:44 unifi google_network_daemon[803]: Internet Systems Consortium DHCP Client 4.3.5

    Apr 5 17:14:44 unifi dhclient[831]: Copyright 2004-2016 Internet Systems Consortium.

    Apr 5 17:14:44 unifi google_network_daemon[803]: Copyright 2004-2016 Internet Systems Consortium.

    [ 9.376870] google_network_daemon[803]: Copyright 2004-2016 Internet Systems Consortium.

    Apr 5 17:14:44 unifi dhclient[831]: All rights reserved.

    Apr 5 17:14:44 unifi google_network_daemon[803]: All rights reserved.

    [ 9.378589] google_network_daemon[803]: All rights reserved.

    Apr 5 17:14:44 unifi google_network_daemon[803]: For info, please visit https://www.isc.org/software/dhcp/

    [ 9.397257] google_network_daemon[803]: For info, please visit https://www.isc.org/software/dhcp/

    Apr 5 17:14:44 unifi dhclient[831]: For info, please visit https://www.isc.org/software/dhcp/

    Apr 5 17:14:44 unifi systemd[1]: Time has been changed

    Apr 5 17:14:44 unifi google_network_daemon[803]: no link-local IPv6 address for eth0

    [ 9.408861] google_network_daemon[803]: no link-local IPv6 address for eth0

    Apr 5 17:14:44 unifi google_network_daemon[803]: If you think you have received this message due to a bug rather

    [ 9.409677] google_network_daemon[803]: If you think you have received this message due to a bug rather

    Apr 5 17:14:44 unifi google_network_daemon[803]: than a configuration issue please read the section on submitting

    [ 9.410948] google_network_daemon[803]: than a configuration issue please read the section on submitting

    Apr 5 17:14:44 unifi google_network_daemon[803]: bugs on either our web page at http://www.isc.org or in the README file

    [ 9.412040] google_network_daemon[803]: bugs on either our web page at http://www.isc.org or in the README file

    Apr 5 17:14:44 unifi google_network_daemon[803]: before submitting a bug. These pages explain the proper

    [ 9.414843] google_network_daemon[803]: before submitting a bug. These pages explain the proper

    Apr 5 17:14:44 unifi google_network_daemon[803]: process and the information we find helpful for debugging..

    [ 9.415488] google_network_daemon[803]: process and the information we find helpful for debugging..

    Apr 5 17:14:44 unifi google_network_daemon[803]: exiting.

    [ 9.416154] google_network_daemon[803]: exiting.

    Apr 5 17:14:44 unifi dhclient[831]:

    Apr 5 17:14:44 unifi systemd[1]: apt-daily.timer: Adding 2h 40min 33.015545s random time.

    Apr 5 17:14:44 unifi dhclient[831]: no link-local IPv6 address for eth0

    Apr 5 17:14:44 unifi systemd[1]: apt-daily-upgrade.timer: Adding 17min 49.428703s random time.

    Apr 5 17:14:44 unifi dhclient[831]:

    Apr 5 17:14:44 unifi systemd[1]: certbot.timer: Adding 3h 22min 15.250929s random time.

    Apr 5 17:14:44 unifi dhclient[831]: If you think you have received this message due to a bug rather

    Apr 5 17:14:44 unifi dhclient[831]: than a configuration issue please read the section on submitting

    Apr 5 17:14:44 unifi dhclient[831]: bugs on either our web page at http://www.isc.org or in the README file

    Apr 5 17:14:44 unifi dhclient[831]: before submitting a bug. These pages explain the proper

    Apr 5 17:14:44 unifi dhclient[831]: process and the information we find helpful for debugging..

    Apr 5 17:14:44 unifi dhclient[831]:

    Apr 5 17:14:44 unifi dhclient[831]: exiting.

    Apr 5 17:14:44 unifi network-setup: WARNING Could not release IPv6 lease on interface [‘eth0′].

    Apr 5 17:14:44 unifi google-clock-skew: INFO Synced system time with hardware clock.

    Apr 5 17:14:44 unifi google-accounts: INFO Adding user dhubnt to the Google sudoers group.

    [ 9.479732] google_accounts_daemon[805]: Adding user dhubnt to group google-sudoers

    Apr 5 17:14:44 unifi google_accounts_daemon[805]: Adding user dhubnt to group google-sudoers

    Debian GNU/Linux 9 unifi ttyS0

    unifi login: Apr 5 17:14:46 unifi dbus[478]: [system] Activating via systemd: service name=’org.freedesktop.timedate1′ unit=’dbus-org.freedesktop.timedate1.service’

    Apr 5 17:14:46 unifi systemd[1]: Starting Time & Date Service…

    Apr 5 17:14:46 unifi dbus[478]: [system] Successfully activated service ‘org.freedesktop.timedate1’

    Apr 5 17:14:46 unifi systemd[1]: Started Time & Date Service.

    Apr 5 17:14:46 unifi startup-script: INFO startup-script-url: Localtime set to America/Vancouver

    Apr 5 17:14:46 unifi systemd[1]: Stopping System Logging Service…

    Apr 5 17:14:46 unifi systemd[1]: Stopped System Logging Service.

    Apr 5 17:14:46 unifi systemd[1]: Starting System Logging Service…

    Apr 5 17:14:46 unifi systemd[1]: Started System Logging Service.

    Apr 5 17:14:46 unifi systemd[1]: Reloading.

    [ 11.999297] systemd[1]: certbot.timer: Adding 2h 23min 30.390866s random time.

    [ 12.010631] systemd[1]: apt-daily.timer: Adding 11h 8min 2.085668s random time.

    Apr 5 17:14:46 unifi systemd[1]: apt-daily-upgrade.timer: Adding 26min 51.592375s random time.

    Apr 5 17:14:46 unifi systemd[1]: unifi-backup.timer: Adding 25min 44.657127s random time.

    Apr 5 17:14:46 unifi systemd[1]: Started Daily backup to dh-unifi-backup timer.

    Apr 5 17:14:46 unifi startup-script: INFO startup-script-url: Backups to dh-unifi-backup set up

    Apr 5 17:14:46 unifi systemd[1]: Stopping Lighttpd Daemon…

    Apr 5 17:14:46 unifi systemd[1]: Stopped Lighttpd Daemon.

    Apr 5 17:14:48 unifi startup-script: INFO startup-script-url: Saving debug log to /var/log/letsencrypt/letsencrypt.log

    Apr 5 17:14:48 unifi startup-script: INFO startup-script-url: Plugins selected: Authenticator standalone, Installer None

    Apr 5 17:14:48 unifi startup-script: INFO startup-script-url: Obtaining a new certificate

    Apr 5 17:14:49 unifi startup-script: INFO startup-script-url: Performing the following challenges:

    Apr 5 17:14:49 unifi startup-script: INFO startup-script-url: http-01 challenge for unifi.myobscureddomain.com

    Apr 5 17:14:49 unifi startup-script: INFO startup-script-url: Waiting for verification…

    Apr 5 17:15:02 unifi startup-script: INFO startup-script-url: Cleaning up challenges

    Apr 5 17:15:02 unifi startup-script: INFO startup-script-url: Failed authorization procedure. unifi.myobscureddomain.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://unifi.myobscureddomain.com/.well-known/acme-challenge/0gSvJmhY-IQURHd_j17oL9O92goax9kW-vc99fMtI-w: Timeout during connect (likely firewall problem)

    Apr 5 17:15:02 unifi systemd[1]: Starting Lighttpd Daemon…

    Apr 5 17:15:02 unifi systemd[1]: Started Lighttpd Daemon.

    Apr 5 17:15:02 unifi startup-script: INFO startup-script-url: Return code 0.

    Apr 5 17:15:02 unifi startup-script: INFO Finished running startup scripts.

    Apr 5 17:15:02 unifi systemd[1]: Started Google Compute Engine Startup Scripts.

    Apr 5 17:15:09 unifi unifi.init[669]: Starting Ubiquiti UniFi Controller: unifi.

    Apr 5 17:15:09 unifi systemd[1]: Started unifi.

    Apr 5 17:15:09 unifi systemd[1]: Reached target Multi-User System.

    Apr 5 17:15:09 unifi systemd[1]: Reached target Graphical Interface.

    Apr 5 17:15:09 unifi systemd[1]: Starting Update UTMP about System Runlevel Changes…

    Apr 5 17:15:09 unifi systemd[1]: Started Update UTMP about System Runlevel Changes.

    Apr 5 17:15:09 unifi systemd[1]: Startup finished in 1.472s (kernel) + 33.865s (userspace) = 35.337s.

    Reply

    1. Looks like Let’s Encrypt servers can’t resolve unifi.myobscureddomain.com. It must to resolve to the public, external IP of the VM from anywhere on the Internet. Check it with nslookup unifi.myobscureddomain.com from you work, local library, in-laws…

      Reply

      1. Thank you for the quick response, Petri. I use Cloudflare, so assumed it’d be quick. Testing with https://www.whatsmydns.net/#A/unifi.myobscureddomain.com works worldwide, so I shut down and restarted the VM.

        Here are the errors from that attempt:

        Apr 6 08:37:54 unifi unifi.init[680]: Starting Ubiquiti UniFi Controller: unifi.

        Apr 6 08:37:54 unifi systemd[1]: Started unifi.

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: An unexpected error occurred:

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: Traceback (most recent call last):

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connection.py”, line 159, in _new_conn

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: (self._dns_host, self.port), self.timeout, **extra_kw)

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/util/connection.py”, line 80, in create_connection

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: raise err

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/util/connection.py”, line 70, in create_connection

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: sock.connect(sa)

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: socket.timeout: timed out

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: During handling of the above exception, another exception occurred:

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: Traceback (most recent call last):

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 600, in urlopen

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: chunked=chunked)

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 343, in _make_request

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: self._validate_conn(conn)

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 841, in _validate_conn

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: conn.connect()

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connection.py”, line 301, in connect

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: conn = self._new_conn()

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connection.py”, line 164, in _new_conn

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: (self.host, self.timeout))

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: urllib3.exceptions.ConnectTimeoutError: (, ‘Connection to acme-v02.api.letsencrypt.org timed out. (connect timeout=45)’)

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: During handling of the above exception, another exception occurred:

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: Traceback (most recent call last):

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/requests/adapters.py”, line 449, in send

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: timeout=timeout

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/connectionpool.py”, line 638, in urlopen

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: _stacktrace=sys.exc_info()[2])

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: File “/usr/lib/python3/dist-packages/urllib3/util/retry.py”, line 398, in increment

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: raise MaxRetryError(_pool, url, error or ResponseError(cause))

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=’acme-v02.api.letsencrypt.org’, port=443): Max retries exceeded with url: /directory (Caused by ConnectTimeoutError(, ‘Connection to acme-v02.api.letsencrypt.org timed out. (connect timeout=45)’))

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: During handling of the above exception, another exception occurred:

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host=’acme-v02.api.letsencrypt.org’, port=443): Max retries exceeded with url: /directory (Caused by ConnectTimeoutError(, ‘Connection to acme-v02.api.letsencrypt.org timed out. (connect timeout=45)’))

        Apr 6 08:38:20 unifi startup-script: INFO startup-script-url: Please see the logfiles in /var/log/letsencrypt for more details.

        Reply

        1. Those errors are from Python. I haven’t used Python in the script. Let’s Encrypt tool is written in Python so it is the probable culprit. Instead of debugging this I suggest you create a new VM. There is something wrong with the current. It could be a user error or a stray cosmic ray has flipped a bit.

          Reply

  176. Hello, I’m trying to use filezilla to connect to my unifi controller which is stored in the google cloud free tier. I keep getting this error msg FATAL ERROR: No supported authentication methods available (server sent: publickey)

    Here are my settings

    Host: sftp://unifi address , user/pass: found in controller, port: blank

    When i add the port i get this msg FATAL ERROR: Remote side unexpectedly closed network connection

    What am i doing wrong?

    Thanks for any advice!

    Reply

    1. You can’t connect to the VM by default. There are no user accounts. The user account is created every time you press the SSH button in the console and deleted when you log out. Of course you can create a new account and a public key for it, but it is a possible attack vector for intruders. You can also give the account a password and edit SSH config to allow for password logins, but that makes it even weaker.

      Reply

        1. You need to use the command line on the VM. You could either go the way I described earlier: create a user, add a password, configure SSH to allow password logins and reverse the changes afterwards. Another option would be to put the OpenVPN config on some public web server (a free blog platform will do) and download the file to the VM using wget or curl. I haven’t used OpenVPN like that, but if the config file is a text file you could just use nano to create the file and paste the contents to nano.

          Reply

  177. Hi Petri,

    Thanks for your brilliant guide. Everything works great.

    I am trying to stop lightttpd as suggested but get the following error:
    $ sudo systemctl stop lighttpd
    sudo: systemctl: command not found

    Any suggestions?

    Reply

    1. That’s definitely odd. Systemctl should be in /bin which is in every user’s path. You can verify the path with echo $PATH. Try ls -l /bin to see if it’s there. If it is you can try full path sudo /bin/systemctl stop lighttpd.

      Reply

  178. Hello Petri (and everyone else). It’s now the 13th of April. Google indicated they will begin charging us for external IP’s starting 1 April. Is anyone running the F-1 Micro VM getting charged yet? Hopefully we are exempt, but thought I’d ask.

    Reply

  179. I’ve had this email from Google today:

    Hello Google Cloud Customer,

    As COVID-19 continues to affect our world, we’re writing to let you know that we have extended the external IP discount period that was scheduled to end on March 31, 2020 until June 30, 2020.

    For background, we increased the price for the use of external IPs with Google Compute Engine (GCE) instances beginning January 1, 2020. To help you quantify the impact of the pricing changes, we automatically discounted your usage for the first three months.

    We are now extending this initial discount period by three months, until June 30, 2020. Your invoice will continue to show your calculated external IP-related charges and the applied discount. You will need to pay for any incurred external IP-related charges shown on your invoice beginning July 1, 2020.

    Reply

    1. As far as I know the first IP address is and will continue to be included in the Free Tier.

      Reply

    1. Look in /var/lib/unifi/sites/
      You’ll need to Force Provision the USG to make it load the file.

      Reply

  181. Great tutorial Petri. GCP have added Debian 10 Buster to their VM images and it’s the default option. The controller wouldn’t boot. I repeated, selecting Debian 9 Stretch instead and it’s booting now.

    Reply

    1. Oh boy, do I wish Ubiquiti would update their software. We are stuck at Stretch because the controller requires Java 8 and MongoDB. Neither are available for Buster. Yes, I have seen DIY projects and you can install them on Buster, but it is not supported. As long as Stretch is supported and Java 8 and MongoDB are supported on Stretch, it is more secure and safer to run Stretch.

      Reply

  182. My storage bucket has all of the .UNF controller updates. I deleted them and there were all copied back over to the bucket with the next sync. Where are those stored in the VM? Can I delete them from there? I have 13 files going back to 5.9.29

    Reply

    1. The automatic backups are in /var/lib/unifi/backup/autobackup, but it is silly to delete them from there, either. The controller will just keep creating them. The proper solution is to limit the number of backups in the controller backup settings. (No command line req’d)™

      Reply

      1. These are not the backups that are in the storage bucket\autobackup folder. I do limit them in the controller. This is the root of the storage bucket. They are all 1.15MB and named:
        5.10.17.unf
        5.10.19.unf
        5.10.20.unf
        5.10.21.unf
        5.10.23.unf
        5.10.24.unf
        5.10.25.unf
        5.10.26.unf
        5.11.39.unf
        5.11.46.unf
        5.11.50.unf
        5.12.35.unf
        5.9.29.unf

        Reply

        1. I hadn’t noticed those. I have them, too. Since there is only one per controller release I assume they were created during controller upgrades (that is my guess). I trust you can safely delete them from the VM and they will not reappear.

          Reply

          1. Thanks for the location. Here is how I removed them via SSH:
            sudo -i
            cd /var/lib/unifi/backup
            rm *.unf
            exit

            Reply

            1. Good!
              As an Unix administrator, I suggest you use the -i switch with rm every time you run rm as root, e.g. rm -i *.unf. Why, you ask. Well, let’s just say I have experience. Once you type rm * .unf by accident, you’ll gain experience, too!

              Reply

  183. Hi Petri,
    For the life of me, I cant get UDP Port 3478 to open up on the GCP. I’ve tested it with many port checkers and it’s closed on my VM. I’ve checked other ports e.g. 22 and they are open. Seeing that STUN warning triangle on my Unifi dashboard is driving me nuts. I know I don’t have to have STUN, but it would be nice. Hence my question: If I delete my current controller and remake it on the GCP, will I lose my static IP address and what does that do to my LetsEncrypt cert? Also, will I be charged for making a new VM?
    Thanks

    Reply

    1. I don’t know what causes the STUN problem and what fixes it. I’ve seen it, but that was a long ago. I thought it was fixed by Ubiquiti somehow. I believe it has more to do with the controller than the VM.
      Answer: If you have a reserved the IP (e.g. it has a name) then it is yours. However, there is a charge when the IP is not in use. You won’t be charged if you first delete the old VM and then create a new VM. Running two VMs will incur a charge. The script will acquire a new certificate from Let’s Encrypt if the DNS name resolves properly. If you have the same IP address there won’t be any delay.

      Reply

      1. Thanks Petri.
        On another topic: How can we determine how big the log file is getting on the VM?
        Is it possible to change the bucket size after already creating the VM? Thanks

        Reply

        1. I believe there are some misconceptions here. The log files are rotated by logrotate and it is controlled by config files under /etc/logrotate.d. The log files are not copied to any bucket. Instead, the backups are copied to the bucket. You decide how often a backup is taken, how many backup files are kept and how large the files are. See Settings > Controller Settings > Backup. (Data Retention Days affects the size of each backup file.) However, the bucket size is unlimited, you just start to pay after the first 5GB, no resizing is possible. If, instead, you meant the Linux system disk size, the answer is no – there is no easy way to resize it. (Yes, a competent Linux sysadmin can do it, but that is left as an exercise to the reader.) It is much easier to just create a new VM with the preferred disk size and restore the latest backup.

          Reply

  184. The issue with Unifi Controller not running on Buster is mostly Java/MongoDB version right?

    I’m testing your script with a small edit to include the Java/MongoDB version check and subsequent reinstallation from GlennR’s install script.

    It’s not a bad solution, but it needs to be manually edited as Ubiquiti doesn’t have an easy place for a script to grab a listing of what versions of MongoDB and Java it requires.

    Thoughts?

    Reply

    1. Yes. The problem is that MongoDB and Java 8 are unsupported on Buster. I wrote my script for non-Linux savvy users. I don’t feel comfortable creating unsupported combinations, even though it is technically possible.
      If you find some good solution, I am interested and willing to adopt it. However, grepping some scripts to determine required versions does not count as a “good solution”.

      Reply

  185. Hey Petri,

    Your ability to write cleanly and simply, with your great knowledge, really made me learn a lot about both GCP and Unifi. I’ve been on Unifi very intimately for years but yet I learned a lot.

    I succeeded on my first try.

    Thank you from Singapore. If you need anything just let me know, sir.

    Reply

  186. I created a new VM on GCP and restored (to fix my Lets Crypt) as per the procedure outlined above. Then I deleted the old VM. Everything was fine but Unifi Cloud kept looking at the old VM but not the new one.

    The fix was I went to Controller settings|Remote Access and toggled `enable remote access’ switch and it worked almost immediately.

    Reply

    1. If you had a reserved IP address the switch should have been transparent (unless there are some credentials exchanged in the background). I don’t know. I don’t use UniFi Cloud myself, because I don’t see the point. I see UniFi Cloud as a means to connect to a controller behind a NAT firewall. If my controller is on GCP and directly available, why would I access it through UniFi cloud?

      Reply

    1. That makes sense, if you need multiple controllers. I have multiple sites on a single controller. With multiple controllers you get another tier, but it requires more maintenance. Whatever suits your needs.

      Reply

  188. I too am now getting STUN errors on unifi after a reboot of the gcp instance. Despite the GCP firewall saying 3478 is open all the port scanners I’ve tried say it is not.

    Petri – I find that over the course of a week or so the controller slows right down, almost to a stop/fail point. I’ve read on unraid forums that the controller versions 5.12.xx have a memory leak, however I’m unable to confirm this via other sources of information. Do you know if this is true and is there anyway of logging memory usage on GCP so we can keep an eye on it?

    Will you be updating the script to the very recently released 5.13.xx any time soon?

    I’d like to stay with GCP as I only have 2 sites and less than 20 devices, it should be sufficient, and indeed has been for almost 18 months. But recently it gets so slow and regularly loses connection with devices from 2 different sites. I’m considering switching to a cheap VPS, or self-host on unraid docker. I’d prefer it to remain in the cloud tho, because then you get notified when something drops offline!

    Thanks for the hard work and continued support

    Reply

    1. The controller will update itself as soon as Ubiquiti releases 5.13.xx to the repository. That’s usually a week or two after the announcement. Hold on to your hat 🙂

      Reply

      1. Ok thanks.

        Any idea on whether or not the current version does indeed have a memory leak?

        Are you using this project in your own set-up?

        Reply

        1. I haven’t seen any ill effects on my personal controller, but the site is really small. A reboot should fix problems caused by a memory leak. You should check the Ubiquiti forums for more information.

          Reply

  189. Petri – thank you for putting this together and maintaining it. I’ve been using it over a year with no issues. Was doing some spring cleaning and decided to do a new VM as you suggest we do periodically. I’m getting the below when the VM starts up. Looking thru previous posts, you reference zone issues maybe(I’m on us-east-1b) and a GRUB bug. I’ve stopped and started the VM a couple of times and the errors below are consistent. Any ideas…?

    Jun 15 14:12:29 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:29 GCEMetadataScripts: startup-script-url: OKDynamic DNS accessed
    Jun 15 14:12:29 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:29 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:29 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:29 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:29 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:29 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:29 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:29 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: The method driver /usr/lib/apt/methods/https could not be found.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: Failed to stop mongodb.service: Unit mongodb.service not loaded.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: Failed to disable unit: No such file or directory
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: /tmp/metadata-scripts606286019/startup-script-url: 167: /tmp/metadata-scripts606286019/startup-script-url: cannot create /etc/fail2ban/filter.d/unifi-controller.conf: Directory nonexistent
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: /tmp/metadata-scripts606286019/startup-script-url: 171: /tmp/metadata-scripts606286019/startup-script-url: cannot create /etc/fail2ban/jail.d/unifi-controller.conf: Directory nonexistent
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: /tmp/metadata-scripts606286019/startup-script-url: 179: /tmp/metadata-scripts606286019/startup-script-url: cannot create /etc/fail2ban/jail.d/unifi-controller.local: Directory nonexistent
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: Failed to reload-or-restart fail2ban.service: Unit fail2ban.service not found.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: E: dpkg was interrupted, you must manually run ‘dpkg –configure -a’ to correct the problem.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: Failed to start dbus.service: Unit dbus.service not found.
    Jun 15 14:12:32 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:32 GCEMetadataScripts: startup-script-url: Trying to start dbus
    Jun 15 14:12:47 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:47 GCEMetadataScripts: startup-script-url: Failed to start dbus.service: Unit dbus.service not found.
    Jun 15 14:12:47 unifi-2 GCEMetadataScripts[644]: 2020/06/15 14:12:47 GCEMetadataScripts: startup-script-url: Failed to create bus connection: No such file or directory

    Reply

    1. That VM is hosed. You should try to create another one, in case that was a transient error. Make sure you select Debian Stretch as OS. I’ll try it too, when I have some spare cycles. Thanks for the report!

      Reply

      1. Petri – I created another VM instance and it worked fine. Not sure what went wrong with the first attempt. Thank you!

        Reply

  190. Hi Petri,

    Great guide! Recently I changed the hostname of the server, but the SSL certificate is still on the old hostname. How do I change this?

    Also there is a stable software update available (system is prompting, version 5.13.29) but it is not installing automatically. Is there a fix for this or do I have to wait a bit longer?

    Reply

    1. Once the new name resolves properly you should update the meta-data of the VM. Then just stop and start the VM to re-run the script.

      Reply

  191. Thank you Petri for the script and tutorial, I’ve used it a couple times now (once for my free trial, and now again after they unceremoniously deleted my VM when it expired). The auto backups worked great and I was able to easily grab the latest config backups from storage before recreating the VM.

    For those having some issues with the certbot certificate issuance, make sure you are NOT putting in your hostname for their new Networking->Hostname field (since it is permanent), only enter the dns-name key/value pair per the instructions. My issue was putting in my FQDN under Hostname which makes Google hard code it into your hosts file with LOCAL address on boot. The certbot script would always resolve to the internal IP for the certificate request when finding it in the hosts file, cancelling out before actually trying to get the cert. I deleted the VM and recreated while leaving the newer Hostname filed blank and the unit got the cert without issue in initial startup.

    Reply

    1. Thanks for the clarification. There are too many ways to go wrong, I can’t anticipate all of them 🙂 A problem report and (hopefully) a resolution may help many others.

      Reply

  192. Unifi controller 5.13.29 was successfully automatically updated today by this script.

    Reply

  193. I have a domain in google domains, I can create dynamic dns it generates a username and password where I put them

    Reply

    1. If you can’t figure out a ddns-url then you can manually make the domain name resolve to the external IP address of your VM.

      Reply

      1. Sorry my English, I loved your post, as an addition if you have a domain in google domain Scroll down to Synthetic Records then Dynamic DNS. obtain your credentials, you’ll need them for the next step. To do this, expand the arrow beside Dynamic DNS and then click View Credentials. Make note of your Username and Password
        it does not generate the ddns-url for you automatically but this is the nomenclature https: // $ {USERNAME}: $ {PASSWORD} @ domains.google.com / nic / update? hostname = $ {dns-name} & myip = $ {IP }
        of course it only works if you have a static ip from your isp

        Reply

        1. No, this has nothing to do with the IP address you get from your ISP. The DNS name must resolve to the external IP address of the VM in GCP.

          Reply

          1. eso explica por que DNS_PROBE_FINISHED_NXDOMAIN jajaja, cambie la variable final al ip de gcp y funciono thanks petri

            Reply

    2. If you have a google domain, you need to do a forwarding on your Google domain console.
      1- Google domains
      2- DNS
      3- Add
      Subdomain forward / Permanent redirect (301) / Foward path / enable SSL
      it will look like: your_Subdomain.yourDoamin.com -> your Unifi external IP address

      4- Once you finish the steps above, you need to use the following settings:
      startup-script-url gs://petri-unifi/startup.sh
      timezone YourTimeZone
      dns-name your_Subdomain.yourDoamin.com
      bucket YourBucket

      Reply

      1. Thanks for your input. I haven’t used Google domains, so I have no experience.

        Reply

  194. hello it’s me again I already made it work with dns dinamy from google domains, the thing is that I can’t enter example dns.dinamyc.com but if I can enter with dns.dinamyc.com:8443 the console doesn’t allow me to register the latter

    Reply

    1. I’m not certain I can follow you, but the port number is not part of the domain name.

      Reply

  195. is there a security risk use set-inform http: //your.server.dns: 8080 / inform being http and not https?

    Reply

    1. No, the connection is authenticated otherwise. Setting up TLS for every inform update would eat too much processing power.

      Reply

  196. Hello Petri, Thank you for this script, write up and support. This has mad things very seamless. I do have a question. I would like to change dns url for my VM. How do I do this and get a new certificate? Thank you again for this!

    Reply

    1. Just edit the metadata in GCP console and stop/start the VM to reboot. The script runs only at boot.

      Reply

      1. the metadata such as ( dns-name | your. site.here ) ? Does it only appear when the vm is stopped? Thank you again Petri for your experties!

        Reply

        1. Yeah, you are right. You need to first stop the VM, then edit the metadata and finally start it again.

          Reply

  197. Hi. I have used your previous unifi controller setup last year and now its about to expire in3 days–Thanks for your help.
    Now I need your help again as i transferred the image to other gcp account but I cant be able to make it work– is it going to be FREE tier again or not?

    Reply

    1. You don’t need a new account. You just need to reconfirm your credit card details in case you exceed the free tier. The free tier includes one micro-sized VM in an U.S. zone and enough resources to run the controller. It has been free for years now and seems to continue to be for the time being.

      Reply

    1. Yes, you are right. (I hadn’t paid attention to that announcement. AC-IW-Pro was a nasty surprise. Fortunately I haven’t deployed a single one IIRC. Plenty of AC-IWs, though)

      Reply

  199. Hello again Petri,
    My controller is growing with sites and devices. I wonder how many sites this micro instance can support? Also, If I were to put this controller on a higher hardware tier would the script still work? If so what configuration do you recommend? As always I really appreciate your work on this. How can I repay you? Do you have Patreon or something similar?

    Reply

    1. The performance is close to the 1st gen. Cloud Key – around 30 devices. After that you need more memory and processing power, disk space shouldn’t be an issue. The same script will do just fine. If you have 4GB or more of memory then the script won’t create a swap file. That will free up disk space. If you have more than 4GB memory you should increase the memory available to Java. There is a commented out section in the script that works. It will add a couple of meta-data fields for the purpose. Or you can do it by hand. Ubiquiti has a high performance tuning document.

      Nah, this is a pro bono project. I wouldn’t expect more money from donations that the trouble of setting it all up. Enjoy!

      Reply

  200. Hi Petri,

    Thanks for a great guide!

    I had some issues when importing the certificate to the keystore but i found after some Googleing that removing all line breaks in the pem-files solved the issue.
    I suspect that when Certbot is renewing the cert this issue will come back, but i guess it’s possible to automate the editing the files to be “one line” in some way.

    Reply

    1. What kind of issues did it solve? That would be easy to implement with sed.

      Reply

      1. Thank you for your answer. I’m asking becasue I have hard time moving my AP-AC-LR to the new controller. I loaded my local controller backup into GCP instance, logged in, first I changed Controller Hostname/IP on GCP, then I changed Controller Hostname/IP on my local controller.

        Problem is, my local controller started to adopting device over and over again. AP never reached GCP controller, even once.
        As a last resort I logged in into AP via SSH and run “set-inform http://your.server.dns:8080/inform“.

        Still nothing.

        Reply

        1. One more thing. Of course on GCP I can see my AP – because I loaded backup – but it has “disconnected” status all the time.

          Reply

        2. You are doing something wrong. The APs should be adopted to your local controller. When you move the controller backup to the GCP controller, it will assume the identity of the local controller. Then you should change the inform-host on the GCP controller to point to itself. When you make the same change on the old, local controller, all the devices will be redirected to the new controller. The controller identity will match so the devices will just assume the IP of the controller has changed. Everyone is happy (esp. the administrator!) You can turn off your old, local controller.

          If you haven’t changed the inform-host on the GCP controller, then it will redirect every device back to the old controller. Under no circumstances should an adopted device be re-adoptable. As a matter of fact, it is a pain if you have lost the controller. An adopted device is tied to the identity on the controller and won’t talk to anyone else. One scenario is where the local controller is redirecting the devices to the cloud and the cloud controller is redirecting them back to the local controller.

          Reply

          1. Hello Petri

            I redo everything and it works fine 😉 Thank you.

            Today I recieved notification about controller 5.14.22 update.

            I’ll better wait 😉

            Reply

  202. Hello Petri, it is safe to upgrade controller to new version 6.0.13?

    kind regards

    Fabrizio

    Reply

    1. You are apparently not familiar with Ubiquiti’s reputation with software and release quality control? 😁 I am not going to rush it. Fortunately Debian repos are updated last, so Ubiquity has time to pull back the release if it breaks havoc among more eager users. You can install 6 manually and try it out if you like, I don’t see any reason why it wouldn’t work. Just take and store a backup before the upgrade. You can always create a new series 5 controller if it goes south, but you can’t restore a backup taken from a later version. You will need the old backup.

      (I don’t get it. Version 6 still requires Java 8 and MongoDB, which are not supported on later Debian releases. We are stuck on Stretch and its support is running out. I would have put more effort on the controller infrastructure instead of any nifty new features, but I am an responsible administrator.)

      Reply

      1. Thanks for the explanation Petri, you are always very kind.
        I upgrade my controller downloading the unifi_sysvinit_all.deb package via wget link.
        I only have one AP so my configuration is very basic and lends itself to any experimentation.
        I am an early adopter and would have been happy to try the new 6 series now but I’ll wait.

        kind regards again

        Fabrizio

        Reply

        1. Sure you can try it out. Just take a backup before you upgrade and store it somewhere. Let us know how it went.

          Reply

  203. Hello, I’ve had my controller up and running now for over a year, thanks for guide! I’ve been trying to log in now since yesterday online and mobile with no luck. I’ve tried stopping and restarting the instance but that doesnt seem work? looks like I’m still on version 5.13.32, any ideas on i can do to fix?
    Thanks!

    Reply

    1. The quick fix is always to create a new VM and restore the latest backup from the bucket. That will take you less than 10 minutes and is painless if you have reserved the IP. You can access the bucket from the GCP console. The date on the last backup will give a hint as to when things went south.

      Reply

        1. You can access your UniFi backups in GCP Console > Storage > Browser. Look for a folder named autobackups inside your backup bucket.

          Reply

          1. looks like i dont have any buckets created? i must of missed a step in your instructions. Is there another option?

            Reply

            1. You can still create a bucket and add it to the VM metadata. Starting the VM will re-run the script. If your VM is still functional, it will copy the backups to the bucket overnight (not instantly!). If that doesn’t work, you can log in to the VM and look for the backups in /usr/lib/unifi/data/backup/autobackup/ (you’ll need root privs). You can then copy a backup to your bucket with scp autobackup_xxxxx.unf gs://yourbucket/backup.unf. The latter requires some Linux shell skills and is left as an exercise to the reader.

              Reply

              1. Thanks Petri, that’s a bit over my head. If i wanted to move back to locally hosting on my home PC what would be the process? i know how to migrate to another site, just need that unifi backup to restore to. appreciate your time and patience

                Reply

                1. If you have the original backup it should still work for this. There are two requirements: You need a controller (local or cloud) which has the same identity that was used when adopting the devices. That’s why you need the backup. Second, you need to direct your devices to this controller. You can do this in the controller, if you have a functional one. If you don’t have, then you can also do this via ssh to each device and issuing set-inform command.

                  Reply

  204. HI Petri,
    I’m hoping you can clarify something.
    I am being charged for egress traffic (a whopping $0.02). Despite the 2 cents, my bigger concern is the 1.27GB egress traffic from America to America. Not sure why that is happening. It could be me, but I only access my controller every couple of days (I do have icons for each client). I know you said we can turn off Lighttpd as a fix for large egress to overseas (I assume America as well). So here’s my question: By turning off Lighttpd, does that stop egress bots hitting my URL or will that stop bots hitting my IP address as well? I know bots will hit a range of IP’s, but haven’t heard of bots hitting URL’s. If Lightpps only stops URL’s, it seems the bots still have the upper hand by pinging a range of IP’s. Hope my question makes sense. Thoughts?

    Reply

    1. Your egress traffic should consist of your devices downloading firmware (very occasionally), you downloading backups (often!), and you accessing the user interface. There are many bot networks also in America and I assume they are the culprit. The connections are always made to the IP address. The DNS name in the URL is resolved before connection. However, the bots won’t try port 8443, they will try port 80. If you don’t have Lighttpd then they can’t connect at all. The only purpose of Lighttpd is to forward all connections to port 80 to port 8443. I added Lighttpd since many users couldn’t remember to type https at the front and :8443 at the end.

      Reply

      1. Got it. Forgot about the ports being needed if you turn off Lightpd. Makes sense now. Thanks for clarifying.

        Reply

  205. Hi Petri, I have been using the controller on google for some time now. I like it very much. It always works and helps me support my family and their wifi. Today I got an email saying my free period will expire in 30 days. That’s unfortunate, but I would be happy to pay. If only I had an idea how much. I found some prices, but they are time based. Sounds like it could work out expensive. As the controller always works I never look on the google cloud console. I started looking around today, and in my storage resources I see the cloud storage which holds 5.11.46.unf up to 5.13.32. Am I maybe using more storage than needed? Should I do more housekeeping in the google console? I have no knowledge of VM or the google platform. I just got this up and running following your ,fantastic, steps.

    Reply

    1. You need to reconfirm your payment information within 30 days. However, the F1.micro instance has been free for years and you have free 5GB for your backup bucket. If you search this page you’ll find that some users have reported $.02 to $0.10 monthly charges for excessive traffic due to cyber attacks. I presume your charges will total less than a dollar a year, but of course Google can change their pricing any time. That’s why you should keep an eye on the notification emails. See https://cloud.google.com/free for current free trier offering.

      Reply

  206. Thanks for the beautiful script and video. followed thoroughly and got unifi setup in 1hour !! Well done Mate!

    Reply

  208. Hi Petri! Love this solution it has worked well the last year or so!
    Unfortunately I logged in last night and the controller was down.
    I log into my google cloud console and it looks like the CPU usage shot up at around 2 AM the previous night and never went down.
    I tried rebooting the Instance and the CPU usage went back down. but I still cannot access the controler. I’m wondering if an update may have broken something. Is there a way to revert this update?
    thanks for your help!

    Reply

    1. Instead of troubleshooting it, I suggest you download the latest backup from your bucket (in GCP Console it’s in Storage > Browse), create a new VM with the same IP address and restore the backup. You’ll be up and running in 15 minutes.

      Reply

      1. you mean by following your whole guide again?
        thankfully I have a backup i’m just not very familiar with the google cloud console is all.

        Reply

      2. Hi Petri,
        After reading your post, I wanted to see where my backup was located. I found it in Storage>Browser. However, I only have on backup that was created back in 2018 (When I created my VM). Is that normal?

        Reply

              1. At the top level of the bucket there should be backups the Controller created during upgrades. The periodic backups are created in a subdirectory called autobackups. How are your backups configured in the Controller? Can you see the list of backups in the Controller?

                Reply

                1. Ok, found them. Should I download a backup to my PC for safe keeping, or are the backups fine where they are in the GCP (If I need to delete and start over)?

                  Reply

                  1. They should be safer in the cloud. Of course it is wise to occasionally copy one to your PC – just in case.

                    Reply

  209. Hi! Thanks for this amazing tool!.

    I ran the script and everything seems to be great.

    Nonetheless the controller version is 5.14.23 and I always get a message that version 6.02 is already available.

    Should the script have updated the version already?.

    How can I easily update the controller? How do I acces the VM through the console and what are the commands to update?.

    I don’t use linux but I’ve followed some tutorials and have basic usage on my synology NAS so any help would be really appreciated!

    Reply

    1. Version 6 is in beta and not for general use. Ubiquiti has a bad reputation releasing immature software so their betas are even worse. Of course you can install whatever you like using the SSH button in GCP console. I strongly recommend taking a backup before any experiments. The auto-upgrade mechanism will install version 6 when it is released to the repositories.

      Reply

      1. Thanks — I’ll wait for the 6 version.

        But I’ve read that we are already with a more recent stable version than 5.14.23

        Shouldn’t it have updated it automatically already?

        How do I update manually to the latest version with SSH.

        Reply

        1. Mine has upgraded to 5.14.23. Wait overnight and check yours.

          I see that my controller is today advertising a manual upgrade to 6.0.20. I’m not going to switch over until they release it to the repos and the automation will work its magic.

          There is a SSH button in the GCP console. Once you have terminal access you need to download the version you like from UI website and follow the installation instructions. Beware, there is a good reason why I have scripted the unattended upgrade, it takes all the pain and errors away.

          During WWII the Royal Air Force realized they had both brave pilots and old pilots, but no brave old pilots. Can you imagine why?

          Yours truly, an old administrator 🙂

          Reply

  210. Installed a new controller yesterday since I had installed the old on wrong location (Finland) and then got to start pay. Since I am so poor I reinstalled in US East and hope to do it the free way… however. After reinstall, I first got the 5.14.X version but today its the 6.X version… For me it seems to work ok so far. But am I the first one or have a managed to do something strage… no manual update as far as I know of.
    Thanks Petri for a great script and forum!

    Reply

  211. Petri, We have been loving this bootstrapped environment but I am circling back for some help.

    Just today, ui pushed out the 6.0.20 update to their repo. It is a disaster, nothing is working right. So I was able to do an

    sudo apt-get purge unifi -y

    sudo dpkg -i ./unifi_sysvinit_all.deb

    Then I restored from the update and at the very least all the unifi is working again. I also ran “sudo mv /etc/apt/apt.conf.d/51unattended-upgrades-unifi ~/” to make it so the new release is not installed again without my say-so. And just to be extra safe I even did “sudo mv /etc/apt/sources.list.d/unifi.list ~/” followed by “sudo apt-get update; sudo apt-cache policy unifi” to make sure that there is no way it could ever be installed in the near future.

    However there is an issue with the certificate. The “/usr/local/sbin/certbotrun.sh” script is saying: “Unable to import the certificate into keystore”. I ran it manually thinking I could just get certbot to renew.

    After going through your script I am thinking that since I removed the unifi version with the purge command that some of the hooks you put in there have been destroyed. What other aspects of your script should I be re-running to reform the environment around the new installation of unifi.

    Based on my understanding of the script it seems as if lighthttpd is just a redirect. So I am fairly convinced I know that I need to get this certificate hooked back in to the java key store which was probably wiped off when I did the apt-get purge. I figured the purge was my best bet of dumping that mongodb cleanly but I didn’t quite think of the certificate there.

    This 6.0.20 disaster is one many will face. Some I assume will get by unscathed but we have had multiple customers calling in today with issues with their guest wifi. It is unfortunate.

    Let me know if you have any suggestions, and thank you for the script, as I said the environment has been lovely until today when UI so rudely pushed bugs to stable. They have lost the trust they earned!

    Reply

    1. I think I answered my own question. The process worked, the error message was a red herring. I cannot find any evidence that what I did was faulty, it looks like you put all the pre/post/deploy hooks directly and properly into certbot so the error message must just be nonsense. When I load the URL in a different machine or a different browser it is not using the cached certificate and correctly shows a certificate signed by the LetsEncrypt CA.

      Thank you. I hope this post is helpful for anyone running into this 6.0.20 hell.

      Reply

      1. Good to hear you are in the clear now. My personal controller has also upgraded to 6.0.20 but it seems to be working fine.

        I dunno about any trust UI has earned. In my experience their quality control sucks. Yet, I have received multiple requests to install version 6 ever since the first betas.

        Reply

    2. Mark,

      I am sorry to bump this post, I asked Petri and he suggested your post, I had ready your post earlier but for me it sounded a bit difficult. I am in no way as advanced of a user as you! Would you be willing to send me a descriptive outline of exactly what you did? I need to get my cloud key back up again after this ridiculous 6.0.20 and 6.0.22 updates. Any help would be great, thanks!!!

      Reply

  212. Hi Petri!
    First of all, thank you for this excellent script and guide.
    A quick suggestion regarding the backup bucket and the VM-setup instructions.
    I think that in GCP the VM is created without the proper permissions for Cloud API access scopes, Storage is read only, and then the backup wont work.
    If the machine is created with out the proper permissions, then it’s easy to stop it, edit them and then reboot it. However, one then also needs to delete all files in /root/.gsutils to force the new permissions to be updated.

    Cheers
    Adam

    Reply

    1. In step 4, at the end of the second paragraph, the instructions say “change Storage to Read Write“. If you missed that, my recommendation is always to create a new VM. Usually you can fix things through the command line, but I can’t support everyone.

      Reply

  213. Hi Petri,
    at 6:00 a.m. EDT on 18 Sep, my Controller is no longer accessible. Is this due to the new roll-out of Version 6? If so, it broke my connection to the controller. Any thoughts on what to do?

    Reply

  215. Petri,

    This has been awesome thank you, thank you, thank you!!

    My controller did the auto update to 6.0.2 and I have had no luck getting it working. Been trying since they pushed the up date. It even updated to the 6.0.22 and it has not helped.

    My question to you is what is the best and simplest way to make my controller rollback to 5.0.14? I have backups back to about a month. I wish I was more experienced but I am always learning. If you could pass along some helpful insight as to rollback my controller that would be awesome.

    Again thanks for this wonderful workup it has been awesome. Look forward to hearing from you!!!!

    Reply

    1. I’m sorry to hear. Mark Amber reported just recently how he was able to remove the latest version with apt-get purge, install the one he had manually downloaded and prevented the unattended upgrades from touching it. He included all the commands, too. User ctrl-F to search this page. After that you need to restore a backup from that specific version. Fortunately the backup name includes the version number. G’luck!

      Reply

  216. Thanks for the instructions. I’ve got the Ubiquiti Controller set up but had a problem with APs not finding it to adopt to until I made a back up of my original test Controller and loaded the back up into the cloud controller.

    My question is, will it be like this all the time?

    My original 3 APs (UAP-AC-M) are adopted into the cloud controller but I now have a 4th AP, out of the box, and it can’t find the controller. I’ve already ssh into it and used the set inform command using the DNS name and also the specific ip address at port 8080 and no adoption. Using the DNS name only, the AP says it can not resolve the name and using the IP address, the AP does nothing.

    I can ping the DNS name off the laptop on the same subnet as the new AP so network-wise it is reachable.

    Is there something else I should look into?

    Reply

    1. When you have an SSH connection to the AP, check that the AP can resolve the name and ping the VM.

      Reply

      1. From the AP using SSH, I’m able to ping the Cloud Controller using its DNS name and also using its proper IP. But when I issue the set inform http://(DNS-Name):8080 the AP says “not found” and when I use the IP address instead of the DNS name in the set inform command it just sits and the controller show nothing new to adopt.

        Reply

    2. The proper form is set-inform http://your.ctrl.dns:8080/inform

      You can check the DNS resolution on the device using nslookup bbc.co.uk for example.

      The devices need to know the controller address somehow. If you have an USG then this is taken care of. Alternatively you can set your DNS resolver to resolve name unifi or add an option to your DHCP server. Otherwise you need to do it manually for each device. See Device Adoption Methods for details, the last three methods are DNS, DHCP and SSH.

      Reply

  217. Petri,
    I first wanted to thank you for putting me in touch with Mark, he was very knowledgeable and willing to help! Now that it is up and going again, I am having another issue I wanted to run by you. My VM keeps saying “Instance “unifi-controller” is overutilized. Consider switching to the machine type: custom (1 vCPU, 2.75 GB memory). Learn more” and then it keeps asking me if I want to increase the “perf”. I have increased it once hoping it wouldn’t affect too much, I can see it has increased what google charges me, I am up to about $11 a month now. Does this seem right or do I have something set wrong? Do I need to make a change to something so it wont charge me?

    Reply

    1. You have fallen for upselling! Of course Google wants you to move to a paid tier. The free instance is good for around 30 devices. Don’t believe everything the salesman says 🙂

      Reply

  218. Hi Petri,
    The letsencrypt certificate I got was expired on arrival. How does renewal work?
    Thanks,
    Adam

    Reply

    1. That is certainly odd. The script doesn’t check for that. (You have to trust something.) If it is a new controller, I would suggest deleting the VM and creating a new one. If it used to work and then suddenly received an expired certificate, it should try to renew next night.
      Renewing manually is simple enough, but you need to add the cert to the Java key store, which is not. The steps are at the end of the script, but it is not for the faint of heart. I certainly hope this is just a random glitch with Let’s Encrypt. I know they are making changes to their certificate chain at the moment.

      Reply

  219. I can no longer connect to the VM controller via SSH. I have stopped the VM and restarted it and still not able to connect. Should I just delete the current VM and create a new one. Any other easier options to make this work. I’m also not able to get the controller up via the app or website.

    Reply

    1. This is worrisome. You are the third person reporting symptoms like this. In the other two, the console logs end with a request to run fsck. That means the boot disk is messed up and the system can’t start. There is nothing you or Ubiquiti has done, but the Debian kernel has run amok for some reason. This is very rare unless there is a bug in the operating system. If there is, this will eventually affect all of us.

      I recommend creating a new VM with the same metadata and IP address. Detach the IP from the old VM first. Once the VM has started you can restore the latest backup and you are up and running in 15 minutes. That’s why we have backups. Cloud computing makes recovery quick, but this shouldn’t happen in the first place.

      Reply

      1. Thanks for the quick reply! I started to create a new VM, but sorry for the questions as I’m not all that familiar with this type of setup. The Machine Configuration now has a “Series” option E2, N2 and N2D and First Generation N1. It defaults to E2 and when selection e2-micro it’s showing a monthly charge of 6.72 USD. So do we still have a free option for this?

        Reply

        2. I was able to get a new VM up and going and able to SSH to the VM. But I’m not able to connect to the controller. Getting refused to connect. I did not change anything, just removed the static IP from the old VM and deleted the old VM and attached the static to the new VM. Here are the firewall settings

          allow-unifi
          Ingress
          unifi
          IP ranges: 0.0.0.0/0
          tcp:8443,8080,8880,8843,6789,443,80
          udp:3478
          Allow
          1000
          Off

          Reply

          1. Did you set the same metadata as your previous VM? My first guess is that the startup script didn’t run. You can check to console logs to see if the script installed Java, MondoDb and all the other bits and pieces.

            Reply

            1. Hi Petri!
              I deleted it and recreated the VM and it’s back working. Thanks for all that you do!!

              Reply

      1. No problem.
        But curl, pipe and bash are important in this case.
        I had to replace whole line in `startup-script-url` field to:
        `curl https://storage.googleapis.com/petri-unifi/startup.sh | bash`
        Just replacing `gs://petri-unifi/startup.sh` to `https://storage.googleapis.com/petri-unifi/startup.sh` didn’t work for me.
        Because somehow instance couldn’t execute it at all.
        I think they messed something in images itself.

        Reply

  221. Really nice script just finish migrating my house and office sites to GCP controller and its working great just to note that i was already using No-IP dDNS services and it was a little bit of a trouble due to the password for the string needed to be “base64 encoding”, no big trouble just change password and done… and hope it helps in case someone else is using noip services also

    username:password
    “base64 encoding”
    https://username:password@dynupdate.no-ip.com/nic/update?hostname=mytest.example.com

    Reply

  222. Hi Petri,
    I have my Lighttpd shut off (which helped stop overseas egress). Oddly enough, I was charged $0.05 for month of October. Again, it’s not the Nickle that concerns me. However, for October I had over 1GB egress from Americas to Americas.
    Is there any way to find out where the egress traffic is going to e.g. an IP address? I just want to find out if it’s me, or some bot that sucks down all my data limits. Thanks

    Reply

    1. No, unless you turn on more verbose logging in UniFi Controller and wade through those logs. The controller is the one responding to queries and only it knows who asked for what. I haven’t played with the settings to find out what to log and which level to capture haphazard web queries. tail -f will be your friend when you try to figure it out and grep when you have it running. Set up logrotate to truncate the logs before they fill up your disk.

      Reply

  223. Hey Petri! Do you have this script for UNMS install as well? If not where would I start in modifying this one to install unms? Thank you and happy thanksgiving!

    Reply

    1. No, I haven’t used UNMS myself. To create a script you need to install it several times by hand to see how it can go wrong 🙂 I suggest you just create a similar VM and use the SSH button to get shell access. Then just follow the Linux installation instructions for UNMS.

      Reply

  224. Hi Petri,

    Thank you for such a detailed tutorial in setting this up. Made the leap from my late 2012 Mac mini to this. From start to finish in 45 minutes. Only hiccup was the need to reprovision one AP on the old controller to make it see the new controller.

    Will keep an eye on egress but so far so good.

    Reply

  225. Hi Petri,
    Ubiquiti has announced end of software support for certain models of Unifi equipment, like the original Unifi AP. Continuing to update the controller software will cause these devices to become read-only in the controller, essentially orphaning them. Is there an easy way to prevent the Unifi controller from being upgraded?

    Reply

    1. Open an SSH connection. Then remove the line c=ubiquiti from /etc/apt/apt.conf.d/51unattended-upgrades-unifi. Probably the easiest command is sudo nano /etc/apt/apt.conf.d/51unattended-upgrades-unifi Use ctrl-x and Y to quit Nano. Then sudo reboot to enforce the change. (Or replace your APs – depending on your definition of “easy”)

      Reply

  227. Hi there!
    thank you for what you are doing.
    Im having an issue lately where my unifi controller has been eating hard drive space in the google cloud.
    I upped the space by an extra 10gb and the message stopped for a few days. now its starting up again. something is eating space and im not sure what. would you be able to help out?

    Reply

    1. I haven’t seen this, so there is something odd going on. If you have increased the disk size then you must have restarted the VM, but apparently the problem didn’t go away. Next you need to SSH into the command line. Google how to use cd and pwd to move around in the directory tree. Use du -sh * to see how much space the directories occupy. If it is a log file that is not being rotated then it should be somewhere under /var/log, but start with /var. Another suspect location would be /usr/local. Running du for root directory / may take a while, but it will point you to the right direction.

      If you are not comfortable with command line, then just download an up to date backup, create a new VM using the same IP address and restore the backup. This problem appears to be specific to this particular VM, thus creating a fresh VM should solve it.

      Reply

  228. Hi there,

    Thanks for your article, it helps me a lot, now I can access my sites from anywhere, anytime.

    I have 2 major questions.

    1. Why is remote access not working with GCP? The status is always “Connecting”, yet I’m not able to see my sites/devices on network.unifi.ui.com.

    2. I am intend to buy UDM Pro / Dream Machine Pro for my other home office and export it into GCP, therefore all of my sites will be accessible from GCP (currently I have 2 sites on GCP). Is that possible?
    The reason I choose UDM Pro is because I need the 2 WAN with failover feature, firewall and Unifi protect (Camera) together into 1 device.

    Reply

    1. 1) You don’t need to connect through unifi.ui.com. The idea behind that service is that you can connect to your internal controller that is behind a NAT/firewall. The controller will connect to ui.com and set up a screen sharing connection. It is unreliable and that was one factor why I started to look for a better alternative to manage my networks remotely. A cloud service is a much better solution.
      2) The main advantage of UDM is that it hosts the controller. You don’t need another one in the cloud. And then you will need to access it remotely through that pesky screen sharing. You can set up the failover with USG, even the smaller one. I haven’t set up Protect so I don’t know what benefits an UDM will provide.

      Reply

  229. Works perfect without any problems. One suggestion: GCP hides the option for the free compute instance very well. Maybe you could point readers to the f1-micro (old N1 in us-central region) vm which is actually free.

    Thank you for the post, it helps a lot and it’s straight forward.

    Reply

  230. Does the startup script go in the line in the vm creation where it requests a startup script URL or does it go in meta data? I cannot get unifi to install and think I am being silly but cannot figure out what is wrong. In my latest iteration, I think the script runs but not much happens, here is the log:

    Info
    2021-01-29 14:58:06.269 EST
    “startup-script: % Total % Received % Xferd Average Speed Time Time Time Current”
    Info
    2021-01-29 14:58:06.269 EST
    “startup-script: Dload Upload Total Spent Left Speed”
    Info
    2021-01-29 14:58:06.322 EST
    “startup-script: 0 0 0 0 0 0 0 0 –:–:– –:–:– –:–:– 0 100 16529 100 16529 0 0 305k 0 –:–:– –:–:– –:–:– 310k”
    Info
    2021-01-29 14:58:06.331 EST
    “startup-script: Script logrotate set up”
    Info
    2021-01-29 14:58:06.336 EST
    “startup-script: MongoDB logrotate set up”
    Info
    2021-01-29 14:58:06.352 EST
    “startup-script: IPv6 disabled”
    Info
    2021-01-29 14:58:06.427 EST
    “startup-script: 592 megabytes of memory detected”
    Info
    2021-01-29 14:58:06.600 EST
    “startup-script: Swap file created”
    Info
    2021-01-29 14:58:08.428 EST
    “startup-script: OK”

    Reply

    1. The startup script field is for the full script, not the link. You can copy the full text into it and it will work. Or you can put the startup-script-url key in the metadata and use gs://petri-unifi/startup.sh as the value. Your log looks peculiar. Did you choose Debian Stretch as the image? I need to create some test VMs to check this, but I am quite busy right now.

      Reply

    2. i believe i am having the same issue. I cannot get into unifi. I rebuilt the vm on google similar to above verifying i am on Debian Stretch still nothing.

      Reply

  231. Hi there. Really appreciate your easy to use tutorials here. Been running my controller on google cloud for about 6 or so months now and everything is great. I had a small $0.01 charge this month and wondered if maybe I have something setup incorrectly as this is the first time it has ever happened. When I dig into the billing history on google cloud it lists in the transaction detail that the charge is for: “Compute Engine Network Internet Egress from Americas to China: 0.021 Gibibytes”.

    Any thoughts on what I can do to limit my charges, but also and more importantly to make sure this isn’t someone gaining unauthorized access to my information.

    Reply

    1. Someone in China is running a script that loads the login screen over and over again. Data transfer to and from China are not included in the free tier. If you want to stop it, you can stop Lighttpd on the VM, since the scripts connect to port 80 typically.

      Reply

    2. Hello. I am having the same thing occur. I have been running on GCP for a little over a year now for free on the micro tier, however, I got notified that Google attempted to charge me $0.02 for my February Invoice. I can’t seem to figure out why, I am wondering if it’s the same reason you got charged.

      Reply

      1. There are bot networks in China that try to access your “site”. They never even try to log in, they just try common web server attacks. However, the volume is such that the page loads cause traffic – and traffic to China is not in the free package. The solution is to pay the cents or disable Lighttpd altogether.

        Reply

        1. Petri, thanks for all the work setting up and maintaining. Great tutorial.

          I have been running for a couple years and started getting the egress charges too.

          A suggestion for those hosting your controller on a domain. Route your requests through cloudflare (free tier) and block direct access to the VM other than through CF. Alter the allow-unifi firewall rule and change the source ip range to the following list (CFs IPs):
          173.245.48.0/20
          103.21.244.0/22
          103.22.200.0/22
          103.31.4.0/22
          141.101.64.0/18
          108.162.192.0/18
          190.93.240.0/20
          188.114.96.0/20
          197.234.240.0/22
          198.41.128.0/17
          162.158.0.0/15
          104.16.0.0/12
          172.64.0.0/13
          131.0.72.0/22
          Add a second firewall rule with a higher priority than the previous one. Named something like block-all, target tag unifi, Source IP range type 0.0.0.0/0, action on match deny, protocols and ports deny all. These rules now restrict all access to solely come through CF.

          Second, set up cloudflare access (free tier) and restrict access to only authenticated users (I used google as IDP). With this restriction, only your list of allowed authenticated users will be able to hit the controller VM.
          https://developers.cloudflare.com/cloudflare-one/setup

          Reply

          1. Thanks for your suggestion. I wrote my instruction for the layman. Your set up is top notch, but I can’t automatize it or trust that anyone could set it up. I’ll leave it here in case some other savvy user makes use of it.

            Reply

          2. Thank you for the idea. I went ahead and configured it the same way you did except instead of using CloudFlare Teams(CloudFlare One), I went ahead and just applied a Firewall rule to block anything from not a North America IP address when accessing the specific subdomain.

            Reply

  232. Hi there. Thanks so much for this guide. It was very helpful and I was able to move my controller to GCP with ease.

    My issue now, is that I cannot adopt new devices.
    When trying to do remote adoption it fails to connect to the controller.
    It states the ipaddress:8080/inform will need to be accessible, and then that it fails to find it.
    I have checked my firewall is setup correctly as per the guide but no luck.
    Do you have any ideas that may be able to help?

    Thanks again!

    Reply

    1. Sorry for the delay. SSH into the device and use the info command to see if the device has the right inform address. If it doesn’t then use the set-inform to set it. If it does have the correct address, then use ping to test the connectivity. I once had a network where the AP:s didn’t receive any default router. You can use the route command to see the routes and even create them.

      Reply

  233. Thanks so much for this tutorial! I’ve been running an instance for over two years now with great success.

    Thought I’d point out that Google have updated the Default Storage Class names when you create a bucket. Looks like that Regional is now called (or equivalent to) Standard.

    Cheers!

    Reply

    1. You can use your GCP based controller directly without going through Ubiquiti’s service. The finicky service and unreliable CloudKeys were the reason I started looking for a better solution. That’s how this all got started.

      Reply

      1. Thanks so much for your response. I was able to get my controller running again. I had to run fsck but everything started afterwards.

        Reply

  235. Thank you so much,

    really well documented.

    Initially it wouldn’t install the cert, it was failing with “Unable to import the certificate into keystore” at “[..] /usr/lib/unifi/lib/ace.jar import_cert [..]”
    I rebooted twice and made some tea ready for a long troubleshooting session.
    The cert was valid – and was looking into the chain while I lost ssh connection.
    I rebooted the machine just in case and it came up perfectly.
    I couldn’t get to the bottom of it, but happy anyway ¯\_(ツ)_/¯

    Thank you!

    Reply

    1. I occasionally see the keystore error message, even though the cert has been imported properly. It could be a Java bug. UniFi Controller requires an ancient Java version.

      Reply

  236. Mr Petri,

    My sincere respect for you. Without your work crafting this amazing script, we wouldn’t be able to have this amazing cloud based management for our UniFi products.

    I’m currently reading some horror stories about the newest UniFi controller software that is currently in RC.

    What is the best way to disable the auto-update of UniFi software on a VM that is already set up? I found out from another comment you made that by removing line 214: “c=ubiquiti”; we basically won’t update any Ubiquiti software with unattended-upgrades. But how can we edit this on an already running VM?

    Will the presence of gs://petri-unifi/startup.sh in the startup-script-url metadata affect this change? Do we have to remove it?

    Thanks in advance!

    Reply

    1. The startup script won’t do anything after the initial setup. You need to remove the c=ubiquiti from /etc/apt/apt.conf.d/51unattended-upgrades-unifi using sudo nano. First you will need to SSH into your VM. There is a button for SSH in the GCP console. Use ctrl-X to save and exit from nano. Google for a “nano editor” cheat sheet or video.

      Reply

      1. Superb! Thanks for your prompt response.

        Does the VM need to restart for the changes to take effect?

        Reply

        1. You probably have found out by now, but a reboot would an easy way to make certain. I believe unattended-upgrades will read the file each time, so restarting is not strictly required.

          Reply

  237. When creating the VM disk, you need to select “Standard persistent disk”. The default now is “Balanced persistent disk”.

    Reply

  238. Great script! I set this up a couple of years ago and it’s been great.

    However I noticed a couple of days ago that the service was no longer working. I tried stopping and starting to no avail. I checked the backups from the bucket and can see it looks like it stopped last week at some point. I downloaded the last couple of backups then deleted the VM.

    I set up a new VM again using the original static IP and linking to the original bucket, restored from last backup and all looked good but it doesn’t look like it’s copying to the bucket any more, and in fact the bucket is now empty. I don’t understand why all the original files would have been deleted from the bucket.

    I’ve looked back through comments to see if I could find any clues about why the files are not being copied to the bucket and used a few of the commands to check the config to use the bucket and got something I didn’t expect;

    sudo /usr/bin/gsutil rsync -r -d /var/lib/unifi/backup gs://unifi-sb

    this retruns;
    CommandException: arg (/var/lib/unifi/backup) does not name a directory, bucket, or bucket subdir.

    /usr/lib/unifi/data/backup/autobackup/
    this returns;
    -bash: /usr/lib/unifi/data/backup/autobackup/: No such file or directory

    Any ideas?

    Thanks.

    Reply

    1. The problem with using rsync is that if the source is empty (as with a new VM) then rsync will also clear the destination bucket. That’s why it is safest to create a new bucket and keep the old one around for a while. As to why there is no autobackup directory I have no clue. Do you have automatic backup enabled in the Controller? Do they show up in the controller?

      Reply

  239. Thank you so much for this beautifully run script and tutorial. The learning experience was invaluable and I am very grateful for how much work you put into this.

    One question I have or perhaps just a data point for future readers of the site. I’ve been running this GCP instance for my Unifi Controller for about 5 or so months. Working exactly as intended. Long enough that my GPC Introductory Offer has expired and I’m paying out of pocket. Totally fine but I wanted to make sure I’m not overpaying or that set something up wrong and could be paying less. Right now it looks like I’ll be paying around $5-6 a month for the micro-instance in US West. Checking the billing shows no charges on egress traffic or charges for storage. Just getting charged for RAM and CPU usage. Does this sound like a normal amount to you?

    Thanks again!

    Reply

    1. Google has been shrinking the offer little by little. At the moment the free VM is only available at us-west1, us-central1 and us-east1. You need to recreate VM in one of these and restore the latest backup. The public IP address will change as well. If you have used the DNS name as the inform address, then all you need to do is to update the DNS.

      Reply

      1. Peachy,
        Petri is correct (as always). If you select us-west1 (or the other free regions) you should not have to pay anything (sans huge egress traffic). My VM has been running for over 2 years now and the most I’ve paid in any one month is about $0.25 (Yes, 25 cents). That was due to big egress traffic to China (bots). To fix this, Per Petri’s instruction, I turned off Lighttpd and it greatly reduced egress traffic. In the last six months the most I’ve paid is $0.02 (2 cents). I can live with that.

        Reply

  240. Can you explain how this works, or maybe point us to something about why it works? I went through all of it yesterday and now have a fully functional cloud controller, but have no clue why it actually works. How is the VM able to access inside of my network and reach the APs simply by having my local controller point to my subdomain for a few seconds and having the VM point to the subdomain for a few seconds?

    I am not complaining because this is brilliant, but I have no idea why what I did worked. I did this whole thing because I was unable to use a software controller on my local wifi to access my APs, so now instead of being able to access them just from my local PC I can now access them from across the globe, which is pretty crazy, but I lack the understanding as to how this functions.

    Reply

    1. Isn’t it amazing? 🙂 There is no way to contact your internal network from the outside. The UniFi devices will open the outbound connection. The last thing your local controller did was to point all devices to the new inform host in the cloud.

      Reply

      1. So when I set up the domain name on my local controller to the one that I set up on that Afraid website, the local APs communicate out to the internet and then the Google one can see it?

        How does it work that once I shut down the local controller after I saw the heartbeat issues that the outside work can now keep talking to inside my network?

        Reply
          1. You change the inform host on the old controller
          2. The devices connect to the controller and receive the new address
          3. The devices will only contact the new cloud controller from that on
          Reply

  241. Has anyone seen this error when trying to stop the egress traffic issue?

    $ sudo systemctl stop lighttpd
    System has not been booted with systemd as init system (PID 1). Can’t operate.
    Failed to connect to bus: Host is down

    Reply

    1. I remember having seen that. Try sudo systemctl dbus start to see if it helps and make it permanent by sudo systemctl dbus enable if it does. You can also query the status with sudo systemctl dbus status.

      Reply

      1. Odd behavior all around:

        Welcome to Cloud Shell! Type “help” to get started.

        Your Cloud Platform project in this session is set to().

        Use “gcloud config set project [PROJECT_ID]” to change to a different project.

        cloudshell:~ ()$ sudo systemctl dbus start

        Unknown operation dbus.

        cloudshell:~ ()$ sudo systemctl dbus status

        Unknown operation dbus.

        cloudshell:~ ()$

        I am opening the console the right way, right? I go to VM Instances, click on the Name of my VM, click on “Serial port 1 (console)”, then “Equivalent command line”, then finally “RUN IN CLOUD SHELL” and it pops up the window on the bottom half of the browser.

        Reply

        1. No, you are not doing it right 🙂 Cloud shell runs on another VM, not on your UniFi Controller. In GCE Console, in VM instances list there should be only one VM. On the right there is a column Connect. Click on the SSH button in the Connect column to get a shell on the VM.

          Reply

          1. Bingo.

            Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
            permitted by applicable law.
            @unifi-controller:~$ sudo systemctl stop lighttpd
            @unifi-controller:~$ sudo systemctl disable lighttpd
            Synchronizing state of lighttpd.service with SysV service script with /lib/systemd/systemd-sysv-install.
            Executing: /lib/systemd/systemd-sysv-install disable lighttpd
            @unifi-controller:~$

            Thanks! Sure helps when you click in the right spot!

            I will watch to see how much I am billed now for my f1 micro instance.

            Reply

  242. Hey Petri,
    Can the controller be rolled back to a previous version in the GUI? If so, how? I also need to stop the unattended updates. The bleeding edge seems to be not so good when speaking ubiquiti (at least a lot of the time). As always thank you for your work and also your willingness to continue to answer our questions.

    Reply

    1. No, it won’t be easy. You can disable the unattended-upgrade before the upgrade, but rolling back requires also restoring an old backup, because the database schema typically changes. (If you search through these comments you can find multiple complaints concerning the Debian repositories lagging a few weeks after a release. Some people even have wanted to install beta or even alpha releases! I trust they will learn in time..)

      Reply

  243. Hi,
    I really don’t want to lock myself out of my controller but I’d like to change the login credentials for my GCP UniFi Controller.
    Would you mind sharing an idiot guide 🙂

    My GCP password was changed recently so this is all good

    Reply

    1. This is going to be difficult, because idiots are very resourceful and persistent 🙂

      You need to edit the UniFi Controller admin account. Where that is, depends on your Controller version and user interface selection. Mine is at 6.1.71 and using the new user interface, so the admin account is in Settings > System Settings > Administration > Admins. Click on the tree vertical dots in the last column to edit your account.

      There is no permanent login account for SSH logins. GCE will create a temporary account every time you click on the SSH button in the GCE Console. That way there is no way to break in since there are no credentials!

      Reply

  244. Hi Petri,
    I’m running my cloud hosted controller (with 25 sites (no captive portals)) on the free GCP tier but I’ve noticed that recently that when trying to access the web interface of my controller, it often times out (on both Chrome and Firefox) yet I’m still able to ping my controller and all is ok.

    I’ve found that the only way around this is stopping the instance and then starting it up again and I’m having to do this more often (2-3 times per week).

    Is there anything that I can do (other than upgrading the GCP tier) to fix this?

    Many thanks as always 🙂

    Reply

    1. 25 sites! Yikes! I have estimated the free tier could support up to 30 devices. Unless you have a single device per site, you probably need to upgrade. Even if you pay $10/mo that would only add up to $5/site/year. That’s negligible IMO.
      The only tuning you could try is disabling Lighttpd, but that won’t win you but a few cycles.

      Reply

      1. Oh wow… thanks for the feedback… average of about 2-3 devices per site so definitely maxing out the box!
        Will have a look at the other tears.

        Cheers!

        Reply

  245. Just wanted to drop a thanks Petri, used this info again today. I looked for a PayPal donate link but couldn’t find one. Really appreciate the work you’ve put into this.

    Reply

  246. I have used your scripts for years….and thank you so much….. but I have outgrown it…..I am thinking about installing my own hardware and running it instead of GCP. Did you create all of this based off what you have learned or is there something out there that has the best practices of all the things you put together that I can do manually without having to read through all the scripts? I am basically looking for something I can follow to implement everything you did but with my own hardware and not GCP. If not maybe I will work through it all and give it to you to post and grow it.

    Reply

    1. I have a long history as an Unix administrator, so I have gathered the bits over time. Most of the script is just installing things and those installations you should do by hand. You can skip Haveged on a non-virtual computer. Then you should decide whether you want to have unattended-upgrades (you shouldn’t, always test first). The only part that requires automation is Let’s Encrypt. However, LE requires a public IP address. If you don’t have one, then you can’t have a certificate but you don’t need to worry about importing the certificate ever couple of months either. Take a look at GCP Unifi Controller startup script explained although it is somewhat dated.

      Reply

    1. That is the first half. Then you need to also restore a backup of the database from the same era. The database schema changes often and old controllers can’t use future database schemas. The database schema changes are not documented, because downgrades are not supported. You can always give it a try.

      Reply

  248. Where is the domain name for LetsEncrypt being sourced from? I recently had to change the domain name for a controller and could not find where to change it for LE, I assume it’s using certbot?

    Reply

    1. Edit the dns-name meta-data and reboot the VM. The script will acquire a new certificate for the new domain.

      Reply

  249. Hi Petri,

    Got a mail from Google that reads:

    “Google Cloud Platform will roll out security and reliability improvements that will increase the sizes of OAuth 2.0 tokens. Please follow the instructions below to use the documented token size limits of 2048 bytes and to remove any logic in your services/code that restricts its ability to process access tokens of certain sizes before August 23, 2021.”

    I don’t want to break anything so is any action required on my side?

    Reply

  250. Hi!
    I still get charged ~$7 each month for “Network Internet Standard Tier Egress from Iowa” and it seems to increase for each month.
    Any idea why?
    I can’t find the reason…
    Regards,
    Mats

    Reply

    1. $7 is a lot of traffic! I suggest you kill Lighttpd as suggested at the top of this page [Note 2020/01]. It is responding to bot attacks. They won’t do any harm, but the do generate traffic.

      Reply

      1. Yes, that’s a lot.
        I have disabled it now, thought I already done it but apparently not.
        Thank you!

        Reply

        1. Strangely it continues to be charged for egress from Iowa.
          About $1 now since first of June for 10 gigabyte of traffic…
          I have no idea why…

          Reply

          1. How about changing the public IP address? Apparently the IP address you are using is getting too much attention for some reason. Just update the DNS to point to the new address and all UniFi devices will follow as soon as the DNS update propagates (could take a couple of days). That is, if you used the DNS name as the inform address.

            Reply

            1. I did that change now.
              The new one becomes a premium IP, not sure what it was before?

              Reply

  251. Hello Petri,
    As always thank you for sharing with the community. I have been running the network controller in a GCP vm for a year now no problems. My setup outgrew the free tier and I have no burned all my credits… I love this platform but I can not justify the close to 200 dollars a month to run this and UNMS on GCP so I have decided to host this on my own hardware. I have used many of the concepts I learned here from your project. I have everything up and reachable from outside the network. I can adopt using L3 methods and it seems to be working well. A few roadblocks I have hit and thought I would reach out as you have been very helpful every time. I have entered a A record on my DNS zone to point to the external IP address that is port forwarded to the box hosting my controller. I have noticed something right away:

    if I type in the FQDN it does not route to the login page. I get a connection refused.
    When I prepend https:// and :8443 I get to the login page.

    This is different behavior than I am used to with the GCP controller. I want to just type the FQDN and it redirect to the port like how it does from the GCP controller. I have not made it to the SSL cert yet. I plan on installing the lets Encrypt cert on this box as well I just need to further dissect and understand how to do this in debian. Does the SSL cert provide this reroute/auto port appending magic? Any direction would be much appreciated. I know this is a bit outside of what this project is intended but I figured I would give you a ping anyway and see how it goes. Again, thank you for all your contributions to the community. I have learned so many things along this journey. Cheers!

    Reply

    1. Bryan,
      in the event GCP is no longer free, would you care to share with us how you set up the controller on your own rig? I was hoping to do the same thing (eventually). Of course, I would still use all of Petri’s excellent instructions and scripts. I am wondering how you entered an A record on your DNS zone to point to the external IP address. If you don’t use GCP, how do you get a permanent external IP address?

      Reply

      1. If you want to set up the controller inside your network, you won’t need an external IP unless you want to access it from the outside. An internal controller can’t get a certificate so you just need to put up with the security warnings in your browser. You can get a permanent IP address from your ISP, but usually the monthly price is higher than a decent VM in GCP. You can also get by with a dynamic external IP using dDNS, unless your ISP blocks incoming connections.

        Reply

        1. Thanks Petri.
          I have a DDNS address. Would that be considered a dynamic external IP using dDNS? Does the Lets Encrypt Certificate synchronize with the DDNS name or with the IP? Not sure how that works. Thanks

          Reply

          1. TLS certificates are issued for DNS names, not IP addresses. Let’s Encrypt works fine with dDNS as long as your ISP allows for incoming connections. Some block them.

            Reply

  252. Looking into this further, It looks like I need Lighttpd to do the magic. Forgive my scripting language but what is < “https://%0:8443”)
    }
    }

    Reply

    1. Yes, Lighttpd does the redirect. The %0 means “whatever got you here”. The redirect means “use whatever got you here, but prepend https:// and append :8443“. You could have multiple names pointing to the same IP or you might use the bare IP address. (The certificate will only apply to a DNS name, so a bare IP address will give you a cert error later on.)

      Reply

  253. Just received this email from google:

    We’ve identified that you’re using an F1-micro instance that is a part of the Compute Engine Free Tier. As we improve the experience of the Free Tier, we will be introducing the E2-micro VM, which is a part of a second generation VM family. It offers additional resources for your use, specifically 0.25 vCPU (which burst to 2 vCPU periodically) and 1 GB of RAM.

    What do I need to know?
    The Free Tier F1-micro VM is changing to the E2-micro VM as the VM to use for free. On August 1, 2021, E2-micro Free Tier will be introduced. Follow these steps to change your machine type to E2-micro to avoid incurring charges for continuing to use F1-micro after August 31, 2021.

    What do I need to do?
    Starting August 1, 2021, change the machine type from F1-micro, or stop your existing F1-micro instance and begin using a E2-micro instance. VMs created with either method will automatically have the Free Tier discount applied to them. The supported regions will remain the same.

    Do you think there are any issues that need to be addressed before switching machine types?

    Reply

    1. There is nothing depending on the instance type. You can run it on any type you choose (or Google makes you choose).

      Reply

  254. Petri,

    Thank you again for this excellent guide. I’ve been running this for free for several years now with no issues. Thank you so much!

    Today, I got an email from Google explaining that they are depricating the F1-Micro tier later this year and replacing it with E2-Micro tier. It appears that it too will be free.

    Wondering if you knew if your guide would still work on the E2 tier…?

    Reply

  255. More changes from Google that look to be lowering the free VM power:

    We’ve identified that you’re using an F1-micro instance that is a part of the Compute Engine Free Tier. As we improve the experience of the Free Tier, we will be introducing the E2-micro VM, which is a part of a second generation VM family. It offers additional resources for your use, specifically 0.25 vCPU (which burst to 2 vCPU periodically) and 1 GB of RAM.

    The Free Tier F1-micro VM is changing to the E2-micro VM as the VM to use for free. On August 1, 2021, E2-micro Free Tier will be introduced. Follow these steps to change your machine type to E2-micro to avoid incurring charges for continuing to use F1-micro after August 31, 2021.

    Starting August 1, 2021, change the machine type from F1-micro, or stop your existing F1-micro instance and begin using a E2-micro instance. VMs created with either method will automatically have the Free Tier discount applied to them. The supported regions will remain the same.

    Reply

    1. I just changed it to the E2-micro and started it back up and everything came back online.

      Reply

        1. Thanks Clemens (and Petri). If I’m not mistaken, the 2vCPU is the periodic burst. I’m just wondering if you put both VM’s next to each other, which one has (had) the greater performance? My guess is the new one, given it has 1GB RAM. However, not sure which VM has the faster clock speed. Probably obvious, but I must be missing something.

          Reply

          1. I just made the switch on mine. E2-micro appears to respond faster. I have always assumed the controller is memory bound on 640MB, so 1GB of memory is the major factor.

            Reply

        1. If I read the email correctly, to stay on the free tier, you should change the instance type between August 1st and 31st.

          Reply

  256. Is Google lowering or increasing the power of the new VM? Google says the new E2-micro VM will have “0.25 vCPU (which burst to 2 vCPU periodically) and 1 GB of RAM” I don’t recall what the F1-micro instance has/had under the hood.

    Reply

  257. Thank you so much for this. I’ve been running it for a few years with no issues. It’s awesome.

    Reply

  258. Hi Petri,
    any idea what “Network Inter Zone Egress” is on the VM? I was charged a whopping $.01 for 0.67 gibibytes. Again, it’s not the penny that’s making me lose sleep. It’s trying to figure out where that .67 gigabytes is going to/from. Any ideas?

    Reply

    1. Inter Zone Egress doesn’t sound familiar, but I assume it is just botnets trying all kinds of WordPress, Drupal and PHP attacks. You can turn off Lighttpd. Lighttpd listens on port 80, where the attacks occur. Its only function is to redirect all requests to https port 8443.

      Reply

      1. Thanks. I disabled Lighttpd. Just curious, when the VM reboots, does it load up Lighttpd again? Is there any way to permanently disable Lighttpd?

        Reply

    1. I wondered why it didn’t say anything about the free tier. That explains it.

      Reply

  260. Hello, I am looking to migrate the instance that’s currently on Google Cloud to AWS or Azure. How should I go about on the migration? I would like to continue using the script, do I need to make adjustments in the script? Thanks!

    Reply

    1. The script is GCP specific in many ways. You’ll be best off if you create manually a Debian Stretch instance and install the software components by hand. Replace all references to VM meta-data by hard-coded values. The script is fairly straightforward and runs from top to bottom. You can follow it as a guide. The Let’s Encrypt part is the messiest. I believe it is easiest to install CertBot by hand and acquire the first certificate manually. After that CertBot will renew it automatically. My script is messy because it anticipates that the DNS is not yet ready and will try to acquire the first certificate repeatedly. After that it is just plain CertBot as if installed by hand.

      Reply

      1. Everyone can do whatever… but powershell core has a super easy module for ACMEv2 Certificate automation [posh-ACME] (LetEncrypt, DigiCert, etc. are ACMEv2 services) If you are not using the setup script…i.e. your on your own, a cron job and a ps1 that runs once a month can grab a new certificate when there is less than 30 days left.

        Reply

  261. Hello Petri, first of all thank you for documenting all of this. I’ve been using this method for over a year without any major issues.
    I just wanted to comment on the Egress charge some other people are seeing. I think there are 2 separate issues here, the bot issue is explained already in the post. The second issue that myself and I think others are seeing, is that I’m being charged for ANY egress traffic excluding China and Australia, even for those under 1GB/month. Bill for my us-east1 instance shows I’m being billed a couple cents a month under SKU “Network Internet Standard Tier Egress from South Carolina” even though the usage is less than 1 GB.

    https://cloud.google.com/network-tiers/pricing
    GCP has 2 tiers for network : Standard and Premium.
    If I’m reading this correctly, only Premium network counts under the Free Tier with the free 1GB, and if I’m using standard then I will be billed for any Egress traffic. Seems like a gotcha from Google..

    Reply

    2. I’m confused (I know, shocking!);
      My network tier is set for premium. Is that a good thing or a bad thing? where is the “gotcha” from Google? I still get hit for a penny or so each month for egress. What’s the difference between Premium and Standard? Just trying to select the best option.

      Thanks

      Reply

      1. Google isn’t very clear on this. “gman” reported that only Premius is in the free tier. This is probably an oversight by Google.

        The difference is that on Standard tier your data is pushed over to the public internet as soon as possible. In Premium it is transferred inside Google’s own high-speed network to the exit point closest to the receiver.

        Reply

        1. Seems to make sense as to why I see a charge of about $0.03 each month, even though my egress is very low. If I’m reading the GCP docs right, the only way to change to the Premium tier involves rebuilding the VM with a new IP. Trying to decide if it’s worth the trouble to save $0.36 each year.

          Reply

          1. A sane opinion for a change! 😁 Recreating the VM takes about fifteen minutes, so it is easy to calculate how you value your time.

            Over here in Finland there was a marketing event where you could get a plastic bucket for free. People would stand in line for an hour to get a bucket. The price of a bucket is around 1€. This never ceases to amaze me 🙄

            Reply

  262. Hello
    i have problem with the certificat generation. Everything esle is working fine.
    external DNS is ok and dns-name match the public DNS
    is there anything i can check ?

    Reply

    1. That means that Let’s Encrypt cannot connect to your server for some reason. Usually the DNS name is not resolvable or points to somewhere else. Another cause might be some firewall rule blocking access to port 80 or 443.

      Check the DNS name once more from your work, a public library, McDonald’s or somewhere else. If you are Linux savvy, then you might want to SSH into the VM (there is a button) and type sudo more /var/log/letsencrypt/letsencrypt.log to look for hints.

      Reply

      1. Thank you for your quick reply
        I managed to generate certificate manualy with letsencrypt using your script manualy
        but when i try to go through the process (install it on java)
        i have “Syntax error: newline unexpected” when i launch the scritps manualy

        Reply

          1. Ok i managed to make it work,
            it was because :
            extIP=\$(curl -fs -H “Metadata-Flavor: Google” “http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip”)
            dnsIP=\$(getent hosts ${dnsname} | cut -d ” ” -f 1)

            was not matching, dnsIP respond with the private adress instead of the public adress. is there a way to correct it ?

            Reply

            1. What does getent hosts your.controller.dns produce? And why? You can try it with other FQDNs as well to see if the VM’s resolver is botched somehow. I hope this isn’t some new feature by Google. Thanks for debugging this!

              Reply

              1. Resolver is ok, i can resolve well all domains
                but uni.mydomain.fr is resolve with the local private ip adress

                Reply

                1. I don’t qualify that as OK. Internal IPs shouldn’t leak into the Internet. Where are your DNS zones hosted?

                  How about if you replace 169.254.169.254 with 8.8.8.8 as your name server in /etc/resolv.conf? You probably need to reboot.

                  As a last resort you can install dig with sudo apt install dnsutils. With dig you can specify the DNS server to see who replies with what: dig @8.8.4.4 uni.mydomain.fr Dig produces a lot of output, just pay attention to the ANSWER SECTION. Query also directly your DNS host.

                  Reply

                2. In the hosts file it has been set to a local ip by Google. Don’t know how to manipulate this so that it resolves to the public ip.

                  Reply

                  1. I am not certain exactly what “it” has been set by Google. In /etc/hosts there are two lines with price IP addresses and DNS names ending in .internal. Those can’t be used as domain names for Let’s Encrypt. You need to set up your public DNS. You can use one of the free dDNS providers, Google Cloud DNS, your domain registrar or anything you like.

                    Reply

        2. You will need to renew the cert every 90 days at least. You want it to happen automagically.

          You are the first to report this problem, so I assume it affects only your system for some reason. DNS is OK since certbot works manually. Instead of wasting your time (and mine) troubleshooting this, you should just delete the VM and create a new one. Sometimes something goes awry.

          Reply

  263. Hi Petri,
    I should probably know this answer to this, but I don’t. I want to set up another Unifi network in my other house (in another State). Do I need to set up another controller? Can you have two controller on the Google VM? If so, How do I set up two on the same VM. Thanks

    Reply

    1. Use the site feature of the Controller. It is just for this purpose!

      Reply

  264. …Also, I see on your instruction for creating the VM you state “By default the Boot disk is 10GB which suffices, but it is tricky to expand afterwards.” I left my VM at 30GB when I created it (not sure why I did that). I think you are going to tell me to create another VM (per question above). Can I change my original Boot disk to 15GB without any problems (so each VM has 15GB)?

    Also Also…My new network at my new house will have a USG and 2 AP’s. Can I just add those devices to my existing controller (devices would be in two different locations)? Or does each site need a new controller?

    Thanks

    Reply

    1. Shrinking a live filesystem is considered an expert-only maneuver. If you want to experiment, go ahead, but take a backup first! I won’t guide you there, though. It is a lot easier to take a backup, delete the VM and create a new one.

      The devices needn’t be at the same location. Actually, the Controller has no way of knowing this. To make management simpler, I would create a new site in the controller, but that is not a requirement.

      Reply

      1. Perfect. Thanks. So to be clear; on the new controller, it can have the same Controller Hostname/IP DDNS entry e.g. mysite.duckdns.org?

        Reply

        1. It should! That way the devices will keep sending their updates to the controller.

          Reply

  266. So how about this one, say you want to migrate back out of the cloud to your local desktop?

    What needs to be done to get the devices back and talking to your local machine?

    Reply

    1. You can take a backup of the cloud controller and restore it on a local controller. The only thing you need to worry about is the Inform host. Change it in the cloud before you take the backup. If your local controller has a static IP address or a DNS name, you can use that as the Inform host. Or you can disable the Inform host override and enable Network discovery instead for dynamic operation. These are in Settings > System settings > Controller configuration.

      No, I have never tried this in practice. So far I have never had a reason to go back.

      Reply

  267. Having issues with the expired R3 certificate (from DST Root CA X3). My domain cert is valid, but it’s not accepted by the browser because it seems to use the old R3 Intermediate from Let’s Encrypt.

    If I check the fullchain.pem file from Lets Encrypt on my server it does contain the valid new intermediate and root cert (ISRG Root X1 and new R3) but it’s not loading them?

    Reply

    1. Thanks for the heads up! I have the same problem. I’ll need to dig into this. It will probably take some time. I guess the problem is that the new chain is not being loaded into Java properly.

      Reply

    2. Just some observations to help.
      – Chrome based browser reports that R3 cert has expired. (Intermediate.)
      – The X3 cert is also expired, but that is listed seperately.
      – Firefox displayes different chain and is happy as a kitten. (Different R3 cert with different dates ~2025. The root cert is named ISRG Root X1 and expires in 2035.

      So Ronny, I am seeing the end-user’s side of what you verified on the serverside, the ‘correct’ fullchain.pem
      —
      For a moment I thought the problem was that Debian Stretch did not have a certain (newer) root certificate, but that would not result in different browser experiences, right?

      The webserver isn’t serving “the right chain” or the client browser is not “choosing the right chain”?

      What part is responsible for picking the correct certificate chain?

      Reply

      1. The problem seems to be on /etc/letsencrypt/renewal-hooks/deploy/unifi in combination of previously hardcoding the $caroot (/usr/share/misc/ca_root.pem) with the old DST Root CA X3 certificate, maybe?

        I’ve tried to debug this but I had no luck on using the keytool to import the keystore on /var/lib/unifi/keystore

        Reply

        1. Yes, that is the problem. The reason for hardcoding was that the cross-signed root key is not available at any fixed URL. A new root key is manually available and I have hardcoded it in gs://petri-unifi/test.sh. There is still a problem with keytool, it won’t import the current chain for some reason. R3 certificate in the chain has expired, too. CertBot will update the chain file as well, but keytool is the problem. PKI and TLS are complicated already, but Java makes things even more obscure.

          Reply

          1. I got it running again, perhaps not reboot-proof, but I wanted to share.

            The operations with the certificate keystore seem to go fine eventhough it complained about the format.
            As you referred to, “java -jar /usr/lib/unifi/lib/ace.jar import_cert $pubcrt $chain” was the problem, which gave the obscure error. (Why didn’t import? Parsing… okay…)

            – I removed the newlines based on post at Unifi community of all files. Not sure why this problem popped up today, seems a seperate issue.
            Now I got feedback message. It told me it was missing the X3 cert, but I knew from reading online we don’t that one.
            – I updated the chain file and removed the X1-crossed-signed certifcate. I used the self-signed X1 root certificate.

            Unfortunately, I have done many operations with the keytool, I am not confident in its state, as it will be deleted every time we run the deploy hook.
            With the –list feature it reports 3 certifates. I can tell you I imported the X1 root certifcate in there as well. (self-signed). My understanding is limited, because I couldn’t explain you why we need to import/copy the certifcate and friends THREE times, before everything is happy again.
            Certbot output
            Keytool import (p12)
            jar import
            Curious to learn which component on debian is actually terminating https at our unifi server.

            I used https://www.sslshopper.com/certificate-decoder.html to see what was happening.

            Hope it helps.

            Reply

          2. Learned something new.

            I guess the first import with keystore and the import via the ace.jar results in the same.

            I guess the challenge was to import a chain instead of a single certificate.
            From here (https://stackoverflow.com/questions/16062072/how-to-add-certificate-chain-to-keystore) I learned how we can import the complete correct certifate chain by concatenating (cat).

            It will replace the jar interactions. So no weird remove-newlines interactions, be just circumvent it as far as I can tell.

            So imagine a combo.pem with $pubcrt, $chain and $caroot (in my case my X1 root self-signed is already in my $chain, but regardless.)

            That combo.pem we use here: (ssh, thus sudo)
            sudo openssl pkcs12 -export -in combo.pem -inkey $privkey -out $p12/tempfile -name unifi -CAfile $chain -caname root -passout pass:aircontrolenterprise

            Now we can import this combined p12 file:
            sudo keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore /var/lib/unifi/keystore -srckeystore $p12/tempfile -srcstoretype PKCS12 -srcstorepass aircontrolenterprise -alias unifi

            sudo sytemctl unifi restart

            This resulted in the desired behaviour that non-Firefox browsers on my Mac where served the complete chain without any expired intermate certificates.

            What went wrong was that the keytool import step was succesfull, but the jar import didn’t, so newer generated letsencrypt certificates where being served without a chain. So some browsers rebuild the certificate chain and choose “the wrong path.” This tweet also helped my understanding (https://twitter.com/webprofusion/status/1443425958115962882?s=20).

            When I forced the jar import by fixing newlines, or by the command above, the result is importing a complete chain of certifactes. Then, after a restart, the browser-clients receives the hint of please-use-this-chain.

            (I used
            sudo keytool -keystore /var/lib/unifi/keystore -list
            sudo keytool -keystore /var/lib/unifi/keystore -list -v
            to see inside the keystore and understand what was happening.)

            I suspect the jar import command might not have been working for some time, but we never noticed before, because the singular certificates where succesfully imported.

            Reply

            1. Thanks for your input! I was aware of the problem with newlines, but never bothered to fix it because it appeared benign. I will fix this sometime soon, but at the moment I have hands full of (paid!) work. FireFox will serve as a temporary workaround.

              Reply

              1. No problem Petri. Understandable, we thank you for your work.

                We got your back ;). ↗️

                Regards.

                Reply

            2. Thank you for this, I followed it and updated to the fullchain.

              Commands I used as root:
              cd /etc/letsencrypt/live/

              openssl pkcs12 -export -in combo.pem -inkey privkey.pem -out tempfile -name unifi -CAfile fullchain.pem -caname root -passout pass:aircontrolenterprise

              keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore
              /var/lib/unifi/keystore -srckeystore tempfile -srcstoretype PKCS12 -srcstorepass aircontrolenterprise -alias unifi

              systemctl restart unifi

              Reply

              1. Thanks, I appreciate, I’ll have to try this out. I believe that the problem has to do with the line breaks problem. Your solution doesn’t address that. Oh boy how I hate wasting time on Java 8 peculiarities.

                Reply

  268. This guide got me setup and running in under 15 minutes so a huge thank you 🙂
    Wish I had done it a week earlier to avoid the SSL games but hey, procrastination got me!

    Reply

  269. When logging in its prompting to download the latest 6.5.54 for the log4j CVE, however even rebooting the GCP is not installing it. When might this happen? As an abundance of caution, we had had our controller powered down

    Reply

    1. Mine is already at 6.5.54. You can SSH to the virtual machine and use commands sudo apt-get update and sudo apt-get upgrade. Then you will see the error messages. I am getting errors from cloud repositories, but they don’t affect this.

      Reply

    2. I fixed refreshing the PGP keys. You need to stop and start the VM to get rid of the apt-get update errors.

      Reply

  270. Thanks for your guide, very straight forward and helpful!!!

    I didn’t realized we need to change the disk type and got a small charge after the trial period. The cost report will show the following item: “Balanced PD Capacity”. I found a link from Reddit that provide direction to change your disk type:
    https://www.reddit.com/r/googlecloud/comments/nu50hm/comment/h7decki/?utm_source=share&utm_medium=web2x&context=3

    Created the new disk and remount as the boot disk of the VM.

    Reply

  271. Hi Petri,

    i was affected bei Log4J since i did not have auto upadtes in place, like with your setup. So i found your page/script and wanted to start a new project.
    But is saw that debian stretch has only 6 months left on updates. Is Buster still not supported ? My previous Project were running on Ubuntu 18.04 and 20.04 which are both based on newer versions of debian and i cant rember that i had trouble with mongo DB versions.

    I cloud try myself but i probabbly would not notice if the script fails and would not work correctly.

    Thanks for your work

    Reply

    1. The last time I checked UniFi controller runs only on (age old) Java 8, which is only supported on Stretch. My script relies on supported features only. I don’t have the resources to support some configuration that the creators have deemed unsupportable. There is also a problem with MongoDB. MongoDB changed their license and you need to stay below some version number to comply.

      Feel free to experiment, though. Create a VM with a later Debian image and run the script. Look for errors in the Serial port 1 (console) screen, remember to refresh. Report back here your results, but you need to solve the problems yourself. I certainly hope there would be a more modern version available.

      Reply

      2. Hi Petri,

        i did a quick check with buster, as you predicted i did not work.
        i did some reasarch on other setup scripts, like https://glennr.nl/s/unifi-network-controller , they all do some kind of downgrade for Java/Mongo DB for newer versions of debian/ubuntu.
        For now i will go with your script and stretch, since i like the update automation.:-)

        Lets wait and see what ubiquity will do if there only official supported Linux versions will not be supported anymore.
        Since also Fragattacks is still not fixed on the gear that i use and not fixed many other ubiquity products, i think nothing will happen…
        ( https://community.ui.com/questions/FragAttacks-New-vulnerabilities-for-WEP-WPA-WPA2-WPA3/3f5e0bd8-f293-4688-8ac8-964515909102?page=8 )

        Well, thanks for your work and time

        Max

        Reply

  272. Thanks for the tutorial it was easy to follow and well documented.
    Unfortunately I am having issues login in for the first time.
    For my domain I added A record for a subdomain on my registrar and pointed it to the external IP that google supplied during firewall rule creation and it is tagged “unifi”
    That subdomain is not reachable, so I tried using the IP with port :8443 not reachable either.
    I check the dns for the subdomain, it shows googles IP.
    Any suggestions?

    Reply

    1. Unreachable means there is no IP route to the VM. In that case you can’t ping the IP address.

      My first suspect is always the IP address you are trying to use. Astonishing many users try to use 192.168.0.1 for the address. Is there a reason you didn’t reserve a static external IP address? You can see your IP addresses in the console at Networking > VPC Network > External IP addresses.

      Reply

  273. Hello,

    I have 2 questions.

    1. How do you change the network level from Standard to Premium and keep the same external IP?

    2. What steps need to happen to change DNS hosting site.s

    Reply
      1. If you have a Reserved IP Address then this is a no-brainer. The easiest solution is to create a new VM and attach the same IP address to it. If you haven’t reserved the address, then please do so now. You will loose your current IP address, but if your Inform Host points to a name, then the APs will resolve eventually the new address.
      2. DNS hosting changes don’t affect the controller at all. As long as the domain name resolves, everything is fine. If there is an outage, then you will loose some statistics, but the UniFi network will continue to run. If you are using the dynamic DNS option, then you need to edit the meta-data field and restart your VM.
      Reply

  274. 1. Its been a while. How can I tell where I got the IP from and if its reserved. I followed your setup with Afraid.org

    Reply

    1. Check Networking > VPC Network > External IP Addresses and look at the Type column. Static means a reserved address. If you reserve a new one, make sure it is in the Premium tier.

      Reply

      1. Question. Could I reserve a new IP that is premium. shut the VM, assign the new IP, reboot, and just repoint my network devices at the new IP without having to rebuild the whole VM?

        Reply

        1. That should work. Just remember, that you need to set inform host before the change, so the devices will get the new IP. If you are using the same DNS name, then there will be just a delay until DNS settles.

          Reply

  275. Hi,
    Thanks for your excellent script and walkthrough!

    I got a question: I changed the metadata for the DNS entry, stopped the machine and restarted it.
    Apparently that’s not enough to issue a new Letsencrypt certificate. Is there something I can do to start the renewal process?

    Reply

  276. Nevermind, another Stop and Start of the VM instance did the trick, certificate is ok now.

    Reply

  277. I’m running the Unifi console for some time now but like to update it to version 7 (beta).
    I made the adjustments so it should download Early Access software and firmware.
    Early Access firmware is downloaded, but the Early Access console software not.
    Do you know what could go wrong?

    Reply

    1. No, I would like to defer introducing new version even further! Fortunately the Debian repos are updated last. Ubiquiti has a bad record of releasing defunct software.

      Reply

  278. Great work. I finally got it working on day 1 after rebuilding the VM 3 times. Either I didn’t set Debian to v9 or I didn’t have my region to the correct free one or I didn’t wait long enough. But finally got it all working. I found watching the video helpful, but ultimately carefully reading each point on this page and the minor updates made the biggest difference. Thanks for doing this labor
    of love!

    Reply

    1. Thanks! “Repetitio est mater studiorum”

      UniFi Controller version 7 is expected, but it will still require Java 8. We are stuck with Debian 9 while its support will end in the summer. I don’t like this at all.

      Reply

  279. Greatly appreciate this write up. I’ve been using this for over a year now for free but recently (past 3-4 months) I noticed that I’m paying about $4 per month for this VM. I’m having a hard time figuring out exactly why it’s costing me and was hoping you might have some insight.

    Thanks,
    Roger

    Reply

    1. $4 is a lot. The typical charges for egress has been dozens of cents per month. Go through the free tier offering and compare where your setup differs. Which zone are you using (us-east1/central1/west1)? What is the instance type (e2-micro)? What kind of disk (standard)? Are all your resources in the same zone? Is your network on the premium tier (as it should)? etc. Google will be happy to charge you for anything beyond the exact offering.

      Reply

      1. Thank you so much for this advice.
        So I’m using:
        Zone: us-west1-b
        Instance Type: e2-micro
        Disk Type: Balanced persistent disk
        I think my resources are all in the same zone. Both disk and instance say us-west1-b
        Network tier: Premium.

        So I guess the disk is the issue? Balanced persistent disk needs to be converted to a Standard Persistent Disk?
        If you can confirm that is the wrong disk type for free tier, I already found a write up on how you can use a snapshot to create a new disk and detach the old one and attach the new one.
        Thanks again. Really appreciate how responsive you are and how much you share the knowledge. Super helpful.

        Reply

        1. Balanced persistent disk is not on the free tier for sure. I don’t know if it alone costs $4/mo or does it make the whole VM chargeable. You’d need a flock of lawyers to make sense of the billing terms.

          Reply

          1. Great, thank you. I just turned it off. Took a snapshot. Created a new disk using the snapshot but as Standard. Detached boot disk. Attached new disk.

            Reply

  280. Love the write-up, thank you so much!

    I just found out that my Controller was offline for a while and hasn’t been updated. Can I force it to update to the latest version or should I wait until it happens automatically?

    Reply

  281. Hi,

    I cant get this to work. I have followed your instructions to the letter have even and deleted and tried again. I cannot get the controller to even load the page. I can ping the IP and I can ping by hostname I set an A record at my domain. Your script seems to run but i cant tell if it completes sucessfully from the logs.

    This is the last few entires:

    2022-02-28 19:48:42.404 EST
    unifi-controller
    “GCE Agent Started (version 20211116.00)”
    Info
    2022-02-28 19:48:42.434 EST
    unifi-controller
    “OSConfig Agent (version 20211117.00-g1) started.”
    Info
    2022-02-28 19:48:42.698 EST
    unifi-controller
    “Starting startup scripts (version 20211116.00).”
    Info
    2022-02-28 19:48:42.700 EST
    unifi-controller
    “Found startup-script-url in metadata.”
    Info
    2022-02-28 19:48:42.974 EST
    unifi-controller
    “startup-script-url: Script logrotate set up”
    Info
    2022-02-28 19:48:42.976 EST
    unifi-controller
    “startup-script-url: MongoDB logrotate set up”
    Info
    2022-02-28 19:48:42.980 EST
    unifi-controller
    “startup-script-url: IPv6 disabled”
    Info
    2022-02-28 19:48:43.069 EST
    unifi-controller
    “startup-script-url: 996 megabytes of memory detected”
    Info
    2022-02-28 19:48:43.148 EST
    unifi-controller
    “startup-script-url: Swap file created”
    Info
    2022-02-28 19:48:43.820 EST
    unifi-controller
    “startup-script-url: OK”
    Error
    2022-02-28 19:48:47.754 EST
    unifi-controller
    “Error configuring IPv6: Internet Systems Consortium DHCP Client 4.3.5 Copyright 2004-2016 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ no link-local IPv6 address for eth0 If you think you have received this message due to a bug rather than a configuration issue please read the section on submitting bugs on either our web page at http://www.isc.org or in the README file before submitting a bug. These pages explain the proper process and the information we find helpful for debugging.. exiting.”

    I have waited over an hour and still nothing any suggestions?

    Reply

  282. Thank you Petri. This must be the best guide I’ve seen on the Internet so far. Worked just fine.

    Reply

    1. Regarding chapter 5. Set Up the Controller.

      What is the reason for adding the DNS name or the IP address of the new controller to the new controller? I understand the reason for adding this to the old controller in order for the current devices to connect to the new controller. But I can’t really understand why this needs to be done to the new controller as well?

      Reply

      1. Very good question! Think: What would happen when the devices connect to the new controller? The new controller would have the old inform address, so it would direct all the devices to talk to the old controller, which would direct them again to the new controller… That’s why you need to make the change first in the new controller.

        Reply

  283. I dont know if its only my VM or it happens to anoyone else, but the daily backup to a bucket, stop working for at least 2 months now, in my bucket theres no autobackups since jan, and the migration updates backups the last one i see in my bucket its from nov 4 months ago, maybe since version 6.5.55 or newer the folder of the backups in the VM has changed.

    …. Ok i was doing some test while writting this, and found the error and the fix, also well kinda dumb of me for not doing a search to se if the problem was already being reported by other users and found out that it is, read some of the post not all and found that some people found a fix to theirs problems and others needed to do a reinstall of the VM.

    The problem is mainly a permision acces denied to the bucket it can be either (read/write/delete) u can easy fix it by:
    1- Go into ur VM and look for the “Service account” and copy the value
    2- Go to IAM & Admin and +Add a new Permision
    3- In Principals – Paste there the VM service account ID
    4- In Role – Select “Storage Object Admin”

    Probably u can also fix it by just changing the permision in the VM settings under the “Cloud API access scopes/Storage”, by just disable->save, edit again enable->save, this will most likely triger CGP to reset the permisions automaticaly.

    Hope that other people find this post usefull.

    Reply

  284. I have also realized that my backups stopped working almost a year ago now.
    I tried using the recommendation from Xariwey but still nothing.

    Would appreciate any advice on how to rectify. Thanks.

    Reply

    1. If u where unable to fix the problems by manually setting up the permissions, then i strongly reccomend to start from fresh, make sure to access in to your controller and manually generate and download a backup before prceding —make sure to have an up to date backup file in your computer—, delete the project and start from fresh.

      Reply

  285. Awesome guide, thanks! Got me up and running in a matter of minutes.

    I’ve been scratching my head a little about something this evening, trying a bunch of things unsuccessfully, and ended up concluding that this will probably take you a matter of minutes to either explain or implement within the startup script haha.

    I’d love it if fail2ban could email me when IPs are banned and unbanned. It seems as though this is something that has been implemented successfully by other people, and I was under the impression that it would be possible to avoid the standard GCP blocking of port 25 by making sendmail send these emails by using my gmail account via port 465, but I keep getting errors on the fail2ban log when it tries to send emails, and I have no idea why.

    Any help would be greatly appreciated!

    Reply

    1. That is exactly why sending e-mail is missing from the instructions and is left as an exercise for the reader! It is a pain to set up and could be implemented in many ways. If you want to use your own credentials, then you should use submission port 587. Another option is to use an external service like Sendgrid. See Google instructions for details.

      Reply

      1. Given the lack of success I had, it makes a lot of sense that you’d steer clear of it haha. Sendgrid looks promising though – thanks for the pointer!

        Reply

  286. On a specific site I have adopted 7 APs to the controller on GCP. Now, the problem is on this specific site I am running a 4G-router + SIM with only 5 GB data each month and it seems as the data is consumed pretty quickly (about 300 MB a day, which is more compared with similar sites without a controller running 24/7 in the cloud). Is there a way to keep the APs running but not consuming all this data? I have been thinking about a few solutions:
    – set an invalid set-inform address when I am done with the configuration.
    – turn off the VM in GCP (but I am worried about what you write in the guide that the static IP address is free as long as it is attached to a running virtual machine).

    But is there a better solution?

    Reply

    1. Don’t set an invalid set-inform address. There is no way to recover.
      You can turn off the VM and pay for the IP address. IIRC it is less than USD10 per month.
      I would block the traffic at the 4G router. There is probably a firewall of sorts. Just create a deny rule for port 8080 in the GCP.

      Reply

      1. That is a better idea indeed. I will check if I am able to create this rule in router. If not, would it be possible to do it the other way around and create the rule in the GCP VM’s firewall instead?

        Reply

        1. I tried adding the rule to the router but still traffic is somehow being allowed. It is a Teltonika RUT950. Not sure what I am doing wrong. I set up a rule to reject all traffic from my GCP IP on port 8080 (also tried all ports). I ended up removing port 8080 from the GCP firewall settings for now. This will be my temporary fix until I find a solution to this. It should work just adding 8080 back into the GCP firewall settings later on I guess?

          Reply

          1. You should reject all traffic TO your GCP IP 😉
            Blocking at GCP is only a partial solution as the initial connection packet still go over the 4G link.

            Reply

            1. I see, worked like a charm. Thank you! 😊 So from my understanding the “Turn off the VM”-solution would also have let the initial connection packets go over the 4G-link right?

              Reply

              1. Correct! You want to stop the packets before they go over the 4G link. The traffic would be lower, because there wouldn’t be no responses, but it would still eat your data plan.

                Reply

                1. Hi Petri,

                  As you can see above I previously rejected all traffic to my GCP IP on port 8080 in order to save my data plan. But now I can see that data is also sent to my GCP IP on port 5514 (UDP). I have not seen this before. Is it okay to block this as well? Thanks!

                  Reply

                  1. New sites will by default log to the controller using UDP 5514. It is cleaner to disable this in the devices than block at the controller. The setting is in Settings > System > System Logging > Syslog.

                    Reply

                    1. I am not sure I fully understand. Please correct me if I have misunderstood anything.

                      As I see it, the APs are sending traffic to my GCP IP (controller) using UDP 5514. I can see this in the traffic analyzer on the router. But the firewall that we set up for the controller on GCP in this guide (allow-unifi) does not have UDP 5514 allowed. So shouldn’t this traffic be blocked anyways?

                      When I said blocked I meant blocking the UDP 5514 traffic to the GCP IP already from router’s firewall, as I have already done for port 8080.

  287. Thank you so much for the post and the script! I have 6 UniFi devices (2 switches and 4 AP’s) and I see constant 0.2KB/s egress traffic, which would result in about 500MB of egress traffic in a month from this baseline traffic alone. Since there’s additional egress traffic from looking at the Controller and downloading firmware updates, I think I will go over 1GB/month. Does this all sound right to you? Or is there some configuration on Controller that can reduce the baseline egress traffic? Thanks!

    Reply

    1. I have turned off remote administration. It keeps a connection open to ui.com. There is no need for remote desktop connection because you can connect to GCP directly. The setting is in Settings > System > Administration. I have also turned off the SSO option. Other than that doesn’t come to my mind right now.

      Reply

  288. Hi Petri,
    I set this up on a friends system a while back. Unfortunately, he did not update his credit card exp date when he was issued a new on. So, google deactivated his VM. I updated his credit card info for him, but it appears the VM is wiped out. Any wan to import the VM that was stored in the bucket?

    Reply

    1. If you have access to the backup bucket then you can download a backup file from there. In GCP Console look under Storage. Create a new VM and in the UniFi wizard at the first step there is an option to restore a backup. If you used a DNS name for the inform host, then just set that DNS name to point to the new VM and you will be set. It is a good practice to also keep a local copy of a backup, even if it is not updated regularly. Without a backup you will need to manually reset each device and re-adopt them.

      Reply

  289. Yeah, it looks like he’s hosed. Nothing at all on the GCP. Where does the Unifi Controller place the auto backups? Are they stored on the GCP somewhere?

    Reply

    1. I found the Unifi backups in his storage. However, the autobackups only go back to Jan 2021. Any idea why they stopped?

      Reply

      1. Either the UniFi controller stopped creating backups or the script copying the files to Google Storage started failing. It is impossible to determine without the controller VM. Anyway, an old backup is much better than no backup. However, if he has added new devices after Jan 2021 then they must be reset and readopted.

        Reply

  290. Thank you so much for this script Petri! I noticed that backup files are no longer being copied to my Storage Bucket daily. I have daily backups on the VM itself in /var/lib/unifi/backup, and running “gsutil rsync….” command manually also works.

    I did notice that when I ran “systemctl status unifi-backup.service” it shows that service is loaded but is inactive (dead). I also tried “systemctl status unifi-backup.timer” and that also says loaded but inactive (dead). Is .timer supposed to be active? If so, I’m not sure what caused that unit to die. It seems like auto copy to storage bucket stopped working around the time I removed your script from running upon startup (due to security concerns per your instructions). Since then, my VM was not using my local timezone (US/Pacific), but was rather using UTC. I saw someone say in a comment that auto copy to bucket was fixed by setting the timezone on the VM, so I wonder if my problem is also caused by not setting appropriate timezone on the VM. Thanks!

    Reply

    1. I looked at startup.sh and noticed that there is no “systemctl enable” for unifi-backup.timer in the script (only “systemctl start”). Same for certbotrun.timer. Could it be that backup service wasn’t running because timer was never enabled? (so after reboot, unifi-backup.timer was never active)

      But then, I don’t see “systemctl enable” for unifi either, but every time I reboot the VM, Unifi controller works.

      Reply

      1. Thank you for your input! When I wrote the script initially, I wasn’t familiar with systemd (I’m actually a BSD admin). Yes, you should enable the .timer units (not the .service units!). If you run the script at every boot, then the script will start them manually each time. Good catch!
        Unfortunately I am abroad for a month, so I can’t fix and test the script, but I will update it by the end of August. You can just manually enable the timer units and you are good to go.

        Reply

  291. Hi Petri,
    when I set up the Virtual Machine, the part that says “Click on Networking, disks, security, management, sole-tenancy to open advanced options. In the Networking section add unifi to the Network Tags field” I tried putting unifi in the Network Tag Field but it wont let me. If gives a syntax error. Any thoughts?

    Reply

    1. The user interface has changed since I made the video, but the tag functionality still works the same. There can’t be any spaces or other special characters in the tag. Other than that, it works (I just tested it, although I didn’t go through the whole process).

      Reply

  292. Hi Petri,

    My controller just updated to 7.1.6.8, and now when I log in it says “No Unifi devices have been adopted”. I have two access points. When I SSH into each one and run “info”, it says they are both connected to the correct inform host. I have rebooted the VM and both access points, no luck.

    Should I manually reissue the command the set the inform host? Will that force a re-adoption?

    Thanks in advance for your help!

    Reply

    1. This has happened to me quite often after version 7 (I think). It usually is resolved by refreshing the page a couple of times. I suspect the reason is age old Java and age old MongoDB combined with spaghetti code. I have not seen this as a permanent state. All the controllers that I manage upgraded to 7.1.68 without problems. Of course that doesn’t prove anything, I might just be lucky.

      If the devices believe they are connected, then they are getting responses from the controller. Fixing the devices wouldn’t help in this case. I would try restoring the latest backup prior the upgrade if the problem persists. Unfortunately UniFi doesn’t support downgrades.

      Reply

  293. Hmmmm, I rebuilt my friend’s VM on GCP. Now he’s being charged for the machine ($1.20 in the first week). Is the free tier no longer available? I put all my settings for East Coast (Virginia). Did I do something wrong?

    Reply

    1. The free tier has been shrinking over time. Now it is limited to just two regions in the U.S. (the last time I checked). The machine type is also limited to a single type, but it has changed. Double check the current requirements.

      Reply

  294. My fault, I picked East Coast Virginia. don’t do that! Pick East Coast South Carolina for the free tier.
    Currently, Google lists the following for the free tier:
    1 non-preemptible e2-micro VM instance per month in one of the following US regions:
    Oregon: us-west1
    Iowa: us-central1
    South Carolina: us-east1
    30 GB-months standard persistent disk
    5 GB-month snapshot storage in the following regions:
    Oregon: us-west1
    Iowa: us-central1
    South Carolina: us-east1
    Taiwan: asia-east1
    Belgium: europe-west1
    1 GB network egress from North America to all region destinations (excluding China and Australia) per month

    I’m having trouble selecting Debian GNU/Linux 9 (Stretch). It doesn’t appear to be an option anymore (unless I’m missing something). Will the controller work on GNU/Linux 10 (Buster) now?

    Reply

    1. Support for Debian Stretch ended at the end of June, 2022. UniFi Controller requires ancient Java and MongoDB versions, which are not available for later Debian releases. There are unsupported ways that you can google and make it work. However, any updates may or may not break it.

      Ubiquiti is selling their own controllers these days. Apparently the run-anywhere distribution hasn’t gotten the attention it deserves. I know that rewriting the entire thing is wildly expensive, but you can only postpone that cost to some extent.

      Reply

    2. Hi Petri,
      this “might” be bad news for all future users of this great process you developed. It appears Debian GNU/Linux 9 (Stretch) is no longer available on the free tier option. I could be missing a step, but I just don’t see it anymore as an option.

      Anyone else have this problem?

      Reply

      1. Same on my site. Looks like it’s not possible to install Unifi without any dirty hacks. Sad.

        Reply

  295. Mine’s an old install (running debian-9-stretch-v20190124) which now I am unable to login to the web interface, it says
    “`
    Network Migration in progress…
    Database Migration in progress…
    Connection error. Please check your controller process state.

    unifi-network-service-helper[7140]: Aug 13 04:09:55 unifi: unifi is still running DB migration. Health check timed out.
    “`
    Any ideas?

    Reply

    1. If you are comfortable with the command line, try sudo reboot in an SSH window that you can start from the GCP console. You can also just Stop and Start it using the buttons. There are some scripts that try to fix common MongoDB problems at startup.

      Reply

  296. Anyone have any luck with the new Debian 10 and setting up the Unifi controller? Or, has anyone figured out how to force a Debian Stretch onto the GCP, as it’s no longer available as an option on GCP.

    Reply

    1. Have you tried the Debian Stretch image from the Marketplace? There are two: select the VM image and not the container image.

      Reply

      1. Debian 9 doesn’t seem to be available to install using that link do you have a way to make Debian 10 work ? By installing packages manually ?

        Reply

        1. Yeah, apparently that image was removed, too, after the support ended. There are hacks how to install Java8 and MongoDB on later Debians, but I can’t start supporting them. Google will guide you. Beware of any updates applied later on, they may or may not break your system. The real solution would be Ubiquiti getting their act together – finally.

          Reply

  297. Hi, this morning I received an email from google:

    Free Trial Ended, Charges Started From Aug 19, 2022
    Shut Down Project
    0 daysleft
    You enabled Compute Engine on Aug 19, 2022. This service is no longer free. Click here to view the billing details.
    To avoid future charges, you can shut down the project:

    I would like to know if other people have also received the email.
    Kind regards

    Reply

    1. Double check the free tier restrictions. As long as you don’t exceed them you won’t be charged. You can check your billing in a couple of days to make sure.

      Reply

  298. It seems that a lot of people always have problems with the backups missing from their bucket metis after some time.
    I would suggest a change in “gsutil rsync -r -d”, either
    Remove the -d flag, that way backup wont fail when the script dont have acces permisson to delete object from the bucket
    or
    Add the -C flag, that way errors during delete object will throw a log error but the procces will continue to attempt to copy the remaining files

    /etc/systemd/system/unifi-backup.service

    Reply

    1. Thank you for your input! I’ll add the -C flag to my script, but ATM I can’t run tests because Stretch fell out of support. I’ll think of something..

      Reply

      1. Hope it helps, this are the command lines to install the controller on Debian 11 Bullseye

        $ sudo apt update
        $ sudo apt install -y ca-certificates apt-transport-https wget
        $ sudo wget -O – https://www.mongodb.org/static/pgp/server-3.6.asc | sudo tee /etc/apt/trusted.gpg.d/mongodb-org.asc
        $ sudo wget -O – https://dl.ui.com/unifi/unifi-repo.gpg | sudo tee /etc/apt/trusted.gpg.d/unifi-repo.gpg
        $ sudo echo ‘deb https://repo.mongodb.org/apt/debian stretch/mongodb-org/3.6 main’ | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
        $ sudo echo ‘deb https://security.debian.org/debian-security stretch/updates main’ | sudo tee /etc/apt/sources.list.d/updates.list
        $ sudo echo ‘deb https://www.ui.com/downloads/unifi/debian stable ubiquiti’ | sudo tee /etc/apt/sources.list.d/100-ubnt-unifi.list
        $ sudo apt update
        $ sudo apt install -y openjdk-8-jre-headless
        $ sudo apt install -y unifi

        hope that helps you out to update the script and it should work on Debian 9/10/11 but i only tested in Debian 11

        Note: dont forget to update ur bash scrip from github the only one up to date is the one in ur GS bucket

        Reply

        1. Thanks! I’ll let this out ASAP in case someone else makes use of it. I tried installing MongoDB and Java 8 on Debian 10 when it came out, but the packages refused to install. I’ll try again when I have some spare cycles.

          Reply

          1. Just tested on Debian 10 Buster, and no problems it installed and manage to get into the setup promp page for the controller, i didnt do more test like fully setup the controller and adding devices to make sure everything runs fine, but i dont see a reason why it should not run ok.

            Reply

  299. This is my second GCP setup with Unifi Controller but this time installing on Debian 11; still working great following the YouTube guide. I followed the comment link and found from Uniquiti forum to install controller from here:
    https://community.ui.com/questions/UniFi-Installation-Scripts-or-UniFi-Easy-Update-Script-or-UniFi-Lets-Encrypt-or-UniFi-Easy-Encrypt-/ccbc7530-dd61-40a7-82ec-22b17f027776?page=349

    My question is that would your script of auto update/cert renew work with an already installed Unifi Application? Thanks.

    Reply

    1. You can follow the script steps manually. You can skip all of the if statements and decide for yourself whether you need to to execute that section or not. I know many have followed the script manually to install UniFi Controller on AWS or Azure.

      Reply

  300. Hi, maybe someone can help me here.
    I noticed that the LetsEncrypt certificate stopped updating.
    It expired on Saturday, October 8, 2022 at 6:58:38 PM and since then no update.
    I already restarted the VM but no luck
    I’m on Debian 9 stretch since 2019
    Controller works fine otherwise, only the certificate doesn’t update.
    Anyone has a clue as to why it doesn’t update?

    Reply

    1. Try renewing manually, so you will see the error message. Log in to the VM from the GCP Console via the SSH button and type sudo certbot renew

      Reply

      1. Hi Petri,

        Thank you for the feedback.
        Did what you instructed. Problem is probably with my public DNS. I activated DNSSEC a while ago and I think it’s causing some issues.

        Reply

  302. Few comments

    1. Regarding Fail2Ban, and configuring controller log level

    The log setting is now found under Settings [icon] > System [tab] > System Logging [section]. You need to uncheck “Auto” Logging Levels, and set Management logging to Verbose.

    2. Syntax fix for dbus start

    Instead
    while [ ! systemctl start dbus && $rounds -lt 12 ]
    Should be
    while ! systemctl start dbus && [ $rounds -lt 12 ]

    3. Debian bullseye (11) support

    I adapted bits from Glennr make this work with Bullseye
    https://community.ui.com/questions/UniFi-Installation-Scripts-or-UniFi-Easy-Update-Script-or-UniFi-Lets-Encrypt-or-UniFi-Easy-Encrypt-/ccbc7530-dd61-40a7-82ec-22b17f027776

    Since I am not sure what is the license of GlennR script, I am outlining the key steps made in that script.

    Key points are
    a) mongo db 3.6

    repo: https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6
    key: https://www.mongodb.org/static/pgp/server-3.6.asc | gpg –dearmor | tee -a /usr/share/keyrings/mongodb-server-3.6.gpg

    b) 1.0.0 (old) libssl

    Downloadable from http://security.ubuntu.com/ubuntu/pool/main/o/openssl1.0/ . Glennr script has grepping magic

    c) openjdk 11, and make sure unifi uses it

    installing openjdk-11-jre-headless
    call update-java-alternatives
    Clear /etc/ssl/certs/java/cacerts. Update ca certs with update-ca-certificates -f
    Comment out JAVA_HOME in /etc/default/unifi
    Configure JAVA_HOME in /etc/environment

    4. apt-key deprecated

    Makes sense probably to migrate away from apt-key at some point, and utilize signed-by syntax in apt sources. Glennr script seems to have the right steps

    https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html

    Reply

    1. Thanks for your input, especially #2: good catch. For 3c: Ubiquiti very definitely states that only Java 8 is supported. They must have some good reason for it, because it hurts their reputation really bad.

      Reply

        1. ..and later Debian releases specifically don’t support Java 8 for some reason, which I assume to be good, too.

          Reply

        2. maybe you are confusing which version of the controller you have installed,

          up to versions 7.2.x (only support for java 8)
          from versions 7.3.x (added support for java 11)

          note: mhh maybe theres no need to upgrade ur script right now petri, just wait till version 7.3 is released and u will only need to change 1 number in your script an 8 to 11, well and also stop using apt-key add since its deprecated

          Reply

          1. UniFi Network Application 7.3.76 released 11 days ago stated ‘Please note that Java 11 is required’

            Reply

            1. 7.3 is not yet available in the repo. I am worried how the transition will go. If it is just published, then the automation will apply it on the current VMs with Java 8. I cannot upgrade the script beforehand to install Java 11, because Java 11 won’t run the current controller.

              Reply

              1. I’m running successful 7.3 on my GCE VM Ubuntu 16.04.7 LTS for the last 3-4 days.
                What I’ve done was the following:
                – sudo add-apt-repository -y ppa:openjdk-r/ppa
                – sudo apt update
                – sudo apt-get install -y openjdk-11-jre-headless
                – sudo apt upgrade unifi

                So far, so good

                Reply

    1. 7.3 will be out Real Soon Now and it will require Java 11 and Java 11 only. (Won’t they ever learn?)

      Reply

      1. Do the current VM’s support Java 11 i.e. can we set up a new instance of Unifi Controller using Google VM with Java 11? Also, I don’t see where the bucket is located anymore on the GUI of the VM. Is that no longer a feature? I was hoping to download a current copy of my configuration.

        Thanks

        Reply

        1. Yes, the current VM should install Java 11 as a dependency as soon as the controller update requires it. It would make good sense to create a new VM with a supported Ubuntu version. I am going to update the script as soon as 7.3 is out and I can test the script.

          You can download backups from the top-left hamburger of the cloud console: Cloud Storage > Buckets

          Reply

          1. The package source is providing Unifi version 7.3.83. I believe it has been providing it for some time. For some reason auto-update wasn’t even trying to upgrade my instance.

            I successfully forced an update by executing `apt upgrade unifi`. That automatically upgraded Java to openjdk-11-jre-headless_11.0.6+10-1~bpo9+1_amd64.deb. Everything seems to be working. I am still on Debian 9 as expected, but am happy to help test a v11 script when you get around to it.

            I got three interactive prompts:

            – Do you have a backup? (I selected yes)
            – GRUB file conflict? I opted for the “package maintainer’s version”
            – GRUB disk installation? It saw two 15GB partitions. I selected both.

            Reply

            1. Thank you! There are some improvements that I want to add to v11. I’ll post it here.

              Reply

              1. Happy to report that I did the same as Jonathan and it worked very smoothly.

                I’ll wait till the next version of the script to replace the VM with Debian 11, but at least for now I’m on the current version of Unifi + Java 11.

                Reply

      2. I was able to update my Debian 9 VM running 7.2.96 UniFi to Debian 10 Buster running 7.3.76.
        I created a snapshot in case I needed to revert back.

        ## Steps I took
        I opened an SSH console in GCP
        used VIM and edited /etc/apt/sources.d/ list files to point from stretch to buster

        ran the following commands after updating all the lists
        sudo apt update
        sudo apt upgrade –without-new-pkgs
        sudo apt full-upgrade
        sudo reboot now

        I updated my ntp.conf when asked and kept my sshd.config when asked. Java 11 was downloaded automatically as a dependency and UniFi had a prompt if a backup existed before installing.

        Overall, pretty solid run to upgrade to Debian 10 and UniFi 7.3.x running Java 11.

        Hope this helps someone else.

        Reply

        5. Today I upgraded Debian 10 to version 11 using the same steps (I upgraded from version 9 to 10 earlier).

          Everything works fine.

          Reply

  304. Also was able to upgrade sucessfully using your instructions Derek.
    Advise: avoid hybernation of your laptop, because it will kill the ssh session during Apt upgrades 😉

    Question to Petri and/or Derek: should I delete the snapshot I made before the upgrade to make sure I stay on the free tier?

    Reply

    1. If you are happy with the result, then you can delete the snapshot. Storage is reasonably priced on Google Cloud, though (as it is on all cloud services). Paying a few bucks for the peace of mind is not wasted IMO.

      Reply

  305. I’d like to upgrade my Google Cloud instance to Debian 11. Does this process sound right:

    – Download controller backup and take snapshot of existing controller VM
    – Spin up new VM with Debian 11; omit “startup-script-url” and “bucket” metadata entries for now
    – Move static IP over to new VM
    – Install Java 11 and UniFi Network Application 7.3.83
    – Restore controller backup
    – Set “startup-script-url” and “bucket” metadata entries and reboot

    Did I miss anything, and will the script work in its current state? Or should I wait until the next update to make sure all the bits and pieces work with the new Java/application versions?

    Reply

    1. That sounds about right, but I haven’t tested it. I’d save the “Move static IP” for last. Derek and some others have made successful in-place upgrades. Use ctrl-F to find Derek’s report (2022-12-22).

      I am still waiting for the next version of the controller, which should upgrade old controllers. It should also automatically pull Java 11 as a dependency. I want to see how it works and be able to fix any problems it might cause.

      Reply

  306. Heads-up, for anyone wanting to delete the current VM, create a new one using Debian 11 and UniFi Network 7.3, these are the steps I made (after a lot of troubleshooting):

    1) Backup your configuration etc, delete / create the new VM
    2) Login via SSH (I use SSH-in-browser from GCP)
    3) Create a sudo password using ‘sudo passwd’
    4) Enter sudo using ‘su’
    5) Set ‘export PATH=”/usr/local/bin:/usr/local/sbin:/usr/bin:/root/bin:/usr/sbin:/bin:/sbin:/usr/local/games:/usr/games”‘
    6) Run ‘apt-get remove -y –purge man-db’ to remove man-db (I was having issues during installations with it, don’t know why)
    7) Manually run the following original or slightly modified commands from the script:

    – ORIGINAL –

    # Set up logging for unattended scripts and UniFi’s MongoDB log
    # Variables $LOG and $MONGOLOG are used later on in the script.

    # Turn off IPv6 for now

    # Create a swap file for small memory instances and increase /run

    # Lighttpd needs a config file and a reload

    – SLIGHTLY MODIFIED –

    # Run only these modified + additional commands for “install stuff”

    ‘apt-get install wget’
    ‘wget -qO – https://www.mongodb.org/static/pgp/server-3.6.asc | sudo apt-key add -‘
    ‘echo “deb http://repo.mongodb.org/apt/debian stretch/mongodb-org/3.6 main” | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list’
    ‘apt-get update’
    ‘apt-get -qq install -y apt-transport-https >/dev/null’
    ‘echo “deb http://www.ubnt.com/downloads/unifi/debian stable ubiquiti” > /etc/apt/sources.list.d/unifi.list’
    ‘curl -Lfs -o /etc/apt/trusted.gpg.d/unifi-repo.gpg https://dl.ubnt.com/unifi/unifi-repo.gpg‘
    ‘apt-get -qq update -y’
    ‘apt-get -qq install -y openjdk-11-jre-headless’
    ‘apt-get -qq install -y unifi’
    (let it run, it takes a while)
    ‘systemctl stop mongod’ # Used to be “mongodb” but we are now using MongoDB 3.6
    ‘systemctl disable mongod’ # Used to be “mongodb” but we are now using MongoDB 3.6

    # Set up automatic repair for broken MongoDB on boot

    Change the line of
    “Before=unifi.service mongodb.service”
    to
    “Before=unifi.service mongod.service” # Used to be “mongodb” but we are now using MongoDB 3.6
    and run all the commands

    8) Stop the VM instance
    9) Add the metadata you need to the VM, including the startup-script-url
    10) Start the VM instance and let it run it’s scripts (takes a while, especially for SSL)
    11) Enjoy!

    Hope that helps someone and especially you Mr. Petri if you’d like to update your script. You are a legend. Thanks for your work.

    Reply

  307. Hi Petri,
    Thanks for this awesome service! I’ve been running my controller on GCP for several years now. Everything is working great so far. If I do nothing (Not update to newest Java 11), will my controller stop working?

    Reply

    1. No, it shouldn’t. The next version of UniFi Controller is supposed to update Java to 11 as a part of its installation. The underlying Debian OS is out of support and may eventually be a security concern. At some point you should create a new VM with the latest Debian and restore a recent UniFi backup to it, but not just yet. I am still waiting because I want to see how smoothly the Java 11 upgrade will go on the existing controllers.

      Reply

  308. Hi, This is great! I can’t seem to get SNMP working any ideas? Should it work automatically once enabled in Unifi or would a port need to be opened in VM? Thank you

    Reply

    1. I’ve never used the SNMP in UniFi Controller, but you should allow at least UDP 161 in the GCP Firewall rules.

      Reply

      1. I have added to the UDP list (3478,161,162) but when I test for port open it is closed?

        Reply

        1. No idea. Like I said, I’ve never tried it. If you log on to the VM, do the ports show up in netstat -tulpn ?

          Reply

          1. I understand, no, 161 doesn’t show up. Others like 8443, 8880, 8080, 22, 80, 5514, 123 are on the list

            Reply

            1. Looks like there is no SNMP service running. Perhaps you can find some help in the Ubiquiti forums?

              Reply

  309. Google Sales Agent says the following: SKU: F449-33EC-A5EF for Compute engine has usage of 379.34 gibibyte hour, however the free tier is only for 30GB.

    How can this happen and is there something to do about this?

    Reply

    1. That sounds odd. Have you created multiple VMs over time? That sounds like there are some leftover hard disk floating around. My instructions are to create a single 15GB disk for the VM. There is no way to grow the disk afterwards. Check Compute Engine > Storage > Disks (and snapshots and images) what’s in there.

      Reply

      1. There is only one 15GB Standard Persistent Disk. Also, there is one backup bucket. The charges are around 6 cents per month. Not much but strange.

        Reply

  310. The afraid.org domain that I was using went offline back in June of last year. I had looked on here a time or two and was basically scared of messing with it, even though it wasn’t really working anymore.

    This morning I updated my metadata for the ddns-url and dns-name and started and stopped the instance. I can now login to the portal again and see my two sites. I have two sites and each have one AP. Very simple setup as you can tell. Both AP’s appear offline via the controller, even though they are up and working.

    Any clue what next steps should be? It might be, because the dns name has been wrong for so long. I did try to login via the IP address, but that always had issues as well. It’s been awhile, but I think it was the same “offline” problem with the IP variant of the site as well.

    Reply

    1. You and the controller itself know the new DNS name, but the devices are still reporting to the old address, which doesn’t resolve. They got it from the Override Inform Host in Settings > System. You should update that information also, but the devices won’t get it automatically, because they don’t know where to contact.

      They easiest remedy is to SSH to the devices. You will need to be in the same network, wireless connection works. If you have two sites, you will need to visit both. Check the SSH credentials from Network Device SSH Authentication in Settings > System. Once you are connected type set-inform http://new.dns.name:8080/inform to get the devices to try the new address. The devices will remember the old address and will revert to that in case the new one won’t respond. You know that the old one won’t respond, so you can use set-inform twice to overwrite the old address completely.

      Reply

      1. I looked everywhere last night for the password to ssh into the AP at my house and was dismayed that I had not documented it somewhere, so unlike me. Woke up this morning before looking on here and had a thought, I wonder if it’s on the controller somewhere and found it. Definitely felt like an idiot. But thank you for those steps. SSH’ing into the AP and running set-inform worked immediately.

        Reply

  311. Hi Petri! Amazing tutorial, I have been using it since a lot time ago.

    My Question is to you or Derek… I followed Derek’s instructions EXCEPT for the “sudo apt upgrade –without-new-pkgs” because GCC SSH said it was not a vlid command so I used “sudo apt upgrade” instead and everything is working fine… BUT how can I know if STRETCH migrated to BUSTER? Because GCC boot says it’s still stretch…

    Thanks for the amazing work!

    Reply

    1. Not yet. My plan is to get to it now in June. I’ve lost hope waiting for Ubiquiti.

      Reply

  313. Has anyone else’s controller started randomly losing its config? Today, for the third time, when I log in there are no access points and no clients. All three times this has happened, I tried rebooting the VM and APs to no avail. The only thing that fixes it is restoring from a backup (thanks, Petri, for the GCP cloud backup routine!)

    I think it might be time to rebuild my controller VM, but I’m holding out for an updated script 🙂

    Any advice?

    Thanks in advance!

    Reply

    1. No, I haven’t seen or heard of such. I run multiple VMs with no other worries except being stuck in 7.2.95. My plan is the get an updated script out by the end of June.

      Reply

      1. Hello, is there any way to use the current script? Even if the required Linux version is no longer offered by Google? Or is it only possible with a new, customized script?

        Reply

        1. I am working on a new version. I have been running a controller for a month now without glitches.

          If you want to try it: The only difference is to use gs://petri-unifi/startup2.sh as the startup-script-url. Use Debian Bullseye (version 11) as the base system. Java 11 is not supported on Bookworm any longer. You might consider this a beta release, since no-one else has tested it, yet. Report back here how it all went ????

          Reply

          2. Hi, it works well for several weeks for now. The only issue that I experience is, that the controller is down for several minutes from time to time. Then it takes about 5 – 10 minutes to come back.

            Reply

            1. I haven’t experienced this. Is it only the user interface or does the graph on the dashboard page also show a gap?

              I am working on a new release on Debian 12 (Bookworm) with UniFi Network Server 7.5 and Java 17. It will also include a new Fail2Ban rule to shield from http door-knocking.

              Reply

              1. It looks like, this process takes more and more CPU till it is at 100% and crashes the controller:

                /usr/bin/java -Dfile.encoding=UTF-8 -Djava.awt.headless=true -Dapple.awt.UIElement=true -Dunifi.core.enabled=false -Dunifi.mongodb.service.enabled=false -Xmx1024M -XX:+UseParallelGC -XX:+ExitOnOutOfMemoryError -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/usr/lib/unifi/logs/hs_err_pidunifi.log -Xlog:gc:logs/gc.log:time:filecount=2,filesize=5M –add-opens java.base/java.lang=ALL-UNNAMED –add-opens java.base/java.time=ALL-UNNAMED –add-opens java.base/sun.security.util=ALL-UNNAMED –add-opens java.base/java.io=ALL-UNNAMED –add-opens java.rmi/sun.rmi.transport=ALL-UNNAMED -jar /usr/lib/unifi/lib/ace.jar start

                Reply

                  1. How many devices do you have?

                    I have that setting in the script, but it is commented out.

                    Reply

                    2. That kind of explains ???? The tiny free tier VM is approximately the size of the first CloudKey, which supported less than 30 devices.

                    3. Hi, of course I have set up a much bigger machine:
                      vCPU: 4
                      Memory: 8,5 GB

                    4. There is another setting Xms, which is (IIRC) the initial amount of memory for JVM. Ubiquiti used to have a doc for larger installs where they explained all the bottlenecks.

                1. I haven’t tested it, yet, but you are welcome to try it out. Report back how did it go!

                  Reply

    2. Hi, my controller is off every hour. Takes 5 to 10 minutes- then it´s back. Also need to rebuilt it and hoping for a new script 🙂

      Reply

  314. Hi, I’ve followed Greg’s method and upgraded mi VM/Controller to ebian 10 and controller is now on version 7.4.156
    Everything seems to be working fine, VM is ok and I can access the controller just fine except that sinds the upgrade I lost SSH access to the VM.
    When I try to connect to the VM by means of SSH in the browser, it throws an error: “Connection Failed. You cannot connect to the VM instance because of an unexpected error. Wait a few moments and try again.”
    I get the option to troubleshoot the problem or retry, when I choose to troubleshoot, it states that the VM status is ok, the Network status is ok, but the wheel keeps spinning at User Permissions. So I suspect that it’s a rights issue but I don’t know what to do as I obviously can’t get in via SSH. it’s been now a couple of days already.

    Reply

    1. That sounds like a problem in the VM. I suggest you create a new one and move your controller there.

      Reply

  316. Hello Petri, I had to set the MGMT log level from MORE to DEBUG because server.log no longer tracked failed logins for fail2ban.

    Reply

    1. Hi Fabrizio,
      What does this do to the size of your log file? Is there any downside to selecting DEBUG mode?

      Reply

  317. Hi

    I used your tutorial for running the Unifi Controller on Google Cloud and it has been working successfully for the past 2 years.

    Now I got an error in the event log:
    “The Console data partition is nearly full. Only 431 MB of 9.78 GB free”

    Do I need to do anything or will that storage be freed automatically after some time?

    Thank you and best regards

    Reply

    1. Umm.. Do you mean The controller data partion…? I haven’t seen that problem myself, but a search on ui.com revealed plenty of evidence. It seems to plague especially the Dream Machines. There is a prune script to clean up the MongoDB databases.

      If you want a quick fix: Take a backup, create a new VM with Debian 11 and startup-script-url: gs://petri-unifi/startup2.sh, move your IP address to the new VM and restore the backup. Create a new backup bucket or at least use a new directory. Otherwise your old backups will be deleted. Everyone will need to do this soon anyways, as support for Debian 9 has ended.

      If you want to debug this: Log on to VM over SSH and try to find the overgrown file. Use sudo find / -type f -size +100M 2> /dev/null There shouldn’t be many hits: the swap file and the kernel core – and I have a Google SDK and the Java module database.

      Reply

      1. Thank you, I managed to increase the partition size on Google Cloud so pushing out the real fix for a bit :)!

        Reply

  318. The New version that was just released 7.5.172 requires Require MongoDB 3.6 or newer. This also drops support for 32-bit systems.

    I followed the install in the past of this. Current version running is MondoDB 3.2.

    Do you have a procedure or guidance how to update to a newer version?

    Reply

    1. There is a new version of the script for Debian 11, which will install Java 11 and MongoDB 3.6. The name is starup2.sh.

      Short instructions: Take a backup, create a new VM with Debian 11 and startup-script-url: gs://petri-unifi/startup2.sh, move your IP address to the new VM and restore the backup. Create a new backup bucket or at least use a new directory, otherwise your old backups will be deleted.

      Everyone will need to do this soon anyways, as support for Debian 9 has ended. I am working on a new web page for this.

      Reply

      1. Hi Petri,
        by “soon” do you mean we have a few months before we need to update to Debian 11 or only a couple of weeks? What happens if we do nothing and continue to use the old Debian?

        Reply

        1. Debian 9 is out of support (and has been for a while already). If someone discovers a security vulnerability in it – or any of its components – you are out of luck. You should upgrade before that occurs. Other than that, the old VMs will keep running till kingdom come, but there won’t be any updates to the system or UniFi Controller. You are stuck at 7.2.95. BTW, 7.5 was just released.

          Reply

      2. Hi Petri,
        A couple of questions before I begin my update.
        1. How do I move my IP address to the new VM?
        2. After I create new VM and move my IP address and restore backup, should I delete the old VM and all its contents? (Not sure how much space I’m using with the old VM).

        Thanks

        Reply

        1. Sorry,
          one more question (I’m trying to make sure I do this right, and not lose the free tier)
          When I originally set up my VM (three years ago) I committed the horrible sin of not following your advice on creating a 15GB Bucket. Rather, I created the full 30GB Bucket (see below)
          unifi-controller
          Details
          Monitoring
          Properties

          Type
          Standard persistent disk
          Size

          30 GB
          Architecture
          —
          Zone
          us-east1-b
          Labels
          None
          In use by
          unifi-controller
          Snapshot schedule
          None
          Source image
          debian-9-stretch-v20181113
          Encryption type
          Google-managed
          Consistency group
          None

          My question is can I make a new VM with a brand new bucket (and keep the old bucket) and if not, how do I make a subfolder on the the old bucket for the new VM? Thanks

          Reply

          1. The free offer still stands for e2-micro instances in us-east1/central1/west1 regions.
            The bucket is for backups and it is in free tier until you have more than 5 GB of data in it.
            The 30 GB is for the boot disk of the VM. You can create a new 15 GB disk for the new VM, but you will have to pay for the seconds they are both in use. As soon as you delete the old VM the old 30 GB disk is released. That is, you might get charged less than a dollar, typically (unless you forget to delete the old one).

            Reply

        2. 1. In GCP Console go to VPC Networks > IP Addresses. At the right end of the row for the external, static address there are three dots. That’s how you can reassign the address to the new VM.
          2. Yes! Once the UniFi devices appear as online in the new Controller you can safely delete the old VM and its disk. As long as you are running the two side-by-side you are over the quota for free tier.

          Reply

          1. Thanks Petri.
            Another question:
            When I set up a new VM, for the firewall settings the only option I have is to enable HTTP or HTTPS.

            “Firewall
            Add tags and firewall rules to allow specific network traffic from the Internet
            Allow HTTP traffic
            Allow HTTPS traffic”

            Do my old firewall settings from the original VM work for the new VM?

            Reply

            1. Use the same network tag (or create a new one with the same settings as in the article above).

              Reply

      3. Hello Petri, after creating a new VM with Debian 11 I had to level up the Management log to Verbose because the server.log no longer kept track of failed logins for fail2ban.

        Reply

    2. Also, do you have any thoughts on the process to upgrade to the latest version of Debian from 9.13 and also switch to a 64bit version

      Reply

      1. The upgrade info is in the earlier reply. It will install Debian 11 with Linux 5.10 on amd64 architecture.

        Do not try to use later than Debian 11 as Java 11 won’t be available.

        Reply

        1. Ok, seemed to work. I forced inform host to the IP on old controller. Took a backup, stopped the VM, removed the IP from the existing VM, created new storage, new virtual machine, and assigned the old IP, with the startup2 script. Loaded from backup and then deleted the old datastore and old VM.

          I also installed google agent monitor. Have you had any issues with that?

          Reply

        2. Unifi Controller version 7.5 now requires Java version 17, so Debian 12 could be used havent tested yet…

          Reply

          1. Thanks for the heads-up! That is good news. I’ve been waiting for a back port of a more modern Lighttpd version, but that is already included in Bookworm. I’ll finally be able to move forward.

            Reply

            1. quick test, runs on WSL2 Debian 12 just need to install libssl1.1, it can be obtained from the “bullseye main” distro

              I will se if i can get some time tomorrow and spin a VM on GCP, to test there.

              Reply

              1. Thanks! I am still testing my Lighttpd rules. I’ll create a new post when I am happy with them.

                Reply

                1. According to this post https://community.ui.com/releases/UniFi-Network-Application-7-5-187/408b64c5-a485-4a37-843c-31e87140be64 – Mongo 4.4 can be used

                  curl -LfsS https://pgp.mongodb.com/server-4.4.asc | gpg -o /etc/apt/trusted.gpg.d/mongodb-server-4.4.gpg –dearmor
                  echo “deb [ signed-by=/etc/apt/trusted.gpg.d/mongodb-server-4.4.gpg ] http://repo.mongodb.org/apt/debian buster/mongodb-org/4.4 main” > /etc/apt/sources.list.d/mongodb-org-4.4.list

                  Tested on WSL2 Debian 12 and works

                  Reply

                  1. Thanks! I’ll work that into the next version for Bookworm and Java 17.

                    Reply

  319. Update on my controller losing devices and clients:

    I followed the instructions above and built a new VM with the V2 script, moved my IP, and restored the backup – easy peasy! The whole process took maybe an hour.

    So far, so good – previously I had to restore a backup in order to see APs and clients, and after a few hours they would disappear again. New controller has been online since about 10pm last night, just logged in and it’s all there.

    If it does happen again, I would start to suspect a bad config in my backup. In that case I will go nuclear and just rebuild the whole controller config on another clean VM. Hopefully it won’t come to that 🙂

    Thanks again Petri, your hard work and support is greatly appreciated!!

    Reply

    1. Update to the update: just logged in, and the controller lost the APs and clients again. Oddly enough, the mobile app still works and shows everything just fine. I’d say it could be a bug in the controller but this has been happening since the previous release. Anyone else seeing this?

      Reply

      1. SOLVED! Apparently there was a problem between the keyboard and the chair 🙂

        It seems that sometimes when I log into the web interface, it loads the default site, not my named site. Since I’m not using the default site, it makes total sense that there are no APs or clients to show. I’m not sure why the browser is flipping back and forth on different logins, but there you have it.

        Reply

        1. Apparently there was a problem between the keyboard and the chair
          You are not the first one, trust me 🙂 I’ve been bitten as well.

          Reply

  320. Anyone know if we can still get the free VM on Google, if we upgrade our VM to Debian 11? I keep seeing where I’ll be charged $7 a month.

    Reply

  321. I must have done something wrong. Can’t log into new VM at all.
    Does anyone have step-by-step instructions on how to upgrade from a previous VM?
    I tried following the steps outlined at top of this blog, but some of the settings have changed on google and I’m not sure if I’m transfering my old ip to my new VM.

    Reply

    1. Also, my internal IP changed on the new VM (External is the same). is that normal?

      Reply

      1. Strange, I can ping my DDNS and the result is my VM external IP. However, I am unable to log into the site via web browser.

        Reply

        1. I’m at a loss. Created another VM according to all instructions above and I still cant log into web GUI for Unifi. I can ping my DDNS URL and it gives back the correct IP, but it will not allow me to log into splash page of GUI (it doesn’t even show up). All I get is the proverbial “Unable to connect. An error occurred during a connection to mysite.duckdns.org:8443. ”

          Anyone have any thoughts?

          Reply

          1. The third time’s a charm!
            Disregard the previous questions (a slew of questions). After the third build of a VM, it all works now. Sorry for the “sky is falling” messages. I wasn’t sure if I was doing everything right. If at first you don’t succeed, try try again (at least three times).

            Reply

  322. Hello Petri, I had to set the management Logging Levels from Normal to Verbose because server.log on new Debian 11 also no longer tracks failed logins for fail2ban.

    Reply

  323. Hi Petri,
    Setup a new GCP VM yesterday, using your v2 script and successfully restored the corresponding UI Controller backups. Worked great with no issues on the UI Controller. There was a noticeable performance improvement afterwards as well. Ignore the “cost calculations” GCP gives you on the way based on selection, the free tier discount isn’t reflected. IMO, this is the most painless way to get the latest UI Controller running on GCP when people used the old script.

    One thing I’m curious about, the VM occasionally throws the below rpc error in the GCP logs. Either 502=bad gateway or service unavailable. Any hints why?

    {
    “insertId”: “some-id”,
    “jsonPayload”: {
    “localTimestamp”: “2023-09-14T18:09:59.4778+08:00”,
    “omitempty”: null,
    “message”: “Error waiting for task (attempt 1 of 10): rpc error: code = Unavailable desc = 502:Bad Gateway”
    },
    “resource”: {
    “type”: “gce_instance”,
    “labels”: {
    “instance_id”: “some-instance-id”,
    “zone”: “us-west1-a”,
    “project_id”: “yup-thats-my-project”
    }
    },
    “timestamp”: “2023-09-14T10:09:59.488031295Z”,
    “severity”: “WARNING”,
    “labels”: {
    “agent_version”: “20230504.00-g1”,
    “instance_name”: “my-instance-name”
    },
    “logName”: “projects/my-project/logs/OSConfigAgent”,
    “sourceLocation”: {
    “file”: “agentendpoint.go”,
    “line”: “414”,
    “function”: “github.com/GoogleCloudPlatform/osconfig/agentendpoint.(*Client).WaitForTaskNotification.func1”
    },
    “receiveTimestamp”: “2023-09-14T10:10:00.561492753Z”
    }

    Reply

    1. I haven’t seen that, but it seems to be just a warning about the monitoring agent not reaching its reporting target. I wouldn’t worry about it. You can of course try to approach Google about it.

      Reply

    1. Hawkeye! Well done! This will be fixed in the new version. I have finally convinced the Lighttpd author to backport us a version that Fail2Ban can act on. This should cut the charges some have received for excess egress traffic.

      Reply

  325. Is anyone getting charged for their “free” tier on GCP? I just got a bill for $1.51 from Google. I can afford the $1.51, but wondering why I’m being charged anything. This started happening when I upgraded to Debian 11.

    Reply

  326. My original install was still going strong from many years ago thanks to your awesome work on this. I did the in place upgrade using the v2 of your script and it seem to work good but I figured I should do a fresh install to clear the junk out. One thing I noticed is that the micro was not enough to process my 400MB backup file. It took hours for it to complete the process. When all was said and done and things were online then I restarted. After a restart it takes a good 5 to 10 minutes for the web page to show up using micro on the fresh install and uses about 60% of alloted memory just idle. Maybe I just need to bump it up a little and pay for the extra tier versus the micro. I have 15 sites and about 70 devices.
    Thank you for the work you have done on this and the upkeep. Cant wait to see the new site for the new version. If you need me to check anything or test anything with the latest stuff let me know. Just wanted to share my update progress.

    Reply

  327. I’m running version 7.4.162 from the controller on Google Cloud Services and like to update it to 8.0.7.
    Is there an easy way to do so?

    Reply

  328. Hi Petri,
    I’m likely the only person who has done this, but thought I’d pass on the info nonetheless. So I found out why I was being charged ~$1.50 a month for the “free” tier on GCP? I was able to chat with Google tech support and they told me I was using a “Balance PD capacity Disk” vice a standard disk. I must have missed that during setup.

    So I decided to delete the whole thing and refresh the controller. I followed all the instructions, but now I cant connect to the Unifi controller on Google. I can ping my reserved IP address, but the web browser (tried both Chrome and Firefox) will not connect. I get the dreaded “This site can’t be reached. myaddress.duckdns.org refused to connect”

    Any idea what I’ve done wrong?

    Reply

  329. I’ve tried three times to rebuild the controller and the same problem. I can ping both the IP and DNS of my controller and all is well. However, I can’t access the web gui. Just wont connect (Also my Unifi IOS App will not connect). I tried looking at the logs and found this of interest:
    jsonPayload: {
    localTimestamp: “2023-12-17T11:13:03.2402-05:00”
    message: “startup-script-url: E: Unable to correct problems, you have held broken packages.”
    omitempty: null

    Not sure if that has anything to do with it.

    Reply

    1. Sorry for the barrage of questions (and problems) but I’m still batting zero.
      I put together my new VM with debian-11-bullseye-v20231212. Am I required to use the startup script “startup2.sh” or should I continue to use the original version “startup.sh”? I’ve tried both but still can’t access the GUI logon screen (However I can ping both the IP and the FQDN).

      Thoughts on what might be wrong?

      Thanks

      Reply

      1. Hmmmmm, strange. Rebuilt VM now 6x and it still won’t connect to gui (will connect to FQDN and IP via ping). Does anyone have any thoughts on what’s up? No idea why it wont connect to GUI splash page.

        Thoughts?

        Reply

        1. Hate to be a pest, but I can no longer get this to work (despite it working for 3+ years).
          I used the default boot disk, which was Debian 12. I see that might be a mistake. Here are the options now for building a VM on GCP:
          Debian GNU/Linux 10 (buster)
          Debian GNU/Linux 11 (bullseye) [Arm 64]
          Debian GNU/Linux 11 (bullseye) [x86/64 AMD 64]
          Debian GNU/Linux 12 (bookworm) [Arm 64]
          Debian GNU/Linux 12 (bookworm) [x86/64 AMD 64]

          Which one should we use if we use the new startup script gs://petri-unifi/startup2.sh?

          For the life of me, I can’t connect to the GUI, yet I can ping the FQDN and IP all day long. Could use some help from anyone in the know, or who might have any ideas.

          Thanks

          Reply

          1. I’ve rebuilt this VM about ten times now with no luck on accessing GUI.
            I’ve looked in my logs and I keep getting this error message:
            startup-script-url: E: Package ‘mongodb-org-server’ has no installation candidate
            startup-script-url: E: Unable to correct problems, you have held broken packages.

            Is that the problem?

            Reply

            2. Petri, could use your help. Tried numerous times to rebuild my controller. Even set up a a new GCP account with $300 in credits. Same result: Can ping my reserved IP address and the FQDN (DDNS) but cannot connect to the controller (GUI). Any ideas on what I’m doing wrong?

              Reply

              1. By chance I come across the issue during my rebuild.
                Seems mongodb-org-server can not be installed (which is critical for unifi binary).

                My quick workaround, instead using 3.6, I setup 4.2 for apt repo.
                cmd> curl -fsSL https://pgp.mongodb.com/server-4.2.asc | sudo gpg -o /usr/share/keyrings/mongodb-server-4.2.gpg –dearmor

                cmd> echo “deb [ signed-by=/usr/share/keyrings/mongodb-server-4.2.gpg ] http://repo.mongodb.org/apt/debian buster/mongodb-org/4.2 main” | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list

                After that, I did a reboot and let the startup did it work. I monitor the whole process via Serial Console and a logged session. It took TOOOOOOO LONG (more than 20 minutes) to allow my unifi login prompt reappeared. 🙂

                Reply

                1. Just came here to say this was the trick. Realized my controller was a bit too out of date on the old Debian when I couldn’t adopt a newer access point. Tried everything I could think of to get it going. Along with the new script.

                  This finally worked like a charm!

                  Reply

            1. Thanks Xariwey. I’m running Debian 11 and startup2.sh. Still no Joy. Can ping the VM and FQDN (DDNS) but the controller will not open. No clue what could be wrong.

              Reply

  330. I was thinking about upgrading to 8.x. Has anyone tried to do it yet? if so, any issues?

    Reply

      1. I had auto upgrade set with in UI admin. I woke up this morning and it had upgraded itself from 7.x to the latest version of 8.x on its own.

        Seems to be working fine

        Reply

        1. Yes, the version 8 finally hit the repos. It took like a month this time. Usually it has taken a couple of weeks.

          Reply

  331. By chance I remembered my little GCP after running 5 years without hiccups, I come here and check what’s going on.

    I took Petri’s comment – Back it up, Start a new VM with startup2.sh, which did not work as expected. Further deep-dive and found it might be something wrong at mongodb site – apt update got failed for 3.6 repo.

    I make a quick comment earlier which tried to add 4.2 repo and it worked. And the result is interesting – I got a unifi 8.0.24 installed. 🙂

    However it is not a perfect fix as it complained for apt error. I think it is a better workaround.

    What I did to get a new 8.0.24 Unifi Controller

    1. Stop my old VM
    2. Create a new VM with meta startup-script-url: gs://petri-unifi/startup2.sh, Take old IP
    3. Add one more meta “serial-port-enable : TRUE” to allow serial console (for state monitoring
    4. Let it run, and got failed (it took around 30+ mins). Make sure startup2.sh stopped by using “ps -ef | grep startup”
    5. Login VM as root (start SSH, then sudo -iuroot)
    6. comment (add a #) first line of /etc/apt/sources.list.d/mongodb-org-3.6.list
    7. Follow instruction from https://www.mongodb.com/docs/v4.4/tutorial/install-mongodb-on-debian/#install-mongodb-community-edition to add related files for mongoDB 4.4
    8. cmd> apt update
    9. reboot and let startup2.sh do the rest

    Reply

    1. Thanks MacJuJu. I was starting to think I was losing my mind trying to get this to work. I’ll try your method tonight when I get home. Thanks for the update.

      PS. Using your method, does everything work fine now? Did this fix the “apt error” you were getting? If not, what is the impact of “apt error’ i.e. does it matter?

      Hopefully, Petri can put his magic touch on this and fix the problem. I suspect he’s on a well-deserved vacation right now.

      Thanks

      Reply

        1. Thanks Xariwey and Happy New Year.
          Can I put your script in like Petri’s’ URL script i.e.” startup-script-url gs://petri-unifi/startup2.sh”?

          If so, do you have the proper gs:// script link e.g. gs://Xariwey-unifi/startup2.sh? (I may be asking a silly question, but I’m unclear on how Linux reaches out to grab a script). Not sure what to put in the startup-script-url option. Thanks again

          Reply

  332. Just drop it in your own Google bucket, i asume u already have one if u followed this guide its the one u should be using to store the backups…

    just drop the file there and use it from there

    Reply

    1. I thought the buckets only backed up the controller (not the VM). Am I wrong on that?
      If the VM is in the bucket (drop your file in there) how do I restore the VM to your file?

      Apologies for not following as well as I should.

      Thanks

      Reply

      1. No problem, but i never say backups for the VM, i just say that if u followed pettri guide u should have a Google Storage Bucket, and implied that it is used for backups, ( unifi controller backups ),

        well that bucket is nothing more that just some storage in the cloud, u can store there the script yourserlf, well just go to your bucket and upload there the file. then grab the url of that file it should look like ( gs:// “your bucket name” /startup2.sh ), and use that url as startup script following the guide from pettri.

        also just in case, did u try using the complete url of pettri script, its is reported that sometimes the GSUtils url dosnt properly work, here try with this first https://storage.googleapis.com/petri-unifi/startup2.sh

        Reply

        1. Thanks Xariwey. I’ll give it a go. I did try the https:// method for Petri’s script, but it didn’t work.

          I will try your method next. Thanks for all the advice.

          Reply

    1. Does this mean we are out of luck until key is renewed (if ever)?

      At this point, I’m willing to go way back to original Debian package (despite security vulnerability) and install an old controller. With the expired key, will I be able to do that? I need something, as right now I cannot control my Unifi equipment via GCP, and I’d rather not configure a local controller and reset all the “inform” settings.

      Thoughts?

      Reply

      1. Who knows when or even if the mongo securty key is going to be updated since mongo db 3.6 is already in End of Life, that means no more updates or support for it.. thats why Ubiquiti updated the controller code to use java 11 then java 17 and mongo 4.4 remember that way back when pettri made this tutorial and the script, unifi used to use java 8…

        my suggestion is to use my startup2.sh script it only has 2 changes compared to pettris script

        type that in the [startup-script-url]
        https://raw.githubusercontent.com/Xariwey/GCP-Unifi-Controller/master/startup2.sh

        Note. also u dont need to delete the entire vm and start all over again whenever u want to try new things or wanna do a fresh install, all u need to do is

        1.- Stop VM
        2.- Detach then Delete boot drive
        3.- Create new Boot Drive for the VM
        4.- Start VM

        this will pretty much work like a format when the VM start it will run startup script and it will install everything u need. and since the backups buckets is still there u have all ur backups easy to restore on ur new controller, ur IP its the same ur firewall rules are the same everything is the same easy peasy…

        Reply

          1. And there it is!!!!!! Xariwey, you are awesome. Works great now. Finally, after many weeks I can log back into my controller. I used the Debian 12 boot disk and your script and everything worked perfectly. Can’t thank you enough!

            A word of caution for anyone updating their controller using this method. I missed the part where Petri said; “Don’t set up two controllers to back up to the same bucket. They will delete each other’s backups.” Unfortunately, building this new VM I wiped out my previous bucket. Fortunately, I downloaded my backup file before building this VM. I don’t know that I would ever need the old backups, but it was kind of cool they were there (dating back to 2018). I guess I’m just being nostalgic. That said, be sure to create a new folder in your bucket. Otherwise, it will overwrite all the old files. Or, download your most recent backup to your computer so you have a copy on hand. Thanks, Xariwey.

            Reply

        3. Should this script still work? I was still running on ver 7. I bought one of the new USW pro switches for which the specs say it requires ver 8. I thought I would be good to go. Selected a new Debian 12 machine and selected above startup-script-url. In the logs I can see things like MongoDB,logrotate and my dynamic DNS. But nothing unifi related. Any tips on getting a V8 controller online would be very much appreciated. I am no Google cloud guru. So far I have managed fine following steps and scripts..

          Reply

          1. It should still work for version 8, but pick Bullseye (Debian 11) as the image. I haven’t created new VMs lately, but it is on my to-do-list 😁

            Reply

        4. I used Xariwey’s script and process, but ran into an issue with the new Debian 12 image. GCP kept giving me a UEFI boot error when attaching a new image/boot disk. Easiest fix I’ve found is to just click into the original VM, then click on create similar. This will essentially duplicate the VM for you but allow you to choose your settings; make the new boot disk then update the script. Only downside to this is the public IP will change, but if your DNS/url is the same then your gear will find the new VM regardless. Took about 15 minutes for the script to fully complete, now everything is working as it used to but on 8.4.x!

          Reply

    1. Yes, I still am. I’ve been busy with some paid projects lately, but I plan to get out a new version of the script for latest Debian and MonogDB.

      Reply

      1. Hi Petri. Thanks a lot! Looking forward to the new startup script and updated blog post… sometime in the near future hopefully? 🙂

        I suppose in the meantime the best thing to do for new installs is to use Xariwey’s script per https://metis.fi/en/2018/02/unifi-on-gcp/#comment-34489 ?

        I am only getting my feet wet with Google Cloud and will attempt to migrate my controller currently hosted at home.

        Reply

        1. Yes, and you can always spin up a new one if something better comes up!

          Reply

        2. I successfully used Xariwey’s script (Shout out to Xariwey). His script was a blessing for those caught between the Java and MongoDB changes. Although (not because of Xariwey’s script) I have noticed my controller is much slower with all the new upgrades. It’s fine by me as once I set all the controller options, I kind of set it and forget it.

          That said, I am looking forward to Petri’s script. Perhaps he can tweak the controller speed.

          Reply

  335. Petri – Do you know when you may be updating your script to cover the latest and greatest? I know you are probably very busy with other stuff

    Reply

    1. My predictions have been worthless so I have tried to abstain from further guesses. Yes, I am working on a revised script with support for the latest versions of everything and some kind of choke for the egress data flow. Use Xariwey’s fork if you are in a hurry. You can always spin up another with more bells and whistles when it comes available.

      Reply

  336. Petri , I would like to thank you for you scipt I used it for years. But nowdays the free Google VM is not the best option to host Unifi controller so I moved to Oracle cloud and host it in docker container as OCI is lot more generous with free stuff than Google

    Reply

    1. Hi Peter,
      You have my attention! Do you have instructions on how to do this on Oracle?
      Is Oracle any faster than Google (in terms of processor power)?
      Thanks

      Reply

    1. Not im happy with the GCP services, and yes i had noticed that with the recent updates using OS and software more up to date kinda push the limits on the free tier GCE, and the UNIFI Controller WEB Interface its kinda slow, but this guide is meant to be an easy setup and forget thing, so get ur controller up and runing make the configurations u need and dont worry about it until it crash in 2-3 years, and if that happens then u can come back here and look for answers hopefully the community will already had a fix or a workarround.

      Reply

      1. Thanks for all the script Petri and Xariwey, just thought I’d let you know the 8.x interface on Debian 12 is far more responsive if you accidentally use a Balanced Persistent Disk. It even allows you to view coverage/radio views, refreshing the charts within a couple of seconds. Sadly it’s not the same with Standard disk although it may be if you limit the history under settings to a day or less as it appears to perform ok after a reboot when the history is wiped. I assume this is just due to the Balanced disk giving ssd performance but does, in my opinion, at least highlight that the bottleneck isn’t the cpu.

        Thanks again for all the help!

        Reply

        1. In GCP the performance of a Disk its directly tied to its zise, if u wanna increase performnace just use the entire free 30gb

          or use a balanced disk with just 10gb ur gonig to get 3x the performance of a 30gb free standard disk

          Reply

  338. I appreciate Petri’s script as I have been using it for several years. Oracle offers more free resources then google and I just moved to there. Someone from Unifi wrote a script to build it in oracle

    Reply

    1. Hi Scott,
      can you provide a link to this script? Also, does the script offer the same things as Petri’s script e.g. “Lets Encrypt”, Fail2Ban, etc?

      Thanks

      Reply

      1. /////https://get.glennr.nl/unifi/install/unifi-8.1.113.sh////////

        WOuld you let me know if you approve of the script or if anything is missing?

        Reply

      2. After install, there is a encryt script to run and also a fail2ban script to run. I have not have time yet. Will do those two scripts next week

        Reply

  339. Thank you to all who have contributed to this (particularly Xariwey & Petri). It took me a while to get it all going as I didn’t read to the bottom of the threads before starting.
    My only suggestion would be to avoid using 0.0.0.0/0 as the ingress address for the server especially if you have a static IP. There’s no point exposing the interface to the entire internet if you don’t need to.
    Anyway a very good job and a good learning experience for me.

    Reply

  340. Hi Robert,
    what do recommend the Ingess IP be changed to? I assume you’re talking about the Ingress on the firewall page. What is the impact on controller access if we change the Ingress IP? Thanks

    Reply
  341. Pingback: The new home network, part II | Benny Ling's Bling

  342. Can`t get this to work even with the startup2.sh script unfortunaly…
    Hope one day this gets fixed. Can`t seem to find any relevant info in the Console as well. It displays info until the swap file is created, then nothing more is displayed in the console.

    Reply

    1. 8.4 isn’t out yet. Your controller should update automatically to 8.2 at least. (That’s what I am running currently)

      Reply

  344. Just to say that I have migrated to Oracle using half your free allowance and now have an VM.Standard.A1.Flex instance with 2 ocpu and 12 GB memory. You need to upgrade from the free account as you cant get your server created under free anymore, no resources, but as long as you stay under the limits of free then you won’t be charged.
    So why am I posting, just to say the usability is like night and day. Maybe I had some issues on my google cloud deployment but on the more recent upgrades it was really starting to grind, 10 sec for the login page then 20 sec to show the dashboard and a further 30 sec to load the settings page. On Oracle, it is now totally responsive. I just host 10 aps across two site, but one is a rental place so I have to be able to help out if clients have issues.
    Anyway, google is fine for the setup and forget but Oracle is the way to go is you want to actually use the controller. It is more “fun” to configure and no way as slick as the solution created by Petri here.
    Maybe someone will help with a full scripted install an set of instructions for there, still haven’t sorted backups and some of the other niceties but it is sooo much better.

    Reply

  345. James A,
    can you post a step by step on how you moved it to Oracle? I agree, the GCP is almost unusable if you actually want to use your controller (which I do).

    Reply

  346. Does anyone have the steps needed for moving to Oracle (Like Petri did w GCP)?
    My controller is almost unusable on GCP. It works, but sloooooowly.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.