---
title: How to manage high availability
description: Canonical makes open source secure, reliable and easy to use, providing
  support for Ubuntu and a portfolio of enterprise-grade technologies. Founded in
  2004, Canonical operates globally with team members in over 80 countries.
url: https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md
---

*Submit*

# How to manage high availability

Region and rack controllers are the backbone of MAAS. By deploying multiple controllers, you automatically gain load-balancing and failover. This ensures that machine provisioning and API access continue even if one controller goes offline.

This page explains how to enable and manage HA for both rack and region controllers, as well as supporting services like PostgreSQL, DHCP, and the MAAS API.

## [Enable HA for rack controllers](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-enable-ha-for-rack-controllers)

Adding a second rack controller automatically enables HA.

### [Snap install](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-snap-install)

```
sudo snap install maas
sudo maas init rack --maas-url $MAAS_URL --secret $SECRET
```

### [Package install](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-package-install)

```
sudo apt install maas-rack-controller
sudo maas-rack register --url $MAAS_URL --secret $SECRET
```

The `$SECRET` is stored at:

* Snap: `/var/snap/maas/common/maas/secret`
* Package: `/var/lib/maas/secret`

You can also follow the UI path: *Controllers* > *Add rack controller*.

## [Manage rack controllers](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-manage-rack-controllers)

Rack controllers manage networks and connected machines.

### [List racks](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-list-racks)

```
maas $PROFILE rack-controllers read | grep hostname | cut -d '"' -f 4
```

Multiple racks are required for HA. Ensure VM hosts can reach each rack controller.

### [Delete a rack controller](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-delete-a-rack-controller)

* UI: *Controllers* > select controller > *Delete*
* CLI:

  ```
  maas $PROFILE rack-controller delete $SYSTEM_ID
  ```

If the controller participates in DHCP HA, disable DHCP HA first. If you don’t remove the software, a reboot will reinstate it.

### [Move a rack controller](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-move-a-rack-controller)

Moving between MAAS instances or versions is not supported and risks data loss. Instead:

1. Delete the old controller.

   ```
   maas $PROFILE rack-controller delete $SYSTEM_ID
   ```
2. Register a new one.

   ```
   sudo maas-rack register --url $NEW_MAAS_URL --secret $NEW_SECRET
   ```

### [Common pitfalls](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-common-pitfalls)

* Don’t run a rack and VM host on the same machine (resource contention).
* Don’t move rack controllers between MAAS versions.
* Don’t connect a rack controller to multiple MAAS instances.

## [Manage region controllers](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-manage-region-controllers)

Region controllers coordinate racks and present the API/UI.

### [Add a region controller](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-add-a-region-controller)

On a secondary host:

```
sudo apt install maas-region-api
```

Copy `regiond.conf` from the primary API server, set ownership, and point it to the primary PostgreSQL database:

```
sudo systemctl stop maas-regiond
sudo scp ubuntu@$PRIMARY_API_SERVER:regiond.conf /etc/maas/regiond.conf
sudo chown root:maas /etc/maas/regiond.conf
sudo chmod 640 /etc/maas/regiond.conf
sudo maas-region local_config_set --database-host $PRIMARY_PG_SERVER
sudo systemctl restart bind9
sudo systemctl start maas-regiond
```

Check logs for errors.

### [Enable HA PostgreSQL](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-enable-ha-postgresql)

All region controllers share the same PostgreSQL DB.

1. Allow the secondary API server:

   ```
   echo "host maasdb maas $SECONDARY_API_IP/32 md5" | sudo tee -a /etc/postgresql/9.5/main/pg_hba.conf
   sudo systemctl restart postgresql
   ```
2. Add the region controller:

   ```
   sudo apt install maas-region-api
   ```
3. Configure it:

   ```
   sudo scp ubuntu@$PRIMARY_API:/etc/maas/regiond.conf /etc/maas/
   sudo maas-region local_config_set --database-host $PRIMARY_PG_SERVER
   sudo systemctl restart bind9
   sudo systemctl start maas-regiond
   ```

### [Boost region performance](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-boost-region-performance)

Increase workers in `/etc/maas/regiond.conf`:

```
num_workers: 8
```

Each worker requires 11 PostgreSQL connections. Recommended: one per CPU, max 8.

## [Load balancing and HA for services](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-load-balancing-and-ha-for-services)

### [BMC load balancing](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-bmc-load-balancing)

Adding a second rack controller automatically balances BMC duties.

### [DHCP HA](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-dhcp-ha)

Rack controllers replicate DHCP leases. Example configuration:

```
failover peer "failover-partner" {
     primary;
     address dhcp-primary.example.com;
     peer address dhcp-secondary.example.com;
     split 255;
}
failover peer "failover-partner" {
     secondary;
     address dhcp-secondary.example.com;
     peer address dhcp-primary.example.com;
}
```

Enable via:

* UI: *Subnets* > VLAN > *Reconfigure DHCP*
* CLI:

  ```
  vid=$(maas maas subnets read | jq -r '.[] | select(.cidr == "10.0.0.0/24") | .vlan.vid')
  fabric_id=$(maas maas fabrics read | jq -r '.[] | select(.name == "fabric-1") | .id')
  maas maas vlan update $fabric_id $vid primary_rack=$(hostname) dhcp_on=true
  ```

### [Multiple region endpoints](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-multiple-region-endpoints)

Define endpoints manually:

* Snap: `/var/snap/maas/current/rackd.conf`
* Package: `/etc/maas/rackd.conf`

```
maas_url:
  - http://<ip1>:<port>/MAAS/
  - http://<ip2>:<port>/MAAS/
```

### [Region controller HA](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-region-controller-ha)

* Requires PostgreSQL HA.
* Each region controller may need 40 DB connections under load. Increase `max_connections` in PostgreSQL.

### [Highly-available API with HAProxy](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-highly-available-api-with-haproxy)

Install HAProxy:

```
sudo apt install haproxy
```

Configure `/etc/haproxy/haproxy.cfg`:

```
frontend maas
    bind    *:80
    default_backend maas

backend maas
    balance source
    server localhost localhost:5240 check
    server maas-api-1 $PRIMARY_API_SERVER_IP:5240 check
    server maas-api-2 $SECONDARY_API_SERVER_IP:5240 check
```

Restart HAProxy:

```
sudo systemctl restart haproxy
```

Use port 80 instead of 5240 for API/UI access.

## [Safety nets](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-safety-nets)

* Always back up PostgreSQL before modifying configs.
* Never mix controller versions between instances.
* Plan database connection limits when scaling workers or regions.

## [Next steps](https://canonical-com-2934.demos.haus/maas/docs/how-to-manage-high-availability?format=md#p-9026-next-steps)

* [About controllers](https://canonical.com/maas/docs/about-controllers)

---

[Previous

Build custom images](https://canonical-com-2934.demos.haus/maas/docs/how-to-build-custom-images)
[Next

Use logging](https://canonical-com-2934.demos.haus/maas/docs/how-to-use-logging)

Last updated 1 year, 1 month ago. [Help improve this document in the forum](https://discourse.maas.io/t/how-to-manage-high-availability/5120).