Quick Overview
Part 10 of the grommunio 2026 series turns the tested active/standby homeserver into a practical step-by-step guide. The goal is one logical homeserver, hs1.cluster.test, that can run on two data nodes and is protected by witness quorum, STONITH, DRBD, /grodata, Pacemaker, Corosync and a service VIP.
The final lab result remains PASS WITH CONDITIONS: the stack was built and tested against planned moves, hard node failures, partitions and witness loss. The limits stay visible as well: production fencing must be real, DRBD is not backup, existing Web sessions were not proven to continue transparently, and load-balancer integration to the hs1 pair remains a known limitation because of separated QEMU L2 networks.
| Area | Result |
|---|---|
| Homeserver HA gate | PASS WITH CONDITIONS |
| MariaDB-only HA | FAIL for full Mail/Web/SMTP continuity |
| Web | PASS WITH CONDITIONS: new login after failover proven |
| IMAPS | PASS |
| SMTP | PASS |
| Load balancer to hs1 | KNOWN LIMITATION |
| Backup/restore marker | PASS WITH CONDITIONS |
Homeserver HA gate
- Result
- PASS WITH CONDITIONS
MariaDB-only HA
- Result
- FAIL for full Mail/Web/SMTP continuity
Web
- Result
- PASS WITH CONDITIONS: new login after failover proven
IMAPS
- Result
- PASS
SMTP
- Result
- PASS
Load balancer to hs1
- Result
- KNOWN LIMITATION
Backup/restore marker
- Result
- PASS WITH CONDITIONS
Tested Environment
The environment was an isolated QEMU/KVM lab on grommunio 2026.06.1. No production ForgeOne systems were changed. These are lab values, not vendor sizing recommendations.
| Role | Hostname | IP | Purpose |
|---|---|---|---|
| Data node A | hs1a.cluster.test | 10.210.10.41 | DRBD peer and possible active service node |
| Data node B | hs1b.cluster.test | 10.210.10.42 | DRBD peer and possible active service node |
| Witness | hs1w.cluster.test | 10.210.10.43 | Quorum-only, no mailbox data |
| Logical homeserver | hs1.cluster.test | 10.210.10.40 | Service VIP and client target |
Data node A
- Hostname
- hs1a.cluster.test
- IP
- 10.210.10.41
- Purpose
- DRBD peer and possible active service node
Data node B
- Hostname
- hs1b.cluster.test
- IP
- 10.210.10.42
- Purpose
- DRBD peer and possible active service node
Witness
- Hostname
- hs1w.cluster.test
- IP
- 10.210.10.43
- Purpose
- Quorum-only, no mailbox data
Logical homeserver
- Hostname
- hs1.cluster.test
- IP
- 10.210.10.40
- Purpose
- Service VIP and client target
grommunio: 2026.06.1Storage: /dev/vdb -> /dev/drbd0 -> /grodataDRBD resource: hs1-grodataLogical endpoint: hs1.cluster.test / 10.210.10.40
Target Architecture
The important distinction is physical node versus logical homeserver. hs1a and hs1b only carry the service stack. Clients use hs1.cluster.test with VIP 10.210.10.40.
quorum onlyhs1w.cluster.test10.210.10.43|hs1a.cluster.test -----+----- hs1b.cluster.test10.210.10.41 10.210.10.42DRBD Secondary/Primary DRBD Primary/Secondary|hs1.cluster.testVIP 10.210.10.40
Resource order is the core of the guide: quorum and fencing first, then DRBD promotion, then /grodata, bind mounts, VIP and the grommunio service stack.
STONITH/quorum-> DRBD promote-> mount /grodata-> bind persistent service paths-> service VIP-> mariadb, redis, php-fpm, gromox, antispam, admin-api, nginx
Prerequisites
Start only with two clean grommunio data nodes on the same software level, a separate witness, static IPs, working DNS or consistent /etc/hosts entries, NTP, root access, a dedicated DRBD data disk and working fencing.
Create a snapshot or backup before changing the system. DRBD mirrors state and mistakes; it is not backup and does not protect against deletion, corruption or incorrect administration.
Step 1: Check the Baseline
Before installing cluster software, both data nodes must look the same. This check decides whether you should continue at all.
hostname -fcat /etc/os-releaserpm -qa | sort | grep -E "grommunio|gromox|mariadb|redis|nginx"systemctl --failedip -br addrlsblkchronyc tracking || timedatectl
Expected result: both data nodes report grommunio 2026.06.1, no unexplained failed units, the same package baseline and the additional /dev/vdb disk for the DRBD resource.
Step 2: Prepare the Cluster Network
All cluster names must resolve consistently. The tested hs1 lab uses 10.210.10.41 for hs1a, 10.210.10.42 for hs1b, 10.210.10.43 for hs1w and 10.210.10.40 as service VIP. `/etc/hosts.d/hs1-cluster.example` was not evidenced as the tested default resolver path, so it is no longer shown as an executable standard instruction.
Classification: ILLUSTRATIVE. Use DNS or the resolver mechanism that is actually active on your distribution; afterwards `getent hosts` and cluster communication must resolve the names consistently.
getent hosts hs1a.cluster.test hs1b.cluster.test hs1w.cluster.test hs1.cluster.testping -c3 hs1b.cluster.testping -c3 hs1w.cluster.test
Verification: `20260913-hs1-corosync-pacemaker-baseline.txt` and the final Corosync configuration with node IPs and node IDs.
Step 3: Install and Configure Corosync and Pacemaker
In the final lab, hs1w is not qdevice/qnetd. It is a regular third Corosync/Pacemaker voting node. It must not run productive resources; its role is quorum and control-plane participation.
Classification: tested for package installation and service start; tested for the Corosync configuration. The evidence proves the final configuration, not a `crm cluster init/join` history, so this article does not claim such a history.
zypper in corosync pacemaker crmsh resource-agentsinstall -m 0400 -o root -g root /path/to/lab-authkey /etc/corosync/authkeyinstall -m 0644 -o root -g root corosync.conf /etc/corosync/corosync.confsystemctl enable --now corosync pacemakercrm statuscorosync-quorumtool -s
totem {version: 2cluster_name: grommunio-hs1-hatransport: knetcrypto_cipher: aes256crypto_hash: sha256}nodelist {node {ring0_addr: 10.210.10.41name: hs1anodeid: 1}node {ring0_addr: 10.210.10.42name: hs1bnodeid: 2}node {ring0_addr: 10.210.10.43name: hs1wnodeid: 3}}quorum {provider: corosync_votequorumtwo_node: 0}logging {to_logfile: yeslogfile: /var/log/cluster/corosync.logto_syslog: yes}
Verification: `cluster-control/hs1-corosync.conf` and `20260913-hs1-corosync-pacemaker-baseline.txt` with expected votes 3, total votes 3 and quorum 2.
Step 4: Classify the Witness Correctly
In this article, Witness describes the role of hs1w in the lab. Technically, hs1w is a normal third Corosync/Pacemaker voting node without DRBD, without `/grodata`, without VIP and without the grommunio service group. No qnetd/qdevice design was tested.
Classification: tested. The final CIB explicitly bans DRBD, filesystem, bind mounts, VIP and service group from hs1w.
Step 5: Configure STONITH
This STONITH setup is lab-only. The tested agent controls QEMU through a host-side controller. In production, replace this with real hardware, hypervisor or cloud fencing.
Classification: tested, but LAB ONLY.
crm configure <<'EOF'primitive fence-hs1a stonith:fence_qemu_part10_hs1 \params plug=hs1a base_url="http://10.0.2.2:18092" pcmk_host_list=hs1a pcmk_reboot_action=off \op monitor interval=60s timeout=20sprimitive fence-hs1b stonith:fence_qemu_part10_hs1 \params plug=hs1b base_url="http://10.0.2.2:18092" pcmk_host_list=hs1b pcmk_reboot_action=off \op monitor interval=60s timeout=20sEOF
Verification: `20260913-final-quant-pacemaker-config.txt`, `20260913-hs1-stonith-resources.txt` and `cluster-control/fence_qemu_part10_hs1`.
Step 6: Configure DRBD
DRBD uses `/dev/vdb` on hs1a and hs1b and exposes `/dev/drbd0` or `/dev/drbd/by-res/hs1-grodata/0`. The published secret is intentionally redacted; the real value must never appear in articles, screenshots or the repository.
Classification: tested because the secret is redacted; the remaining values are taken from the tested file.
zypper in drbd-utils kernel-default-extra drbd-kmp-defaultmodprobe drbdinstall -m 0600 -o root -g root hs1-grodata.res /etc/drbd.d/hs1-grodata.resdrbdadm create-md hs1-grodatadrbdadm up hs1-grodatadrbdadm status hs1-grodata
resource hs1-grodata {protocol C;net {cram-hmac-alg sha256;shared-secret "<redacted-lab-secret>";}disk {on-io-error detach;}on hs1a {device /dev/drbd0;disk /dev/vdb;address 10.210.10.41:7790;meta-disk internal;}on hs1b {device /dev/drbd0;disk /dev/vdb;address 10.210.10.42:7790;meta-disk internal;}}
Verification: `cluster-control/hs1-grodata.res`, `20260913-hs1-drbd-module-install.txt`, `20260913-hs1-drbd-connected.txt` and `20260913-hs1-drbd-sync-complete.txt`.
Step 8: Migrate Data
The migration was verified for seven paths. During copying, uncontrolled writes to the old paths must be stopped. The lab evidence proves the resulting target state; a complete production-safe stop sequence including Postfix was not evidenced as exact shell history.
Classification: tested for the verified target state; the stop sequence is marked RECOMMENDED and does not replace a maintenance plan.
systemctl stop nginx grommunio-admin-api grommunio-antispam php-fpm 'redis@grommunio' mariadb gromox-http gromox-midb gromox-zcore gromox-event gromox-timer gromox-imap gromox-pop3 gromox-delivery-queue gromox-deliverysystemctl stop postfixrsync -aHAX /var/lib/mysql/ /grodata/mysql/rsync -aHAX /var/lib/redis/ /grodata/redis/rsync -aHAX /var/lib/gromox/ /grodata/gromox/rsync -aHAX /var/lib/grommunio-web/ /grodata/grommunio-web/rsync -aHAX /var/lib/grommunio-antispam/ /grodata/grommunio-antispam/rsync -aHAX /var/lib/grommunio-dav/ /grodata/grommunio-dav/rsync -aHAX /var/lib/grommunio-admin-api/ /grodata/grommunio-admin-api/
| Path | Owner | Mode |
|---|---|---|
| /grodata/mysql | mysql:mysql | 700 |
| /grodata/redis | redis:redis | 750 |
| /grodata/gromox | gromox:gromox | 770 |
| /grodata/grommunio-web | groweb:groweb | 770 |
| /grodata/grommunio-antispam | groas:grommunio | 750 |
| /grodata/grommunio-dav | root:grodav | 770 |
| /grodata/grommunio-admin-api | grommunio:grommunio | 750 |
/grodata/mysql
- Owner
- mysql:mysql
- Mode
- 700
/grodata/redis
- Owner
- redis:redis
- Mode
- 750
/grodata/gromox
- Owner
- gromox:gromox
- Mode
- 770
/grodata/grommunio-web
- Owner
- groweb:groweb
- Mode
- 770
/grodata/grommunio-antispam
- Owner
- groas:grommunio
- Mode
- 750
/grodata/grommunio-dav
- Owner
- root:grodav
- Mode
- 770
/grodata/grommunio-admin-api
- Owner
- grommunio:grommunio
- Mode
- 750
Verification: `20260913-hs1-grodata-copy-metrics.txt`, `20260913-hs1-local-dir-prep-port-36122.txt` and `20260913-hs1-local-dir-prep-port-36222.txt`. No broad `chown -R` correction is retained as article guidance.
Step 9: Configure Bind Mounts
All seven bind mounts are part of the final resource group. Showing only mysql, redis and gromox would be misleading here.
Classification: tested from the final CIB.
crm configure <<'EOF'primitive p-bind-mysql Filesystem \params device="/grodata/mysql" directory="/var/lib/mysql" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-redis Filesystem \params device="/grodata/redis" directory="/var/lib/redis" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-gromox Filesystem \params device="/grodata/gromox" directory="/var/lib/gromox" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-grommunio-web Filesystem \params device="/grodata/grommunio-web" directory="/var/lib/grommunio-web" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-grommunio-antispam Filesystem \params device="/grodata/grommunio-antispam" directory="/var/lib/grommunio-antispam" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-grommunio-dav Filesystem \params device="/grodata/grommunio-dav" directory="/var/lib/grommunio-dav" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20primitive p-bind-grommunio-admin-api Filesystem \params device="/grodata/grommunio-admin-api" directory="/var/lib/grommunio-admin-api" fstype=none options=bind run_fsck=no \op start timeout=40s interval=0s \op stop timeout=60s interval=0s \op monitor interval=20s timeout=40s \op_params OCF_CHECK_LEVEL=20group g-grodata-binds p-bind-mysql p-bind-redis p-bind-gromox p-bind-grommunio-web p-bind-grommunio-antispam p-bind-grommunio-dav p-bind-grommunio-admin-apiEOF
Step 10: Configure the Service VIP and grommunio Service Group
The service group starts on the active data node after `/grodata` and the bind mounts. In the tested lab, Postfix is explicitly not part of `g-grommunio-svc`; it remains systemd-managed. This also explains the later queue observation: Postfix can accept submission while local delivery to Gromox on `::1:24` is not ready yet.
Classification: tested from the final CIB.
crm configure <<'EOF'primitive p-vip-hs1 IPaddr2 \params ip=10.210.10.40 cidr_netmask=24 nic=enp0s3 \op monitor interval=10s timeout=20sprimitive p-mariadb systemd:mariadb op start timeout=120s interval=0s op stop timeout=120s interval=0s op monitor interval=20s timeout=30sprimitive p-redis-grommunio systemd:redis@grommunio op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-php-fpm systemd:php-fpm op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-http systemd:gromox-http op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30s meta target-role=Startedprimitive p-gromox-midb systemd:gromox-midb op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30s meta target-role=Startedprimitive p-gromox-zcore systemd:gromox-zcore op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-event systemd:gromox-event op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-timer systemd:gromox-timer op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-imap systemd:gromox-imap op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-pop3 systemd:gromox-pop3 op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-delivery-queue systemd:gromox-delivery-queue op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sprimitive p-gromox-delivery systemd:gromox-delivery op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30s meta target-role=Startedprimitive p-grommunio-antispam systemd:grommunio-antispam op start timeout=90s interval=0s op stop timeout=90s interval=0s op monitor interval=30s timeout=30sprimitive p-grommunio-admin-api systemd:grommunio-admin-api op start timeout=90s interval=0s op stop timeout=90s interval=0s op monitor interval=30s timeout=30sprimitive p-nginx systemd:nginx op start timeout=60s interval=0s op stop timeout=60s interval=0s op monitor interval=20s timeout=30sgroup g-grommunio-svc p-mariadb p-redis-grommunio p-php-fpm p-gromox-http p-gromox-midb p-gromox-zcore p-gromox-event p-gromox-timer p-gromox-imap p-gromox-pop3 p-gromox-delivery-queue p-gromox-delivery p-grommunio-antispam p-grommunio-admin-api p-nginx \meta target-role=StartedEOF
Step 11: Set Ordering, Colocation and Witness Bans
The final ordering is group-based: DRBD promotion before `/grodata`, `/grodata` before all seven bind mounts, bind mounts before VIP and VIP before the service group. hs1w is banned from data and service resources.
Classification: tested from the final CIB.
crm configure <<'EOF'colocation col-fs-with-drbd inf: p-fs-grodata ms-drbd-hs1:Promotedcolocation col-binds-with-fs inf: g-grodata-binds p-fs-grodatacolocation col-vip-with-fs inf: p-vip-hs1 p-fs-grodatacolocation col-svc-with-binds inf: g-grommunio-svc g-grodata-bindscolocation col-svc-with-vip inf: g-grommunio-svc p-vip-hs1location loc-fence-hs1a-not-on-hs1a fence-hs1a -inf: hs1alocation loc-fence-hs1b-not-on-hs1b fence-hs1b -inf: hs1blocation loc-no-drbd-witness ms-drbd-hs1 -inf: hs1wlocation loc-no-fs-witness p-fs-grodata -inf: hs1wlocation loc-no-binds-witness g-grodata-binds -inf: hs1wlocation loc-no-vip-witness p-vip-hs1 -inf: hs1wlocation loc-no-svc-witness g-grommunio-svc -inf: hs1worder ord-drbd-before-fs Mandatory: ms-drbd-hs1:promote p-fs-grodata:startorder ord-fs-before-binds Mandatory: p-fs-grodata:start g-grodata-binds:startorder ord-binds-before-vip Mandatory: g-grodata-binds:start p-vip-hs1:startorder ord-vip-before-svc Mandatory: p-vip-hs1:start g-grommunio-svc:startEOF
Step 12: Configure the Logical Homeserver Identity
The active stack presents the logical homeserver `hs1.cluster.test`. Both data nodes use the same `host_id`, while only the Pacemaker-active node runs the services. EXMDB allows the VIP and both physical data-node addresses.
Classification: tested for values and validation, tested for the redacted execution of configuration changes.
# /etc/gromox/gromox.cfghost_id=hs1.cluster.test# /etc/gromox/http.cfghost_id=hs1.cluster.test# /etc/gromox/exmdb_provider.cfglisten_ip=::exmdb_hosts_allow=::1 127.0.0.1 10.210.10.40 10.210.10.41 10.210.10.42 ::ffff:10.210.10.40 ::ffff:10.210.10.41 ::ffff:10.210.10.42
grommunio-admin server show hs1.cluster.testgrommunio-admin domain show ha1.example.testgrommunio-admin user show ha-user@ha1.example.testcrm statusfindmnt /grodata /var/lib/mysql /var/lib/gromox
Verification: `20260913-hs1-gromox-logical-endpoint-config.txt`, `20260913-hs1-exmdb-acl-expanded.txt`, `20260913-hs1-logical-server-domain-user.txt` and `20260913-hs1-full-stack-baseline-active.txt`.
Step 13: First Start
After configuration, Pacemaker starts the complete chain. Expected state: STONITH on a suitable peer or hs1w, one promoted DRBD clone, `/grodata`, seven bind mounts, VIP and `g-grommunio-svc` on the same data node.
Classification: tested for validation commands.
crm statuscrm configure showfindmnt /grodata /var/lib/mysql /var/lib/redis /var/lib/gromox /var/lib/grommunio-web /var/lib/grommunio-antispam /var/lib/grommunio-dav /var/lib/grommunio-admin-apiip addr show enp0s3 | grep 10.210.10.40
Step 14: Baseline Mailbox Test
The baseline counts only when Web, IMAPS and SMTP work through the service VIP and the queue is empty after delivery. Test credentials and passwords remain internal.
Classification: tested for the tested logic, credentials redacted.
curl -kI https://10.210.10.40/web/python3 imaps_probe_redacted.py --host 10.210.10.40 --user ha-user@ha1.example.testpython3 smtp_submission_probe_redacted.py --host 10.210.10.40 --port 587 --user ha-user@ha1.example.testpostqueue -p
Verification: `20260913-hs1-vip-mail-final-baseline-redacted.txt`, `20260913-hs1-full-stack-baseline-active.txt`, `final-quant-baseline-protocol-probe.txt` and `final-quant-baseline-web-results.txt`.
Test HA
Every failover test used the same structure: starting state, failure injection, expected behavior, observed behavior, service test, data integrity and result.
Planned Move
crm resource move g-grommunio-svc hs1bcrm statusdrbdadm status hs1-grodatafindmnt /grodataip addr show | grep 10.210.10.40crm resource clear g-grommunio-svc
Result: PASS. The planned move worked. In the quantitative run, five planned moves showed mail-delivery recovery between 6.188 and 7.410 seconds, median 7.076 seconds.
Hard Node Failure
In the lab, the active node was powered off through QEMU control. In production, the equivalent test is a controlled hypervisor, hardware or cloud fencing test.
# Lab example, do not copy as a production command:# power off the active QEMU node hardcrm statusdrbdadm status hs1-grodatafindmnt /grodataip addr show | grep 10.210.10.40mailq
Result: PASS WITH CONDITIONS. The stack came back on the remaining node. One valid hard-failure service sample was 25.798 seconds, mail delivery was 33.206 seconds. This is one lab sample, not a statistical RTO.
Network Partition and Split-Brain Protection
The active-partition tests checked whether the cluster prevents dual ownership. The goal is not availability at any price, but no unsafe promotion.
| Test | Expectation | Result |
|---|---|---|
| Active partition | one side takes over in a controlled way, no dual-primary state | PASS |
| Fencing unavailable | cluster blocks safely instead of promoting unsafely | PASS WITH CONDITIONS / SAFE BLOCKED |
| Old primary returns | no automatic overwrite of the active side | PASS |
Active partition
- Expectation
- one side takes over in a controlled way, no dual-primary state
- Result
- PASS
Fencing unavailable
- Expectation
- cluster blocks safely instead of promoting unsafely
- Result
- PASS WITH CONDITIONS / SAFE BLOCKED
Old primary returns
- Expectation
- no automatic overwrite of the active side
- Result
- PASS
Witness Loss plus Active Node Loss
If the witness is missing and the active data node then fails, the remaining node must not take over blindly. In the lab this correctly ended in SAFE_UNAVAILABLE.
crm statuscorosync-quorumtool -s# Expected without quorum: no unsafe promotion, manual recovery by runbook
Postfix and Delivery Queue during Hard Failover
The most important finding from the quantitative hard-failure tests was a readiness window: Postfix was not part of the Pacemaker service group and could accept SMTP while gromox-delivery-queue on ::1:24 was not ready yet.
Postfix submission accepted mail-> local delivery to ::1:24 was not ready yet-> message stayed deferred in Postfix-> message was later delivered after delivery-queue readiness / queue retry
This is not observed message loss; it is delayed local delivery. In the verified benchmark data, 0/10 SMTP-accepted messages were lost. This is explicitly not a guaranteed RPO=0 claim.
Assess Web, IMAP and SMTP
Web was functionally proven as a new login through the hs1 VIP before and after failover. Existing Web sessions were not classified as transparently continued; a historical test showed the login form again after reload.
| Protocol | Acceptance |
|---|---|
| Web new login | PASS WITH CONDITIONS |
| Existing Web session | NOT PROVEN |
| IMAPS | PASS |
| SMTP AUTH/submission | PASS |
| Mail delivery during hard failover | DELAYED possible, no loss observed in verified samples |
Web new login
- Acceptance
- PASS WITH CONDITIONS
Existing Web session
- Acceptance
- NOT PROVEN
IMAPS
- Acceptance
- PASS
SMTP AUTH/submission
- Acceptance
- PASS
Mail delivery during hard failover
- Acceptance
- DELAYED possible, no loss observed in verified samples
Classify the Load Balancer
The earlier HAProxy/VRRP load-balancer layer was built separately. For the final hs1 pair, end-to-end integration remained a lab limitation because the LB VMs and the hs1 pair were on separate QEMU L2 segments.
LB lab LAN: 10.210.10.20/21/22 on QEMU socket LAN 230.10.20.26:1210hs1 HA LAN: 10.210.10.40/41/42/43 on QEMU socket LAN 230.10.20.27:1211Result: same written IP range, but separate lab L2 segments.
Backup and Restore
DRBD is not backup. The lab checked a narrow restore marker, but not a complete backup strategy. Production needs independent backups, restore tests, retention, offsite copies and documented recovery objectives.
Why MariaDB-only HA Is Not Enough
The historical MariaDB-only approach remains important, but it belongs in lessons learned, not in the setup path. The lab showed that database failover alone is not enough for end-user continuity of Web, IMAP and SMTP. grommunio needs the consistent homeserver stack with data, services, VIP, quorum and fencing.
DB moved: partially successfulAdmin/Postfix maps: partially usableWeb/IMAP/SMTP continuity: not sufficientMARIADB_HA_GATE = FAIL
Measured Failover Times
| Scenario | Measurement | Classification |
|---|---|---|
| Planned homeserver move | 5 samples: min 6.188 s, median 7.076 s, mean 6.929 s, max 7.410 s | publishable lab observation for mail-delivery recovery |
| Hard-failure service RTO | 1 valid sample: 25.798 s | single sample only, not a statistic |
| Hard-failure mail delivery | 1 valid sample: 33.206 s | delayed queue delivery, no SLA statement |
| SMTP-accepted messages | 0/10 verified accepted messages lost | observation, no guaranteed RPO=0 claim |
| Web RTO | not measured reliably | do not publish a number |
Planned homeserver move
- Measurement
- 5 samples: min 6.188 s, median 7.076 s, mean 6.929 s, max 7.410 s
- Classification
- publishable lab observation for mail-delivery recovery
Hard-failure service RTO
- Measurement
- 1 valid sample: 25.798 s
- Classification
- single sample only, not a statistic
Hard-failure mail delivery
- Measurement
- 1 valid sample: 33.206 s
- Classification
- delayed queue delivery, no SLA statement
SMTP-accepted messages
- Measurement
- 0/10 verified accepted messages lost
- Classification
- observation, no guaranteed RPO=0 claim
Web RTO
- Measurement
- not measured reliably
- Classification
- do not publish a number
Technical Acceptance
| Area | Check | Result |
|---|---|---|
| Cluster bootstrap | Manual Corosync configuration evidenced; init/join history not evidenced | PASS WITH CONDITIONS |
| Witness | hs1w is a regular third voting node, not qdevice/qnetd | PASS |
| STONITH | Lab fencing fully documented, production fencing required | PASS_WITH_CONDITIONS |
| DRBD | Resource, clone and sync evidenced; secret redacted | PASS |
| /grodata | Filesystem resource and mounts present | PASS |
| Bind mounts | all seven bind mounts in group, ordering and colocation | PASS |
| Service group | final units and order taken from CIB | PASS |
| Postfix | systemd-managed outside Pacemaker documented; port 24 explained | PASS |
| Logical homeserver | `hs1.cluster.test`, `host_id` and EXMDB evidenced | PASS |
| Data migration | target owners and modes verified; no broad chown commands | PASS |
| Service stop sequence | production stop order is recommendation, not lab history | FAIL |
| Clean-room reproduction | no fresh ha2a/ha2b/ha2w built solely from the article | FAIL |
Cluster bootstrap
- Check
- Manual Corosync configuration evidenced; init/join history not evidenced
- Result
- PASS WITH CONDITIONS
Witness
- Check
- hs1w is a regular third voting node, not qdevice/qnetd
- Result
- PASS
STONITH
- Check
- Lab fencing fully documented, production fencing required
- Result
- PASS_WITH_CONDITIONS
DRBD
- Check
- Resource, clone and sync evidenced; secret redacted
- Result
- PASS
/grodata
- Check
- Filesystem resource and mounts present
- Result
- PASS
Bind mounts
- Check
- all seven bind mounts in group, ordering and colocation
- Result
- PASS
Service group
- Check
- final units and order taken from CIB
- Result
- PASS
Postfix
- Check
- systemd-managed outside Pacemaker documented; port 24 explained
- Result
- PASS
Logical homeserver
- Check
- `hs1.cluster.test`, `host_id` and EXMDB evidenced
- Result
- PASS
Data migration
- Check
- target owners and modes verified; no broad chown commands
- Result
- PASS
Service stop sequence
- Check
- production stop order is recommendation, not lab history
- Result
- FAIL
Clean-room reproduction
- Check
- no fresh ha2a/ha2b/ha2w built solely from the article
- Result
- FAIL
Reproducibility audit: NOT TESTED = 0, TECHNICAL_EXPANSION_REQUIRED = 2, ILLUSTRATIVE_COMMAND_BLOCKS = 1. Every executable snippet without complete evidence was removed, marked as redacted/normalized, or visibly classified as illustrative.
Troubleshooting and Lessons Learned
DRBD module missing: drbd-utils alone was not enough in the lab. kernel-default-extra and drbd-kmp-default were required.
Wrong homeserver identity: if host_id and the logical homeserver are inconsistent, EXMDB or access errors can appear.
Fencing unavailable: the desired behavior is blocking, not risky continuation. Recovery then needs manual cleanup by runbook.
Postfix reachable too early: SMTP reachability is not automatically mail-delivery readiness. Monitoring should treat queue state, port 24 and gromox-delivery-queue separately.
Production Notes
The lab deliberately keeps critical warnings visible: real fencing, independent backups, monitoring, documented recovery procedures, repeated hard-failure measurements, load-balancer end-to-end tests and a session/reconnect concept belong on the pre-go-live checklist.
Conclusion
The tested path is not MariaDB-only and not simply multi-server. The lab became viable as a complete logical homeserver: DRBD, /grodata, bind mounts, VIP, service stack, quorum and STONITH in a controlled order. That order is what turns the lab report into a reproducible HA guide.
grommunio 2026 Step by Step
The articles build on each other chronologically. Start with the appliance and then add mail security, central login, video meetings, chat, files, archiving, device management, multitenancy and high availability.
Part 1: Install grommunio 2026
Part 2: grommunio Antispam with Rspamd
Part 3: grommunio Auth with Keycloak and MFA
Part 6: grommunio Files and Office
Part 8: grommunio Mobile Device Management
Part 9: Classify grommunio multitenancy
Part 10: grommunio Cluster and High Availability (this article)
Review the cluster design?
ForgeOne supports grommunio architecture, HA labs, migration planning, operations, monitoring and documented failover runbooks. Discuss a grommunio project






