Best Practice for VM Snapshots and Upgrade Procedure in a 3-Node Segura PAM Cluster (No Arbitrator)

PAM Core
, ,
1

Hello Team,

I have a question regarding the recommended upgrade procedure for a 3-node Segura PAM cluster where all three nodes are Segura instances and no Arbitrator is deployed .

Most of the documentation and community discussions describe the upgrade process for a 2-node cluster with an Arbitrator , where the general approach is to remove the Arbitrator from the cluster, upgrade the nodes in reverse order, and then rejoin the Arbitrator.

However, I could not find guidance for a 3-node cluster without an Arbitrator .

I would appreciate clarification on the following points:

  1. What is the recommended procedure for taking VM snapshots before the upgrade?
  • Should VM snapshots be taken for all three nodes?
  • Is there a recommended order for taking the snapshots?
  • Should the nodes be powered off before taking snapshots, or are online snapshots supported?
  1. What are the required pre-upgrade checks?
  • Replication status
  • Cluster health
  • Remote backup verification
  • Master Key verification
  • Any additional recommended checks
  1. What is the recommended node upgrade sequence for a 3-node cluster?
  • Which node should be upgraded first?
  • Should one node be removed from the cluster before upgrading?
  • Is the process similar to the 2-node + Arbitrator architecture, or is it different?
  1. During the upgrade, should the cluster maintain quorum with the remaining two nodes while one node is upgraded, or is there a recommended maintenance procedure?
  2. If an upgrade fails and rollback is required, should all three VMs be restored from snapshots taken at the same point in time, or is it supported to restore only the upgraded node?

If anyone has performed an in-place upgrade of a 3-node Segura PAM cluster or can share the recommended procedure or official documentation, it would be greatly appreciated.

Thank you.

2

1. What is the recommended procedure for taking VM snapshots before the upgrade?

For a 3-node cluster, take snapshots in reverse order, starting from the last member and ending with the primary node.

Turn off the third node, take the snapshot, power it back on, and wait until it has fully resynchronized with the cluster. Then repeat the same process for the second node, and finally for the primary node.

  • Should VM snapshots be taken for all three nodes?
    Yes, it is recommended to take snapshots of all three nodes.

  • Is there a recommended order for taking the snapshots?
    Yes. In clustered environments, always take snapshots in reverse order (Node 3 → Node 2 → Node 1 (Primary)).

  • Should the nodes be powered off before taking snapshots, or are online snapshots supported?
    It is recommended to power off one node at a time, take the snapshot while it is offline, power it back on, and verify that it has fully resynchronized before proceeding to the next node.

2. What are the required pre-upgrade checks?

First, make sure snapshots have been taken. Then verify the following:

  • Replication status
  • Cluster health
  • Remote backup verification
  • Master Key verification

3. What is the recommended node upgrade sequence for a 3-node cluster?

  • Which node should be upgraded first?
    Upgrade the primary node first, followed by the secondary node, and finally the third node.

  • Should one node be removed from the cluster before upgrading?
    No.

  • Is the process similar to the 2-node + Arbitrator architecture, or is it different?
    The upgrade process is similar to a standard 2-node cluster, except that there is no arbitrator involved.

4. During the upgrade, should the cluster maintain quorum with the remaining two nodes while one node is upgraded, or is there a recommended maintenance procedure?

Yes. In a 3-node cluster, when one node is being upgraded, the remaining two nodes still maintain quorum (2 out of 3 nodes, or approximately 66.7%). This allows the cluster to continue operating normally, making the upgrade process generally smoother than in a 2-node deployment.

5. If an upgrade fails and rollback is required, should all three VMs be restored from snapshots taken at the same point in time, or is it supported to restore only the upgraded node?

This question is best answered by Segura Support, as the official recommendation may depend on the specific scenario.

Based on my understanding, if the upgrade fails on a specific node (for example, the second node), you should be able to restore that node from its snapshot, allow it to rejoin the cluster, and then retry the upgrade before proceeding to the next node. However, I recommend waiting for official confirmation from Segura Support before following this approach in a production environment.