The question this post answers
How do you fail over VCF Operations to another site without L2 stretch or NSX Federation, and keep the same IPs?
Many VCF customers run two sites with no stretched Layer 2 network, no NSX Federation and no routed IP mobility. Yet VCF Operations must come back at the recovery site with the same IP addresses and DNS names, so that users, adapters and integrations keep working after a disaster.
This post shows a proven design that does exactly that on VCF 9.1.1, using VCF Protection and Recovery (P&R) 9.1.1 with vSphere Replication. It is based on a real implementation runbook and aligned with Broadcom's VCF 9.1 documentation.
The answer in short
Replicate the data, not the network. P&R replicates the VCF Operations nodes from Site A to Site B every 15 minutes over a separate, routed replication network.
Pre-build an identical VLAN at Site B and keep it dormant. It has the same VLAN ID, subnet and gateway IP as Site A, but its gateway stays shut down and unadvertised.
Move the route, not the IPs. During failover, the network team withdraws the subnet at Site A and activates it at Site B. The recovered nodes power on with their original IPs; nothing is re-addressed and DNS does not change.
Gate every power-on behind the network. The recovery plan's first step is a network cutover prompt, so VCF Operations never runs with the subnet active at both sites.
What this post covers
| Part | What you will learn |
|---|---|
| 1. Recovery model | Which VCF 9.1 management components fail over to the other site, and which are restored in place |
| 2. Reference architecture | How the two sites, the P&R appliances, the replication network and the dormant VLAN fit together |
| 3. Planning | Design decisions, RPO/RTO, prerequisites, IP budget, firewall flows and subnet states |
| 4. Implementation | The six build stages, from replication networking to the recovery plan and its prompts |
| 5. Recovery workflows | Network cutover, test recovery, disaster failover, reprotect and planned failback |
| 6. Day-2 operations | Keeping the DR design valid: drift, new hosts, upgrades, testing and monitoring |
Environment at a glance: two VCF 9.1.1 instances (Site A protected, Site B recovery), separate management vCenters and SSO domains, vSAN ESA principal storage, a 15-minute RPO and a 4-hour RTO target.
Who it is for: VCF architects, VMware engineers, and the network and operations teams who build and run DR for VCF.
The VCF 9.1 recovery model: fleet level vs instance level
In VCF 9.1, fleet-level components can be recovered at another VCF Instance when a site is lost. Instance-level components are never failed over; each VCF Instance is restored in place from its own backups. Every DR design starts by placing each management component on one side of that line.
Broadcom covers the two paths in separate guides: VCF Component Backup and Restore, VCF Instance Backup and Restore and VCF Fleet Disaster Recovery.
Only the left column ever crosses to the other site. Everything on the right is rebuilt where it stood, in the order shown underneath; the table below adds the backup method for each component.
| Component | Level | Backup method | Component failure | Site or instance failure |
|---|---|---|---|---|
| VCF Operations (analytics cluster) | Fleet | Image-based, all nodes together, to a third-party repository | Restore the cluster from a consistent image backup | Fail over with P&R (vSphere Replication) to the other instance |
| VCF Operations for Networks | Fleet | Image-based, third-party repository | Image restore | Fail over with its own P&R protection group and recovery plan |
| VCF Management Services (fleet and SDDC lifecycle, depot, identity, log services) | Fleet | File-based, SFTP | Restore or redeploy the affected service | Redeploy the services runtime at the recovery instance and restore from SFTP |
| Log management | Fleet | Included in the Management Services backup | Restore with Management Services | Recovered with Management Services |
| VCF Automation | Fleet | File-based, SFTP | Redeploy and restore | Deploy a new instance in the recovery VCF Instance and restore the backup |
| VCF License Server | Fleet | None | Redeploy and reconfigure licensing | Redeploy and reconfigure licensing |
| vCenter (incl. vSphere Supervisor) | Instance | File-based, SFTP (image-based optional) | File-based restore | Restored in place with the VCF Instance |
| NSX Manager cluster and NSX Edges | Instance | File-based, SFTP (Edge config inside the NSX Manager backup) | Redeploy the failed node, or restore the cluster | Restored in place with the VCF Instance |
| SDDC Manager | Instance | File-based, SFTP; credentials database through a separate API backup | File-based restore, then inventory reconciliation | Restored in place; back it up after all other components |
| ESX hosts | Instance | None; keep as-built documentation | Reinstall the host and re-add it to the cluster | Rebuilt during VCF Instance restore |
| vSAN ESA (management) | Instance | No backup; data protected by the storage policy (FTT) | vSAN rebuilds from surviving copies | Data returns through VM and file-based restores |
| P&R 9.1.1 appliance | Instance | Not listed as a backup item | Redeploy, re-register and re-pair | Redeploy at the restored instance, then reprotect |
Two placement rules follow from this table. Keep the SFTP server and the image-backup repository outside the site they protect, or replicate them, so backups survive a site loss. And treat backup and replication as complementary: replication brings VCF Operations back fast at the other site, while backups bring it back from corruption that replication would faithfully copy.
When a whole instance must be rebuilt, Broadcom's restore order is: external dependencies (DNS, NTP, identity, certificates, SFTP reachability), management hosts and vSAN, vCenter, NSX Manager, SDDC Manager, VCF Operations, VCF Management Services, VCF Automation and VCF Operations for Networks, P&R, workload domains, then vSphere Supervisor.
What VCF Protection and Recovery 9.1 brings
VCF Protection and Recovery 9.1 is a single appliance that combines the orchestration of Site Recovery Manager with vSphere Replication. Broadcom states that standalone SRM and vSphere Replication appliances are deprecated in 9.1, and that 9.0.2 standalone appliances cannot be upgraded in place (KB 444864).
The appliance contains the protection and recovery service, vSphere Replication, and vSAN snapshots and replication. One appliance can be deployed per vCenter. A scale-out appliance (protection and recovery service only) is needed only beyond 5,000 protected VMs or for fan-in and fan-out topologies.
For the fleet components that are redeployed rather than replicated, Broadcom ships supporting scripts for the VCF 9.1 Fleet DR failover and failback tasks (KB 443506). They update fleet component registration, the Identity Broker instance association and component VIPs, and should only be used from the documentation steps that reference them.
Reference architecture: replicate the data, move the subnet
VCF Operations data replicates continuously over a routed replication network. The VCF Operations subnet moves between sites only through a gated network cutover, because the identical Site B VLAN stays dormant until then.
Read the gap between the sites row by row: the P&R appliances are paired, replicas flow one way, the replication VLANs are routed all the time, and the VCF Operations VLAN is active at only one site.
Two networks behave very differently in this design. The replication VLANs are routed between sites all the time. The VCF Operations VLAN exists at both sites but is routed from only one of them at any moment.
| Component | Site A (protected) | Site B (recovery) | Rule |
|---|---|---|---|
| Management vCenter, NSX, SDDC Manager | Existing | Existing | Never replicated |
| P&R 9.1.1 appliance (with vSphere Replication) | 1 | 1 | Paired |
| VCF Operations nodes | Running on the Ops port group | Replicas on vSAN, placeholder VMs | Same IPs after failover |
| VCF Operations VLAN and gateway | Active and advertised | Identical, dormant | Exactly one site active |
| Replication VLAN and VMkernel per host | VLAN 1650, routed | VLAN 2650, routed | Always on |
| Test network | None | Isolated test port group | Tests only |
| Cloud Proxies | Local, not replicated | Local, not replicated | Handled by recovery plan prompts |
Scope of the worked example. In scope: the VCF Operations analytics cluster (primary, replica and data nodes), Cloud Proxies handled as operator prompts, one P&R appliance per site, the network cutover of the VCF Operations VLAN, and the full test, failover, reprotect and failback cycle. Out of scope: VCF Operations for Networks, VCF Automation and VCF Management Services; vCenter, NSX Managers and SDDC Manager, which each site keeps and never replicates; array-based replication; and L2 extension, NSX Federation or re-addressing.
Key risk: duplicate subnets. If the Site A and Site B VCF Operations subnets are ever active and routed at the same time, traffic is black-holed or split. Every failover, test and failback step in this design includes an explicit network gate to prevent it.
Planning the DR design
Planning settles seven design decisions, the replication settings, the prerequisites and the network model before anything is deployed. Every prerequisite must be confirmed and evidenced; any item that fails is a blocker.
Design decisions
vSphere Replication protects each VCF Operations node with a 15-minute RPO onto Site B vSAN ESA, and recovery keeps the original IPs by activating the identical Site B VLAN.
| # | Decision | Value | Rationale |
|---|---|---|---|
| D1 | Replication method | vSphere Replication through the P&R 9.1.1 appliance | Documented VCF Fleet DR method; no array dependency |
| D2 | Replication scope | VCF Operations analytics nodes only | Management components stay local to each site |
| D3 | IP handling | Same IPs; no IP customization in the recovery plan | Identical VLAN and subnet pre-built at Site B |
| D4 | Network failover | Manual, gated cutover by the network team | No L2 stretch, NSX Federation or routed IP mobility |
| D5 | Replication network | Dedicated, routed replication VLAN per site, one VMkernel per management host | Replication isolated from management and vSAN traffic |
| D6 | Test recovery network | Isolated, non-routed port group at Site B | Test VMs use production IPs and must not reach production |
| D7 | Target RTO | 4 hours or less, including network cutover | Design objective; must be proven in testing |
Replication settings per VCF Operations node
| Setting | Value | Note |
|---|---|---|
| RPO | 15 minutes | Range is 1 minute to 24 hours; below 5 minutes needs an Advanced Cyber Compliance subscription |
| Point-in-time instances | 3 copies over 24 hours | Adds vSAN capacity at Site B |
| Guest OS quiescing | Disabled | Crash-consistent, per the VCF Fleet DR guidance |
| Network compression | Enabled | Uses more host CPU at both ends |
| Network encryption | Enabled | Required for Enhanced vSphere Replication from vSphere Replication 9.0.3 |
| Target datastore | Site B management vSAN ESA datastore | |
| Target VM storage policy | vSAN ESA RAID-5 (FTT=1) or the cluster default ESA policy | On 4 to 5 hosts, RAID-5 2+1 is about 1.5× raw |
| Disk format | Same as source | Thin on vSAN |
Prerequisites checklist
Platform
☐ Both sites run VCF 9.1.1 with healthy management domains on vSAN ESA, and two separate management vCenters in separate SSO domains (P&R cannot replicate within one vCenter)
☐ Site B management vSAN has capacity for the replicas × policy overhead plus point-in-time growth, and stays below about 65% raw utilisation on a 4-host cluster
☐ The P&R 9.1.1 binaries and entitlement are available for both sites, and the vCenter versions are supported in the Broadcom Interoperability Matrix
☐ A VCF Operations backup exists and has been tested independently of replication
P&R appliance (per site)
| Item | Requirement |
|---|---|
| Appliance | One P&R 9.1.1 appliance per management vCenter; scale-out appliance not used |
| vCPU and memory | 8 vCPU, 24 GB RAM |
| Disk | About 826 GB thick provisioned (per the OVA) |
| Network | One static IP on the VM management network, with forward and reverse DNS |
| Time | NTP on vCenter, ESX hosts and P&R (KB 1318) |
| Names | FQDNs everywhere, never short hostnames, to avoid certificate thumbprint and DNS failures |
VCF Operations network at Site B (identical, dormant VLAN)
☐ Same VLAN ID, subnet, mask and gateway IP as Site A
☐ A Site B distributed port group on that VLAN, with the same teaming and MTU as Site A
☐ The Site B gateway SVI is shut down or not advertised in normal operation
☐ The network team has a written, tested cutover and cutback procedure and a named on-call owner
☐ Firewall, NAT and load-balancer rules for VCF Operations exist at Site B, or are scripted for cutover
☐ DNS records stay unchanged (the IPs do not change), and DNS and NTP servers from both sites are configured on every node
Replication network
☐ A replication VLAN and subnet at each site, routed between the sites. The "no L3 for failover" constraint applies only to the VCF Operations subnet; vSphere Replication cannot work without routed connectivity between replication VMkernels
☐ MTU validated end to end: jumbo frames where supported, or a consistent 1500
☐ Bandwidth for the initial full sync of all nodes and 15-minute deltas
Test isolation and operations
☐ An isolated Site B test port group with no route to production, or the P&R auto-created isolated network
☐ Change approval, an incident commander role and network on-call defined for DR invocation
IP budget
With 4 management hosts per site, the build needs 10 new IPs and 2 new routed VLANs. VCF Operations needs no new IPs, because Site B reuses the identical VLAN. The formula is 2 + H(A) + H(B), where H is the number of management hosts per site.
| Item | Site A | Site B | Total |
|---|---|---|---|
| P&R appliance IP (management network) | 1 | 1 | 2 |
| Replication VMkernel IPs (1 per management host) | 4 | 4 | 8 |
| New IPs | 5 | 5 | 10 |
| New replication VLAN and port group (L3 routable) | 1 | 1 | 2 |
| VCF Operations IPs | Existing | Same IPs | 0 new |
Addressing worksheet (example values)
| Object | Site A | Site B |
|---|---|---|
| P&R appliance | pr-a.sitea.corp.local, 10.10.10.30/24 | pr-b.siteb.corp.local, 10.20.10.30/24 |
| Replication VLAN, subnet, gateway | 1650, 10.10.50.0/28, 10.10.50.1 | 2650, 10.20.50.0/28, 10.20.50.1 |
| Replication port group | dpg-mgmt-a-vr-1650 | dpg-mgmt-b-vr-2650 |
| Replication VMkernel IPs | 10.10.50.11 to .14 | 10.20.50.11 to .14 |
| VCF Operations VLAN, subnet, gateway | 10.10.30.0/24, 10.10.30.1 | Same 10.10.30.0/24, 10.10.30.1 |
| VCF Operations node IPs | 10.10.30.11 (primary), .12 (replica), .13 (data) | Same |
Firewall flows
| # | Source | Destination | Purpose |
|---|---|---|---|
| F1 | P&R A and P&R B | Local vCenter | Registration and operations |
| F2 | P&R A ↔ P&R B | Each other | Site pairing |
| F3 | P&R A ↔ vCenter B, P&R B ↔ vCenter A | Peer vCenter | Cross-site operations |
| F4 | Replication subnet A ↔ replication subnet B | Both directions | vSphere Replication data |
| F5 | Replication subnets | Remote P&R appliance | Replication control |
| F6 | Admin jump host | P&R A and B on 443 and 5480 | UI and appliance management |
| F7 | VCF Operations subnet (Site B when active) | vCenters, NSX, DNS, NTP, LDAP, SMTP, adapter endpoints | Same rules as Site A, staged at Site B |
| F8 | ESX hosts running protected VMs | ESX hosts of the target cluster, TCP 32032 | Enhanced vSphere Replication data |
Take the exact ports for Protection and Recovery 9.1 from ports.broadcom.com.
Subnet states
The VLAN can exist on the Site B switches and port group at all times; only the routing (SVI or route advertisement) moves between sites.
A test never changes routing. Only a declared DR or an approved planned migration moves the gateway, and the way back always runs through reprotect and a planned migration with a cutback.
Implementation: from replication network to recovery plan
The build runs in six stages, each repeated at both sites where it applies: replication networking, P&R deployment, registration, site pairing and mappings, replication, then the protection group and recovery plan.
Each stage has a check that gates the next: vmkping before deployment, Running services before pairing, Connected sites before mapping, a completed full sync before the protection group.
Stage 1: Replication networking (both sites)
Each management host gets one replication VMkernel on a dedicated port group, so vSphere Replication traffic leaves on its own routed VLAN rather than on management (vmk0) or vSAN.
In the management vCenter, create a distributed port group on the management VDS:
dpg-mgmt-a-vr-1650(Site Bdpg-mgmt-b-vr-2650), static binding, elastic allocation, 8 ports, VLAN 1650 (Site B 2650), load balancing "Route based on physical NIC load".On every management host, add a VMkernel adapter on that port group with the vSphere Replication and vSphere Replication NFC services. Every host sends during failover and receives during failback.
Set the agreed MTU (9000 only if jumbo frames are validated end to end), a static IP from the worksheet, and Override default gateway for this adapter with the replication gateway.
On vSAN hosts the adapter is normally vmk3 (vmk0 management, vmk1 vMotion, vmk2 vSAN).
Validate in both directions from a host shell, using
-s 1472for a 1500 MTU:
vmkping -I vmk3 -d -s 8972 10.20.50.11
vmkping -I vmk3 10.20.10.30 Operational rule: any host added to a management cluster later must get a replication VMkernel before it can run or receive protected VMs.
Stage 2: Deploy the P&R 9.1.1 appliance (per site)
Download the appliance from the Broadcom Support Portal under My Downloads > Protection and Recovery > VMware Protection and Recovery > 9.1.1, and verify the checksum. The scale-out appliance is not needed.
In the local management vCenter, right-click the management cluster and select Deploy OVF Template.
Name the VM (
pr-aat Site A,pr-bat Site B), select the management cluster, the management vSAN ESA datastore and default policy, and map Network 1 to the VM management port group.On Customize template: enable SSHD only if policy allows (disable after the build); hostname = the lowercase FQDN with matching A and PTR records; root password for the OS and admin password for the Appliance Management Interface; NTP and DNS servers from both sites; IPv4 static addressing from the worksheet.
Power on, then confirm
https://<pr-fqdn>:5480opens and forward and reverse DNS resolve.In the appliance management interface, use Certificate > Generate CSR (FQDN and IP as SANs), have the enterprise CA sign it, and upload it with Certificate > Change.
Create a service account in each vCenter (for example
svc-pr-a@vsphere.local) in the Administrators group or a custom P&R role, and store it in the vault.
Stage 3: Register P&R with the local vCenter
Each appliance registers with the management vCenter at its own site.
Open
https://<pr-fqdn>:5480, log in as admin, and select Summary > Configure appliance.Enter the vCenter FQDN in lowercase, port 443 and the service account, then verify the certificate thumbprint.
On Name and Extension: a unique site name (Site-A, Site-B), the DR team mailbox, the appliance FQDN as local host, and keep the default extension ID
com.vmware.vcDr. Both appliances in a pair must use the same extension ID; a custom ID is only for shared recovery site topologies.Confirm the services show Running and the P&R plug-in appears in the vSphere Client after logging out and back in.
Stage 4: Pair sites, license and map inventory
The network mapping is the core of this design: the Site A Ops port group maps to the identical-VLAN Site B port group for real recoveries, and to the isolated test port group for tests.
In P&R at Site A, select New Site Pair and choose Pair with a peer vCenter located in a different Single Sign-On domain. Enter the Site B vCenter address exactly as it was used when P&R B was registered.
Confirm both sites show Connected, then apply and verify the P&R licence at both sites.
Create mappings with Prepare mappings manually. Automatic mapping matches objects by name and is available only for networks and folders; resource mappings are always manual.
Select Prepare reverse mappings for each mapping; reprotect and failback need them.
On Test networks, select the isolated test port group (or accept the auto-created isolated network). Never use the production Site B port group for tests, because test VMs keep production IPs.
Leave IP customization unconfigured. No IP subnet mapping rules are needed because the subnet is identical at both sites.
| Mapping | Site A | Site B | Reverse |
|---|---|---|---|
| Network (recovery) | Ops port group A | Ops port group B (identical VLAN, same subnet) | Yes |
| Network (test) | Isolated test port group (no route) | ||
| Folder | VCF Ops folder A | VCF Ops folder B | Yes |
| Resource | Management cluster A | Management cluster B | Yes |
| Storage policy (if used) | Management vSAN ESA policy A | Management vSAN ESA policy B | Yes |
Stage 5: Configure replication
The RPO sets the maximum tolerable data loss: a smaller RPO loses less data but uses more bandwidth. Before starting, confirm network encryption is enabled for Enhanced replication and TCP 32032 is open from the source ESX hosts to the target cluster hosts.
In P&R, open Replications > Outgoing > Create new replication and accept automatic vSphere Replication server assignment.
Select the VCF Operations primary, replica and data nodes only.
Target the Site B management vSAN ESA datastore with the agreed policy and Same as source disk format. Leave Select seeds cleared unless pre-seeded copies exist (wrong seeds can cause data loss), and keep Auto-include new disks selected.
Run all Replication mapping tests and confirm they pass.
Apply the settings from the planning table: RPO 15 minutes, 3 point-in-time instances per day for 1 day, quiescing off, compression and encryption on.
Choose Do not add to protection group now and Do not add to recovery plan now, finish, and watch the Initial Full Sync until every VM shows OK. Record how long it took.
Stage 6: Protection group and recovery plan
One replication set, one protection group and one recovery plan cover all VCF Operations nodes. VCF Operations for Networks uses its own protection group outside this scope.
Create protection group VCF-Operations-PG, direction Site A → Site B, type Individual VMs (vSphere Replication), with all VCF Operations nodes. Confirm every VM shows Protected (OK) and placeholder VMs exist at Site B.
Create recovery plan VCF-Operations-RP with that protection group and keep the site-level test network mapping.
Set startup priorities: primary analytics node in priority group 1, replica node in 2, data nodes in 3. For each VM, confirm IP customization is off.
Add the operator prompts as user steps of type Prompt, shown in the recovery sequence below. The VCF 9.1 procedure for customizing the VCF Operations recovery plan expects the load balancer, Cloud Proxy and node-status prompts; the network cutover gate is this design's own addition.
Recovery workflows: test, failover, reprotect and failback
One recovery plan serves four workflows. What changes between them is where the plan runs, whether the network cuts over, and what node state the operator expects at each prompt.
The recovery plan sequence and the network gate
The network cutover gate is the first prompt, before any priority 1 VM powers on. It is never dismissed until the network team confirms in writing; powering on VCF Operations with the subnet active at both sites causes a duplicate-IP outage. Confirm the real prompt order during the first test.
The plan pauses at every numbered box until an operator dismisses it, so each node is checked before the next priority group starts.
| Order | Position | Prompt | Content |
|---|---|---|---|
| 1 | First step, before power-on of priority 1 | Network cutover gate | Real recovery: Site A subnet withdrawn or down; Site B SVI up and advertised; Site B firewall, NAT and LB rules active; change number recorded. Test: VMs on the isolated test port group and Site B still dormant |
| 2 | Before priority 1 | Verify load balancer | VCF Operations LB or VIP available at Site B |
| 3 | Before priority 1 | Shut down Cloud Proxies | DR: shut down reachable Cloud Proxies at Site B. Planned migration: at both sites |
| 4 | Before priority 1 | Check primary node | In /admin. Planned: Not Running / Offline. DR: Running / Waiting for Analytics |
| 5 | Before priority 2 | Check replica node | Same as the primary check |
| 6 | Before priority 3 | Check data nodes | Planned: Not Running / Offline. DR: Running / Online |
| 7 | After the last power-on | Power on Cloud Proxies, bring cluster online | Select Bring Cluster Online if needed and wait for all nodes Online |
Network cutover procedure (network team)
The VCF Operations subnet must be active at exactly one site. P&R waits at the gate prompt while the network team works.
Confirm the trigger: DR declared, or planned migration approved.
Deactivate at Site A: shut the Ops VLAN SVI or withdraw its route, and confirm it is no longer advertised. If Site A is down, confirm it is unreachable instead.
Activate at Site B: bring up the SVI with the same gateway IP, advertise the subnet into the core and WAN, and activate the staged firewall, NAT and LB rules.
Verify: traceroute to the gateway reaches Site B; only one route to the subnet exists; DNS, NTP, LDAP and the vCenter B endpoints are reachable.
Record the change number and time. Only then does the P&R operator dismiss the gate.
Cutback for failback runs the same steps in reverse. Rollback: if Site B activation fails and Site A is still healthy, do not dismiss the gate; re-activate Site A, cancel the recovery plan run if no VMs have powered on, and escalate.
Never have both gateways up and advertised at the same time. If Site A comes back on its own after a DR, for example after a power restore, keep the Site A SVI down until the planned failback.
Workflow 1: Non-disruptive test recovery
A test powers on copies of the VCF Operations nodes at Site B on the isolated test port group, with their production IPs, while production keeps running at Site A. No network cutover happens during a test.
Before: protection group OK, recovery plan Ready, replication OK for every node; the network team confirms the test port group has no gateway, uplink route or path to production; a test jump host is on the test port group; a change record is open.
Run: select Test with Replicate recent changes to recovery site. At the gate, confirm isolation. At the Cloud Proxy prompt, do not shut down production Cloud Proxies; dismiss with a note. Check node state from the jump host at each stage.
Measure: record elapsed time to Test complete as the test RTO. Estimate network cutover time separately with the network team.
Clean up: run Cleanup; the test VMs are powered off and unregistered and the test images discarded. The plan returns to Ready.
Pass criteria: all nodes Online inside the bubble; prompt order as designed; no duplicate-IP alarms or production impact; test RTO plus network cutover estimate within 4 hours.
Workflow 2: Unplanned disaster failover
Run only when Site A, or VCF Operations at Site A, is lost and DR is declared. Everything runs from Site B.
Activation gate: the incident commander declares DR and records the time; the change owner gets emergency approval; network on-call is engaged; Site B vCenter, vSAN health, DNS, NTP and the load balancer are confirmed available.
In the Site B vCenter, open Protection and Recovery (Site A shows as unreachable), select VCF-Operations-RP > Run > Disaster recovery, and acknowledge the permanent changes.
P&R skips the protected-site steps (errors there are expected) and recovers the VMs from the latest replica on Site B vSAN.
At the gate, the network team runs the cutover. Dismiss only after their written confirmation.
Work through the LB, Cloud Proxy and node prompts using the DR targets, then power on Cloud Proxies and wait for all nodes Online.
Validate: cluster healthy; users log in through the usual FQDN or VIP (DNS unchanged, now served from Site B); adapters and Cloud Proxies collecting; a single active route to the subnet; the DR event record complete.
Workflow 3 and 4: Reprotect and planned failback
Failback has three stages and needs both sites up: reprotect (replicate B → A), a planned migration with a network cutback, then reprotect again to restore A → B.
Prerequisites: Site A vCenter, hosts, vSAN, replication VMkernels and P&R A healthy, site pair Connected; Site A gateway still down; last recovery run successful; Site A vSAN capacity for the reverse replicas; cutback window approved.
Reprotect B → A: select Reprotect on the plan. The first sync can take as long as the initial full sync. If it stops part-way, fix the cause and run Reprotect again. Recommended: run a test towards Site A on a Site A isolated port group.
Planned migration A ← B: select Run > Planned migration. P&R shuts down VCF Operations at Site B, runs a final sync and recovers at Site A. The network team runs the cutback at the gate. Shut down Cloud Proxies at both sites; the node prompts expect Not Running / Offline, the normal in-between state. Then bring the cluster online and validate.
Restore protection A → B: select Reprotect again, confirm direction Site A → Site B and plan Ready, confirm the Site B gateway is dormant, then run a test and cleanup before closing the change.
Day-2 operations: keeping the DR design true
The two ongoing risks are the dormant Site B VLAN drifting from Site A's configuration, and new hosts or nodes missing from replication. A DR design that is not tested and maintained decays quietly.
| Event | Action | Owner |
|---|---|---|
| Any change to the Site A VCF Operations VLAN, subnet, firewall, NAT or LB | Apply the identical change to the dormant Site B config in the same change record | Network |
| VCF Operations upgrade or node add (Fleet LCM) | Confirm every node is replicated (new nodes need replication and the protection group), then run a test | VCF admin |
| New management ESX host at either site | Add the replication VMkernel before the host runs protected VMs | VCF admin |
| P&R 9.1.x patch or upgrade | Upgrade both sites in the documented order, then confirm replication is OK | VCF admin |
| Site B vSAN capacity above 70% | Expand capacity or reduce point-in-time instances | VCF admin |
| Quarterly | Test recovery plus a tabletop of the network cutover | DR owner and Network |
| Every 6 to 12 months | Full planned migration and failback, including the real network cutover | DR owner and Network |
Monitoring: alert on replication status other than OK and on RPO violations; alert on recovery plan status other than Ready; and have the network team monitor that the VCF Operations subnet is advertised from exactly one site.
Implementation sign-off
☐ Replication port groups and VMkernels on every management host; vmkping passes in both directions
☐ P&R appliances deployed with CA-signed certificates, registered, paired and licensed
☐ Identical VLAN, subnet and gateway built at Site B and kept dormant; firewall, NAT and LB rules staged
☐ Network, test network, folder, resource and storage-policy mappings in both directions
☐ All nodes replicating with RPO 15 minutes, 3 point-in-time copies, quiescing off, compression and encryption on
☐ Protection group OK, recovery plan Ready, startup priorities set, all seven prompts configured
☐ Network cutover procedure signed off by the network team and tested in a maintenance window
☐ Test recovery passed, cleanup completed, RTO recorded
Record every DR event
Keep one record per test, failover, reprotect or migration: event type, change or incident number, who declared and approved, start and end time (UTC), network cutover change and time, who verified the single route, prompt outcomes and node states, measured RTO including cutover, recovery point used (latest or point in time), issues and actions, and sign-off. Over time these records are your proof that the 4-hour RTO is real.
Comments
Post a Comment