A resilient clinic network runbook is a dependency map plus a set of tested transitions. It should show how the clinic moves from normal service to a known fallback, how each critical path is validated and how the team returns to normal without losing track of temporary changes.
This article focuses on architecture, implementation, failure modes and testing. It complements a buyer-facing clinic downtime checklist being prepared by Sakal Network, but does not duplicate that operational guide.
Start with service paths, not a device inventory
A flat list of routers, switches and access points does not explain whether reception can reach the appointment platform. Model each critical service as an end-to-end path.
Create separate paths for VoIP, payment terminals, printing, secure remote support and any local clinical equipment that communicates over the network. Mark shared dependencies. DNS, power, identity and the edge device often sit underneath several apparently independent services.
Define health checks for every layer
“The internet is down” is too vague for a runbook. Use checks that isolate layers:
- Power: edge device, switch and access point are powered; UPS state is known.
- Local link: expected interface is up; device receives the intended VLAN and address.
- Gateway: default gateway responds from the affected segment.
- DNS: approved resolver returns expected results.
- External path: a stable endpoint is reachable without relying on the target application.
- Identity: test account can complete the expected authentication flow.
- Application: synthetic or manual check confirms the critical transaction, not only the login page.
Record the exact command or user action, expected result, timeout and evidence location. Avoid public endpoints that may change without notice.
Design failover as an explicit state change
Automatic failover is useful only when the fallback path is understood and monitored. Model at least four states:
- Primary healthy
- Primary degraded but still carrying traffic
- Fallback active
- Recovery and return to primary
Define the signal that permits each transition. Packet loss, latency, DNS failure and application failure are different conditions. A link can be technically up while the application path is unusable.
If using a secondary circuit or mobile connection, specify which traffic is allowed during fallback. Voice, appointment access and remote support may be prioritised while operating-system updates, guest Wi-Fi and large backups are paused. Confirm that the fallback plan does not depend on a device or account stored only on the failed network.
Account for IP allowlists and session behaviour
A secondary internet path may present a different public address. Check whether practice systems, remote-access tools, payment services or vendor portals use source-IP allowlists. Document the change process and the expected delay.
Existing sessions may not survive a path change. The runbook should tell staff which applications must be closed and reopened, whether devices need a fresh DHCP lease and how to confirm that traffic has moved to the intended circuit.
Keep fallback addressing and DNS predictable
A failover plan can break when the secondary router supplies a different subnet or DNS policy. Prefer a design where the normal edge device retains control of internal addressing and changes only the upstream path. If that is not possible, document the alternate address range, reservations, routing and access-control implications.
Do not bypass segmentation simply to make fallback easier. Guest, staff, voice and specialised-device networks should retain their intended boundaries. Any emergency exception needs an owner, expiry time and rollback step.
Build a runbook with executable steps
Each procedure should contain:
- Trigger: the observed condition that starts the procedure.
- Authority: who may activate or cancel it.
- Pre-checks: evidence that distinguishes the suspected failure.
- Actions: numbered commands or physical steps with device names.
- Validation: service-path checks and expected results.
- Communication: who is told, by which approved channel.
- Rollback: how to return to the previous state.
- Evidence: logs, screenshots, timestamps and ticket reference.
Avoid instructions such as “restart everything”. Restarting several components at once destroys diagnostic evidence and makes it harder to identify the actual dependency.
Test with progressive scope
Begin with a tabletop review. Walk through a primary-circuit failure and ask whether the team can reach the runbook, vendor details and management interface without the failed service.
Next, validate components in a controlled window:
- Confirm configuration backups and administrative access.
- Test the secondary circuit without changing production routing.
- Validate DNS, identity and a representative application through the fallback path.
- Move one controlled test device or VLAN where the design permits.
- Run a time-bounded failover with agreed observers.
- Return to primary and verify that temporary rules, routes and pauses are removed.
Use a pre-approved stop condition. If patient-facing work, phones or payment paths behave unexpectedly, halt the exercise and restore the last known state.
Test application transactions, not just connectivity
A successful ping does not prove that reception can search an appointment, that a VoIP handset can place a call or that a user can complete multi-factor authentication.
Create a small validation matrix with test account, action, expected result and evidence. Use non-production or synthetic data where possible. Never put real patient information into monitoring scripts or screenshots used for general troubleshooting.
Monitor both the primary and fallback paths
If the secondary circuit is checked only during an incident, it is not a reliable fallback. Monitor link state, address assignment and a controlled external check. Alert on expired subscriptions, disabled SIMs, configuration drift and unreachable management interfaces.
Keep fallback alerts distinct from production alerts. A degraded backup path is important, but it should not create so much noise that the team ignores a real outage.
Capture evidence and turn failures into changes
For each test, record the planned scenario, start and end time, participants, state transitions, validation results, rollback result and follow-up owner. Store evidence in the same change or service-management system used for the runbook.
Classify findings:
- architecture gap;
- configuration drift;
- missing access or credential path;
- unclear operational decision;
- vendor dependency;
- monitoring or evidence gap.
Assign a due date based on operational impact. Do not mark the test complete merely because primary service was restored.
Common failure modes
- The secondary connection works, but the practice platform rejects the new public address.
- DNS remains pointed at an unreachable internal resolver.
- VoIP and staff data share a fallback path with no traffic policy.
- The runbook is stored on a drive that cannot be reached during the outage.
- Only one administrator can access the edge device.
- Temporary firewall rules remain after the exercise.
- Monitoring checks the router but not the business transaction.
Implementation checklist
- End-to-end service-path diagrams
- Named devices, interfaces, VLANs and owners
- Primary and fallback state definitions
- Layered health checks with expected outputs
- Traffic policy for constrained fallback
- IP allowlist and session-impact review
- Runbook available through an independent path
- Progressive test plan with stop and rollback conditions
- Application-level validation matrix
- Monitoring for both circuits and configuration drift
- Evidence log and assigned remediation actions
Where this fits in a managed IT programme
The network runbook should connect to backup testing, identity administration, vendor escalation and the clinic’s operational downtime plan. Treating it as a one-off router document leaves the most important dependencies outside the exercise.
For Singapore SMEs that need help mapping and operating these controls, see Sakal Network managed IT support. A contextual link to the related clinic downtime article will be added after that article has a public canonical URL.
Teams reviewing approved business technology options can also browse the Sakal Shop catalogue.