Skip to content

CA-432914: Don't let clusterd tracing setup fail Cluster_host.enable - #7379

Open
MarkSymsCtx wants to merge 1 commit into
xapi-project:masterfrom
MarkSymsCtx:private/marksy/CA-432914
Open

MarkSymsCtx wants to merge 1 commit into
xapi-project:masterfrom
MarkSymsCtx:private/marksy/CA-432914

Conversation

@MarkSymsCtx

Copy link
Copy Markdown
Contributor

Cluster_host.enable pushed the tracing configuration to xapi-clusterd (Xapi_observer.initialise_observer) before enabling it, and any failure there failed the whole enable.

xapi-clusterd handles local calls one at a time. When it is waiting for quorum, which is normal while a pool's hosts are coming back up, the observer call queues behind that wait. We saw a pool boot where the coordinator's enable blocked for 6m40s on Observer.set_trace_log_dir and then failed with Observer_error(0). The enable was abandoned, so its GFS2 SR could not be plugged.

Tracing is not a precondition for clustering. This configures it once clusterd has been enabled (and after a join), and only logs a warning if that fails.

Testing:

  • New unit test test_enable_survives_observer_failure: with a clusterd whose observer calls fail, the enable succeeds and clusterd is enabled before tracing is configured. Without the fix it fails with the same Observer_error(0) seen in the field.
  • test_clustering (46 tests) passes.
  • An internal BVT/BST regression suite run on a build with this change: no failures attributable to it.

🤖 Generated with Claude Code

Cluster_host.enable pushed the tracing configuration to xapi-clusterd
before enabling it, and a failure there failed the whole enable. A
clusterd waiting for quorum leaves that RPC unanswered: on a pool booting
after an unclean shutdown the coordinator's enable blocked for 6m40s,
then failed with Observer_error(0), and the GFS2 SR stayed unplugged.

Tracing is not a precondition for clustering. Configure it once clusterd
is enabled, and only warn if that fails.

Signed-off-by: Mark Syms <mark.syms@citrix.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant