Skip to content

xapi: Fix pool.join breaking due to dangling ref - #7365

Open
last-genius wants to merge 1 commit into
xapi-project:masterfrom
last-genius:asv/dangling_local_cache_sr
Open

last-genius wants to merge 1 commit into
xapi-project:masterfrom
last-genius:asv/dangling_local_cache_sr

Conversation

@last-genius

Copy link
Copy Markdown
Contributor

SR.forget can create a dangling reference in Host.local_cache_sr. During
pool join, this will prevent the supporter host from creating its host object
on the coordinator, which leads to a broken supporter host:

[debug||pool.join|xapi_pool] Found no host with uuid = 'SUPPORTER' on the master, so creating one.
[debug||pool.join|xapi_pool] Ignoring exception: Db_exn.DBCache_NotFound("missing row", "SR", "OpaqueRef:DANGLING")
[error||pool.join|backtrace] Raised Db_exn.DBCache_NotFound("missing row", "SR", "OpaqueRef:DANGLING")
[error||pool.join|backtrace] 1/5 xapi Raised at file ocaml/database/db_cache_impl.ml, line 151
[error||pool.join|backtrace] 2/5 xapi Called from file ocaml/xapi/db_actions.ml, line 14885
[error||pool.join|backtrace] 3/5 xapi Called from file ocaml/xapi/xapi_pool.ml, line 1101
[error||pool.join|backtrace] 4/5 xapi Called from file ocaml/xapi/xapi_pool.ml, line 1539
[error||pool.join|backtrace] 5/5 xapi Called from file lib/backtrace.ml, line 110

In case of an error, simply set Host.local_cache_sr to Ref.null and continue.
Thus, after the fix:

[debug||pool.join|xapi_pool] Found no host with uuid = 'SUPPORTER' on the master, so creating one.
[debug||pool.join|xapi_pool] Creating host object on master

@last-genius

Copy link
Copy Markdown
Contributor Author

I tested this manually - pool.join works after the fix and is broken without it.

Comment thread ocaml/xapi/message_forwarding.ml Outdated
let remote_fn = Client.SR.forget ~sr in
with_sr_marked ~__context ~sr ~doc:"SR.forget" ~op:`forget (fun () ->
Xapi_sr.assert_all_pbds_unplugged ~__context ~sr ;
Xapi_sr.assert_sr_not_local_cache ~__context ~sr ;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this new assert, if a host is ejected from the pool, and if one of its SR has a local cache, it will raise an exception right? Can't we disable local storage when PBDs are unplugged? Somewhere before https://github.com/xapi-project/xen-api/blob/master/ocaml/xapi/xapi_pool.ml#L2330 maybe. I'm thinking of adding the same function than disable_local_storage_caching but without the "bacon assertion".
Or maybe it is ok for now to accept this kind of failure?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn, you're right, we don't want this to happen. I've dropped the message_forwarding commit, only keeping the guard in xapi_pool

SR.forget can create a dangling reference in Host.local_cache_sr. During
pool join, this will prevent the supporter host from creating its host object
on the coordinator, which leads to a broken supporter host:

    [debug||pool.join|xapi_pool] Found no host with uuid = 'SUPPORTER' on the master, so creating one.
    [debug||pool.join|xapi_pool] Ignoring exception: Db_exn.DBCache_NotFound("missing row", "SR", "OpaqueRef:DANGLING")
    [error||pool.join|backtrace] Raised Db_exn.DBCache_NotFound("missing row", "SR", "OpaqueRef:DANGLING")
    [error||pool.join|backtrace] 1/5 xapi Raised at file ocaml/database/db_cache_impl.ml, line 151
    [error||pool.join|backtrace] 2/5 xapi Called from file ocaml/xapi/db_actions.ml, line 14885
    [error||pool.join|backtrace] 3/5 xapi Called from file ocaml/xapi/xapi_pool.ml, line 1101
    [error||pool.join|backtrace] 4/5 xapi Called from file ocaml/xapi/xapi_pool.ml, line 1539
    [error||pool.join|backtrace] 5/5 xapi Called from file lib/backtrace.ml, line 110

In case of an error, simply set Host.local_cache_sr to Ref.null and continue.
Thus, after the fix:

    [debug||pool.join|xapi_pool] Found no host with uuid = 'SUPPORTER' on the master, so creating one.
    [debug||pool.join|xapi_pool] Creating host object on master

Signed-off-by: Andrii Sultanov <andriy.sultanov@vates.tech>
@last-genius
last-genius force-pushed the asv/dangling_local_cache_sr branch from c0b7c21 to 1cea848 Compare November 4, 2026 13:34

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants