Repository navigation
Conversation
PyMongo's server_selector keyword filters servers after read preference and before the latency window. Expose the same MongoClient option so an application can send eligible operations to one replica set member or mongos without turning the topology into a standalone.
|
Hi @tap1r , thanks again for the PR and your clear write-up. The reason that the Node Driver does not have this specific functionality is because the custom server selector is optional in the Server Selection specification. We considered it before (NODE-1491) and decided not to add this support because we did not have a concrete use case that required it. Can you provide some more information about your use case? Some questions about your case:
If none of the approaches work, could you tell us why? Unfortunately, without a concrete use case, we are unlikely to add this optional functionality to the Node Driver. |
|
Thanks @PavelSafronov, and for digging up NODE-1491. I'm glad you mentioned that the Server Selection spec allows a custom selector function. That's the reason I took this approach in the first place. PyMongo exposes it as Taking your questions in turn:
I really tried, and I couldn't find any combination of tags and read preferences that sends commands to a specific member deterministically. The one workaround that does work is the internal How would you suggest solving this? |
Skip the user filter for the connect handshake and endSessions. Reject servers that were not eligible, pass copies into the filter, and select the original descriptions.
# Conflicts: # src/connection_string.ts
Deep-copy electionId, topologyVersion, $clusterTime, and error before the user filter runs, so an in-place edit cannot change SDAM state. Document preference versus a hard pin, which operations can see a secondary, and how transactions call the filter. Export the helper from the bundled test barrel.
|
Thanks for the detailed reply. I tried your rolling-compact workflow on an Atlas replica set (8.0) using a direct connection per member instead of a custom selector with the following approach:
The driver sends that command only to that member, and it sends Note on Atlas: Atlas doesn’t allow database users to run You mentioned in the original post that “directConnection… skips replica-set discovery”. In this workflow, discovery only needs to happen once to build the member list, and the regular client handles that. Would this approach work for your use case? |
|
Yes. That is essentially the implementation I use today. Performing topology discovery on the main client, then creating a directConnection client per member, is how these commands get a hostname at all. compact is only one of them, and it is not guaranteed to be a short-running command. The primary does not replicate it; the documented procedure is to run it on each secondary in turn. Runtime grows as a function of collection size and database write load, and a single namespace can run for hours under pathological conditions. The direct clients, one or more of them, have to stay up for that entire run, each with its own copy of TLS, auth, serverApi, compression, and timeouts. The same pattern shows up for the other member-local admin commands: validate, dbStats, collStats, fsync / fsyncLock, replSetResizeOplog, replSetMaintenance, setProfilingLevel, planCacheClear, getParameter / setParameter, and reading or killing operations on one process. None of these are cluster-wide commands. Each one has to land on a chosen mongod. Atlas does not provide a way to do that with the existing selection options. The predefined replica-set tags (nodeType, provider, region, availability zone) select a class of nodes, and users cannot change them. localThresholdMS and maxStalenessSeconds only rank eligible members. A mongos host list does not apply. So the working approach is to leave the discovered topology and open a Single client directly to that member. That connection rewrites the default primary read preference to primaryPreferred, which is why the secondary accepts the command. That workaround is why I asked for serverSelector. The main client has already discovered the member and already has a pool to it. The spec allows an application to make the final choice after normal selection. NODE-1491 deferred the hook for lack of a use case; this is that use case, and it is a class of commands rather than one example. replSetStepDown on Atlas is a separate limitation, and I am not asking the selector to solve it. Whether the driver exposes the optional hook is a product decision. The heavy lifting for the implementation is already in this pull request: selection order, the operations that must skip the hook, copies that cannot change SDAM state, documentation, and tests. The direct client shows that the need is real, and it is what I have to maintain until the driver can target one member of a topology it is already monitoring. |
Summary
PyMongo's
server_selectorkeyword is a driver-specificMongoClientoption. It lets an application narrow server selection to one member of a replica set or one mongos, while the driver keeps discovering the rest of the deployment. The Node driver already hasdirectConnection, which is different: that option forces a Single topology and skips replica-set discovery.This adds the same hook as
serverSelectoronMongoClient. It is not a URI option, because the value is a function.Selection order matches PyMongo:
serverSelectorsees that list, beforelocalThresholdMS, so it can choose a slower eligible member.The function is not called when no server is eligible, for a Single topology (
directConnection), for a load-balanced topology, or for operations pinned to a server (getMore,killCursors). Writes still only see writable servers, so this does not send a normal write to a secondary.Omitting the option leaves selection unchanged.
Test plan
Topology#selectServerwiringtsc --noEmittest/integration/server-selection/server_selector.test.tson a replica set (needs the driver's auth test user; not run against the local deployment)EOF