Skip to content

Add characterization tests for async and legacy network sync - #415

Merged
rtibbles merged 3 commits into
release-v0.9.xfrom
net-characterization-363
Nov 18, 2026
Merged

rtibbles merged 3 commits into
release-v0.9.xfrom
net-characterization-363

Conversation

@bjester

@bjester bjester commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Adds characterization tests that pin what a full network push and pull send and store, for the sync layer refactors in M2–M4. They cover both the async flow (ASYNC_OPERATIONS) and the legacy flow used with servers before 0.6.0
  • A spy on SessionWrapper.request records each request and passes the call through to the real method. Per request, it records the method, path, query params, body encoding (json/gzip), body keys, the values of the stable body fields (filter, push, records_total, records_transferred, transfer_stage), the record count of each pushed chunk, and is_retryable. Generated IDs are normalized to <id>
  • Asserts the final local transfer session fields, the Buffer, Store and app rows on both sides, and the byte counts the server stores
  • Fails a sync that sends more than 100 requests or runs longer than 120s, so a stalled sync fails instead of hanging the test run
  • For the legacy tests, the live server runs with MORANGO_TEST_LEGACY_SERVER, which turns off ASYNC_OPERATIONS and FSIC_V2_FORMAT in server2_settings.py. LiveServer and BasePushPullClientTestCase take extra server environment variables for that
  • Extracts the live server harness from test_syncsession.py into integration/helpers.py as BasePushPullClientTestCase, so the new tests reuse it without re-running PushPullClientTestCase's tests

Reviewer guidance

  • The recording goes beyond the plan's (method, path, body keys, transfer_stage) tuple. Mutation checks showed the narrower tuple let these through, and the expanded recording catches each: dropping is_retryable, sending buffer chunks as JSON instead of gzip, un-flipping the byte counts in the pull PATCH, and adding a query param to the buffer GET
  • The legacy tests emulate an older server, and the client keeps its full capabilities, as a current Kolibri does when it syncs with an older one. Tests that pin today's legacy network sync behavior #364 proposed reducing the client capabilities instead, but that fails today on the server for both push and pull, which is now tracked in Clients that disallow async operations cannot sync with servers that allow them #416
  • The legacy pull moves 11 records, not 10. The client creates the transfer session before it serializes, so it sends an empty client_fsic, and the legacy server queues the user from the initial pull again. This is today's behavior, pinned as-is
  • URL paths are literal strings on purpose, not reverse() or the api_urls constants. The client sends them to peers on other morango versions, so a renamed route must fail these tests
  • chunk_size is set in the test class rather than inherited, because the expected request lists depend on it and the harness is not frozen
  • Once this merges, test_network_characterization.py must not change until M4 is complete

Issues addressed

Closes #363
Closes #364

Related: #416

AI usage

Claude Code implemented the tests from the issues and the implementation plan. I had a separate Claude subagent review the async tests against the issue and spec. Its mutation checks showed regressions that the plan's narrower request tuple missed, so I had the recording strengthened, and directed extracting the shared harness into integration/helpers.py. While implementing the legacy tests, Claude found that the approach in #364 fails against the current code, so we switched to emulating a legacy server and filed #416. Claude ran the integration suite and the mutation checks to verify the changes.

🤖 Generated with Claude Code

@bjester
bjester added this pull request to stack #412 November 5, 2026 21:41
@bjester bjester changed the title Net characterization 363 Add characterization tests for async network push and pull Nov 5, 2026
@bjester bjester changed the title Add characterization tests for async network push and pull Add characterization tests for async and legacy network sync Nov 5, 2026
@bjester
bjester marked this pull request as ready for review November 6, 2026 18:34
@bjester
bjester requested review from a team and leq-bot November 6, 2026 18:34
@bjester
bjester removed this pull request from stack #412 November 7, 2026 00:24
@bjester
bjester added this pull request to stack #418 November 7, 2026 00:24
@bjester
bjester force-pushed the net-characterization-363 branch from 842b8b6 to d93e3d1 Compare November 13, 2026 19:38
@rtibbles rtibbles self-assigned this Nov 17, 2026
@rtibbles
rtibbles force-pushed the net-characterization-363 branch from d93e3d1 to 950d6e2 Compare November 17, 2026 19:20
Base automatically changed from sandboxing-defense to release-v0.9.x November 18, 2026 16:36
@rtibbles
rtibbles force-pushed the net-characterization-363 branch from 950d6e2 to 6a7244a Compare November 18, 2026 16:36
@bjester
bjester requested a review from rtibblesbot November 18, 2026 18:23

rtibblesbot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Waiting for changes

Outcome: reviewed d4c5757

Last updated: 2026-11-18 18:57 UTC

@rtibblesbot rtibblesbot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #415: the characterization suite covers the #363/#364 acceptance criteria. The harness move into helpers.py is faithful. CI passes.

  • suggestion: legacy tests reduce the server's capabilities, not the client's as #364 specifies (inline).
  • suggestion: two docstring/comment inaccuracies in the recorder (inline).
  • pre-existing: assert in second_environment is stripped under python -O (inline).

Next step: Needs a full code review by a human engineer.

  • Decide whether the legacy characterization should emulate a reduced-capability client (as #364 states) or a legacy server, since M2–M4 must pass it unedited.

Written by rtibblesbot, an LLM-based coding agent.

bjester and others added 3 commits November 18, 2026 10:44
The second server, second_environment and the sync session setup move
from test_syncsession.py into integration/helpers.py, with the setup in
a new BasePushPullClientTestCase. PushPullClientTestCase extends it and
keeps its tests and the activity timestamp checks they use.

Other test modules can now reuse the harness without importing
PushPullClientTestCase, which would collect and run its tests again.

Refs #363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pins what a full push and a full pull with ASYNC_OPERATIONS send and
store, so the refactors of the sync layer can show they change neither.

A spy on SessionWrapper.request records each request, then passes the
call through to the real method. For each request it records the
method, path, query params, body encoding, body keys, the values of
the body fields that are stable between runs, the record count of each
pushed chunk, and the is_retryable flag. Generated IDs are normalized
to <id>. The tests compare the recording with fixed expected lists for
every transfer stage.

The tests also assert the final local transfer session, the Buffer,
Store and app rows on both sides, and the byte counts the server
stores. The spy fails a sync that sends more than 100 requests or runs
longer than 120 seconds, so a stalled sync fails instead of hanging.

Closes #363

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pins what a full push and a full pull send and store when the server
lacks ASYNC_OPERATIONS, so the refactors of the LegacyNetwork*
operations can show they change neither.

The tests run against a live server that emulates a morango version
before 0.6.0: the MORANGO_TEST_LEGACY_SERVER variable turns off both
ASYNC_OPERATIONS and FSIC_V2_FORMAT on the server, and LiveServer now
takes extra environment variables for that. The client keeps its full
set of capabilities, as a current Kolibri does when it syncs with an
older one. Reducing only the client capabilities, as #364 first
proposed, fails today on the server for both push and pull (#416).

The expected lists show that the client creates the transfer session
and sends no stage updates for serializing, queuing, dequeuing, or
deserializing. The legacy pull moves 11 records instead of 10: the
client creates the transfer session before it serializes, so it sends
an empty client_fsic, and the server queues the user from the initial
pull again.

Closes #364

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bjester
bjester force-pushed the net-characterization-363 branch from 6a7244a to d4c5757 Compare November 18, 2026 18:44

@rtibblesbot rtibblesbot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #415: no new findings. All 4 prior findings are resolved or acknowledged. CI is still pending.

Prior-finding status

RESOLVED — tests/testapp/tests/integration/test_network_characterization.py:129 — comment says "keys shared by its records" but code takes the union
RESOLVED — tests/testapp/tests/integration/test_network_characterization.py:53 — encoding docstring omits "data"
RESOLVED — tests/testapp/tests/integration/helpers.py:45 — second_environment assert skipped under python -O
ACKNOWLEDGED — tests/testapp/tests/integration/test_network_characterization.py:435 — legacy tests reduce the server's capabilities, not the client's (client-side gap deferred to #416)

Next step: This is a low-risk change; it needs a light-touch code review by a human engineer.


Written by rtibblesbot, an LLM-based coding agent.

@rtibbles
rtibbles merged commit 89473cf into release-v0.9.x Nov 18, 2026
75 checks passed

Copy link
Copy Markdown
Contributor

Quick check on rtibblesbot's review of this PR: react 👍 helpful · 😕 mixed · 👎 not helpful. Reply with anything specific.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tests that pin today's legacy network sync behavior Tests that pin today's async network push and pull behavior

3 participants