Skip to content

Finalize archives into a compact file at the final path #382

Description

@bjester

Overview

A completed archive becomes a compact file at its final path, and all temporary files and locks are removed. This issue is part of milestone M5 (archive).

Background & Motivation

During export, morango writes the archive to <path>.partial. The final path must exist only when the export completed. A reader can then trust that a file at the final path is a complete archive.

The dequeue step moves rows from Buffer to Store in the archive. This leaves free pages in the file. VACUUM reclaims them, but it needs up to twice the file size in free space. The target disk does not always have this space. The archive uses incremental auto-vacuum instead, which reclaims pages in place.

Design: spec. Plan: implementation plan, Task 21.

Description & Expected Outcomes

The archive keeps its manifest as a typed value. A write of a new manifest replaces all manifest entries.

When the archive is finalized, the archive records completed_at in the manifest and reclaims free pages. Then the archive closes its connection and removes the archive alias. It renames <path>.partial to <path> and releases the archive lock.

After finalization, a call to discard does nothing. The final file stays in place.

Deliverables & Contracts

The feature delivers these capabilities:

  • Archive.write_manifest(manifest) replaces all manifest entries.
  • Archive.manifest returns the current manifest value.
  • Archive.finalize() produces the final file and cleans up all temporary state.

Acceptance Criteria

  • After finalize, path exists, and <path>.partial, <path>.partial-journal and <path>.lock do not exist.
  • After finalize, the archive alias is not in connections.databases, and finalized is True.
  • A reopened archive shows completed_at set, and all other manifest fields are unchanged.
  • If 5,000 buffer rows are inserted and deleted before finalize, the final file is smaller than the partial file before finalize.
  • A call to discard after finalize does not change or delete the final file.
  • The rename test passes in the windows tox environment.

Technical Pointers & Architecture

  • Target Components / Context: morango/sync/archive.py (Archive, ArchiveManifest).
  • Related Patterns: Spec §3.5 describes the finalize sequence.
  • Data Model & Schema Considerations: Uses ArchiveManifestEntry rows from Store archive metadata in a morango table #376. No new schema.
  • Resilience & Failure Modes: If finalization fails, the export fails. The caller calls close(), which discards the partial file. There is no resume before Phase 3.

Notes & Tradeoffs

Metadata

  • Complexity: Medium
  • Target Branch: release-v0.9.x

AI Usage

Drafted with Claude (Claude Code) from the approved design spec and implementation plan. The author reviewed the requirements, and the code references were checked against the release-v0.9.x codebase.

Activity

  1. ArpitaSamanta10 commented on Nov 19, 2026

    @ArpitaSamanta10

    @bjester
    I would love to work on this issue can you assign this for me

    /assign

  2. learning-equality-bot commented on Nov 19, 2026

    @learning-equality-bot
    Contributor

    Hi! 👋

    Thanks so much for your interest! This issue is not open for contribution. Visit Contributing guidelines to learn about the contributing process and how to find suitable issues. If there are no unassigned 'help wanted' issues available, please wait until new ones are added.

    We really appreciate your willingness to help. 😊

    Are you preparing for Google Summer of Code? See our GSoC guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions