Skip to content

create --json: report the deduplicated size of the new archive (usize is computed but only printed in the text --stats block) #10335

Description

@ioanalytica

Have you checked borgbackup docs, FAQ, and open GitHub issues?

Yes. Searched for deduplicated_size, usize, --json in issues; the closest is #10329 (repository-level repo-info --stats), which is about the repository, not the archive just created.

Is this a BUG / ISSUE report or a QUESTION?

Feature request.

System information

borg 2.0.0b24 (also checked against master src/borg/archive.py and src/borg/archiver/create_cmd.py as of today), Linux, rest:// repository.

Describe the problem

borg create --json is the only machine-readable place to learn what a backup run did, and it does not report the deduplicated size of the new archive, although create already knows it.

  • Statistics.as_dict() (src/borg/archive.py) returns original_size, nfiles, hashing_time, chunking_time, files_stats, store_stats. usize is not in it.
  • Statistics.usize is accumulated during create (self.usize += size for every chunk that is new to the repository) and printed in the human --stats block as Deduplicated size:.
  • create_cmd.py: args.stats |= args.json, then if args.json: json_print(...) else log_multi(..., str(archive.stats), ...). With --json the text block is not emitted, so a caller that wants the JSON document has no way to get the deduplicated size at all, not even by scraping stderr.
  • For info, calc_stats() sets stats.usize = 0 # this is expensive to compute, so the value cannot be recovered later from the archive either. The only after-the-fact path is borg analyze (b23), which walks a set of archives.

Concretely, borg create --stats --json on b24 yields

"stats": {"chunking_time": 0.0, "files_stats": {...}, "hashing_time": 0.0, "nfiles": 717, "original_size": 195587663, "store_stats": {}}

and borg info --json aid:… for the same archive yields the same set, with original_size and nfiles and nothing about deduplication. A frontend that stores per-archive numbers ends up with "unknown" for the one figure that says what the archive costs, while borg had it in hand a moment earlier.

Proposal

Add "deduplicated_size": FileSize(self.usize) to the create --json output. The value is already computed there, so the cost is one dict entry. Two ways to fit it in:

  1. Statistics.as_dict() always includes deduplicated_size. For info it would be 0, which is misleading; so either
  2. as_dict() takes a flag (want_unique, mirroring calc_stats) and create passes it, or create_cmd adds the key to the archive dict it hands to basic_json_data, and info leaves the key out.

Option 2 keeps info --json honest (absent means "not computed", not "zero"). The dry-run branch would stay as it is (nfiles, original_size only), since nothing is written there.

Borg 1.x had deduplicated_size in this exact place (it meant deduplicated compressed size then; per changes.rst 2.0.0a2 the 2.x value is the deduplicated original size). Callers that read the 1.x field will read the 2.x field the same way; only the semantics changed, which the changelog already documents.

Happy to test a build against the repositories above and report the numbers.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions