Skip to content

VDatum references: add MTL/DTL, stop extracting grids into the working directory, and three smaller fixes #190

Closed

Description

@mmacferrin

Found while moving IVERT's vertical datum handling onto transformez 1.0 (continuous-dems/ivert#132). All against transformez 1.0.0 (92dc0a2) with fetchez 0.9.2.dev1.

1. Add vdatum:mtl and vdatum:dtl

reference/vdatum.py lists the mean tide level and diurnal tide level surfaces among the VDatum regional surfaces:

_VDATUM_REGIONAL_SURFACES = {"dtl", "mhhw", "mhw", "mllw", "mlw", "msl", "mtl", "tss"}

But CUSTOM_VERTICAL_REFERENCES and OPERATION_BINDINGS in reference/bindings.py have entries only for vdatum:mllw, mlw, msl, mhw and mhhw, so neither surface can be reached:

>>> transformez.generate_grid(region=[-76.5, -76.4, 38.9, 39.0], increment="3s",
...                           datum_in="vdatum:mtl", datum_out=5703)
InvalidReferenceError: Unknown custom reference: 'vdatum:mtl'

EPSG has no vertical CRS for mean tide level, so a transformez reference is the only way to name it. Adding vdatum:mtl and vdatum:dtl alongside the other tidal surfaces would make both available. I haven't been able to check that their grids download and apply, since there is no reference to test them through yet.

2. VDatum model grids are extracted into the current working directory

ReferenceFetcher._fetch_vdatum_model_grid (reference/fetcher.py:505) extracts without an output directory:

for grid_path in p_f_extract(archive_path, members=[archive_member]):

fetchez's p_f_extract() defaults outdir to "./", so the grid lands wherever the caller happens to be running, not in cache_dir. The other two p_f_extract() calls in the same file pass one. A transform through the xgeoid20b model is enough to see it:

# run from an empty directory
transformez.generate_grid(region=[-118.5, -118.4, 33.6, 33.7], increment="3s",
                          datum_in=4979, datum_out="vdatum:mllw",
                          cache_dir="/some/other/cache")
# the working directory now holds xgeoid20b/ and .vdatum_XGEOID20B.zip.<hash>.extract.lock

In IVERT, a few validation runs left 199 MB of xgeoid20b/ plus the lock files in the repository root. Since the grid is not in the cache, it is presumably extracted again from each new working directory. The line was last changed in #189.

3. InvalidReferenceError and UnsupportedReferenceError derive from Python's built-in ReferenceError

reference/parser.py:

class InvalidReferenceError(ReferenceError):
    pass

class UnsupportedReferenceError(ReferenceError):
    pass

ReferenceError isn't defined in the module, so these subclass the built-in one, which Python uses for dead weak references. Callers can't catch "any transformez reference error" through one base class, and except ValueError misses them, although the neighbouring ReferenceInputError is a ValueError. A transformez base class, perhaps deriving from ValueError, would let callers catch all three together.

4. epsg:5714 is labelled as MLLW

reference/bindings.py:

"epsg:5714": VerticalReference(
    id="epsg:5714",
    name="Mean Lower Low Water (Generic)",

EPSG:5714 is MSL height; the name looks copied from the epsg:5866 entry above it. Only the label is wrong: transforms through epsg:5714 match vdatum:msl exactly.

5. (Enhancement) Accept common short names

parse_reference() accepts EPSG codes, PROJ names ("NAVD88 height") and transformez IDs, but not the short names people usually type: navd88, egm2008, egm96, prvd02, vivd09, cgvd2013 all raise InvalidReferenceError. IVERT keeps its own alias table for this (ivert/vdatum_lookup.py), and globato has a smaller one in streams/readers/bag.py. A shared alias table in transformez would let both drop theirs.

Activity

  1. mmacferrin commented on Sep 30, 2026

    @mmacferrin
    ContributorAuthor

    If you want @matth-love, I can build and submit a PR that'll address these and close the issue. You don't need to handle them yourself, I'd just need you to review it first (make sure I'm not introducing any bugs) before merging it.

  2. matth-love commented on Sep 30, 2026

    @matth-love
    Contributor

    Yes, Mike. These all look good to me if you want to try and correct them. I would skip doing anything about 4 right now; that is deliberate atm to handle legacy poorly defined datums in BAG files. We plan to remove that when we add the proper mapping/handling of this in the globato BAG reader.

  3. mmacferrin commented on Sep 30, 2026

    @mmacferrin
    ContributorAuthor

    See #191, pending your review @matth-love. And like you noted, I left out item number 4 for now, until it's resolved in Globato's BAG reader.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions