Skip to content

Fix issue #1055 and clean up the nix and packaging environments - #1057

Open
simoninns wants to merge 2 commits into
happycube:mainfrom
simoninns:main
Open

Fix issue #1055 and clean up the nix and packaging environments#1057
simoninns wants to merge 2 commits into
happycube:mainfrom
simoninns:main

Conversation

@simoninns

Copy link
Copy Markdown
Collaborator

Fix issue #1055 and clean up the nix and packaging environments

This started as a fix for #1055ld-cut was not usable from an installed
copy of ld-decode — and ended up as a cleanup of everything around it: the
last two shell-outs to tools that are not part of this repo are gone, the
flake installs a self-contained set of commands, and all three release
packages are built and smoke tested the same way.

Issue #1055: ld-cut

ld-cut was a loose script at the repo root. It was not listed in
pyproject.toml, so pip install and the flake never put it on PATH, and
writing .lds output shelled out to ld-lds-converter — a C++ tool from the
separate ld-decode-tools repo — so it failed even when ld-cut itself was
reachable.

  • The body moved to lddecode/cut.py behind a main(), and ld-cut is now a
    console entry point in pyproject.toml. The root ld-cut stays as a thin
    shim for running from a source checkout.
  • .lds output is packed in process by lddecode.lds, so it no longer
    depends on anything being on PATH.

ld-lds-converter ported to Python

lddecode/lds.py is an in-tree port of the C++ ld-lds-converter, installed
as ld-lds-converter-py. tests/test_lds.py checks the packing and unpacking
against the original tool's format, so output stays byte-identical to the
legacy ld-lds-converter.exe.

ld-compress rewritten in Python

scripts/ld-compress (a bash script) became lddecode/compress.py, installed
as ld-compress, with tests/test_compress.py covering it and a root
ld-compress shim for source checkouts. Along the way:

  • Progress bar. Percentage, bytes read, throughput and ETA. Drawn only
    when stderr is a terminal, so redirecting to a log file keeps the log clean;
    -n turns it off at a terminal, -p forces it on when it is not one.
  • GPU (FlaLDF/OpenCL) encoding removed. flac 1.5.0 encodes multithreaded
    on its own, so the extra code path and its external dependency bought
    nothing. Compression levels are now flac's 1–8, default 8.
  • External dependencies reduced to one. flac is the only program
    ld-compress runs; it needs 1.5.0 or later for -j. Uncompress (-u) and
    verify (-v) need nothing external at all — they decode through PyAV, the
    same FFmpeg binding ld-decode already uses, and pack with lddecode.lds.
  • Windows is no longer a special case. It is the same command as on Linux
    and macOS, and it finds the flac.exe shipped beside it with no PATH setup.
  • pv, xxd, awk, openssl and the hand-rolled ffmpeg pipelines the old
    script relied on are all gone.

Nix flake

  • ld-cut, ld-compress and ld-lds-converter-py are installed and exposed
    as nix run .#<tool> apps alongside ld-decode and ld-ldf-reader-py.
  • The installed commands get flac and ffmpeg, plus the package's own
    bin, prefixed onto PATH — prefixed rather than replaced, so the user's own
    tools stay reachable. Nothing has to be installed separately any more.
  • mainProgram set, flac added to the dev shell.

Release packages

All three now ship the same five commands plus flac 1.5.0, and all three
prove it before publishing: --version on the tools, then a full
ld-lds-converter-pyld-compressld-compress -u round trip compared
byte for byte against the original.

  • AppImage. Rebuilt around a python-build-standalone interpreter instead
    of a Nix closure. Nix binaries have absolute /nix/store paths baked in, so
    the previous AppImage could not start on a machine without that exact store
    path. The tools are self-locating shell wrappers, pip's build-time console
    scripts are stripped, and the build fails if anything in the bundle still
    refers to the build directory. flac is compiled statically so it does not
    raise the bundle's glibc requirement. The relocation test runs from a copy
    at a different path with env -i.
  • macOS DMG. PyInstaller builds for all five tools; --windowed is used
    only for the ones that are never a stdio filter, since ld-compress and
    ld-lds-converter-py read and write pipes. flac is built statically from
    source rather than taken from Homebrew, so there are no dylibs to relocate.
    Nested executables are signed individually before the bundle is sealed,
    which Apple silicon requires.
  • Windows ZIP. .bat wrappers for all five tools in bin\, alongside
    Xiph's official flac.exe 1.5.0 and libFLAC.dll and their licences.
    ld-compress.bat runs a real script file rather than python -c, so
    argv[0] points at bin\ and the flac lookup works.

Tests

  • tests/test_lds.py.lds packing/unpacking, including format
    compatibility with the C++ converter.
  • tests/test_compress.py — compression, uncompression, verification, CLI
    behaviour.
  • ctest gains python-unit-tests, cut-ntsc-lds (ld-cut writing .lds),
    decode-ntsc-lds (that the result decodes) and compress-lds-round-trip
    (cmake_modules/LdsRoundTrip.cmake, skipped when flac is absent).

Docs

INSTALL.md, BUILD.md, the three docs/installation/ pages and
docs/user-guide/scripts.md updated for the new command set, the bundled
flac, the removed GPU path and the new ld-compress options. The Windows
drag-and-drop .bat recipe now just calls ld-compress. Also corrected
nix profile install to nix profile add.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant