Skip to content

[BUG] Dataset downloader reports success and replaces data after a failed download #2276

Open

Description

@0z5a

Describe the bug

datasets/get_test_data.sh can return exit status 0 after a download fails, while removing and recreating an existing destination directory. A developer can therefore lose the prior dataset and receive a success status even though the replacement was never downloaded or extracted.

Steps to reproduce

From a source checkout, this uses a temporary directory and a fake wget; it does not touch an existing dataset or make a network request:

task_dir=$(mktemp -d)
mkdir -p "$task_dir/bin" "$task_dir/work/cvrp"
cp datasets/get_test_data.sh "$task_dir/work/"
printf 'existing data\n' > "$task_dir/work/cvrp/sentinel"
printf '#!/bin/sh\nexit 8\n' > "$task_dir/bin/wget"
chmod +x "$task_dir/bin/wget"
(
  cd "$task_dir/work"
  env -u CUOPT_S3_URI -u CUOPT_AWS_ACCESS_KEY_ID -u CUOPT_AWS_SECRET_ACCESS_KEY \
    PATH="$task_dir/bin:$PATH" ./get_test_data.sh --cvrp
  printf 'exit status: %s\n' "$?"
  if test -f cvrp/sentinel; then echo 'original preserved'; else echo 'original removed'; fi
)

Observed: the script reports a failed download and missing archive, prints exit status: 0, and prints original removed.

Expected behavior

Report a nonzero status when a selected dataset cannot be downloaded or extracted, and preserve an existing destination unless a complete replacement is ready. The same distinction should hold for any selected dataset, not just CVRP.

Environment and scope

Reproduced on Linux against main at 52d14b4c97de5591a8054ef1971a92e10617aed7. The fake-download case exercises the HTTP fallback in a disposable directory. An isolated ordinary run also observed an HTTP 404 followed by status 0; the issue is the script's failure handling rather than the availability of any particular external URL.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions