Repository navigation
Add tests for the AWS Batch executor boto schemas - #72609
Merged
Merged
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
The three marshmallow schemas in boto_schema.py sit on the OVERLOOKED_TESTS allowlist and, unlike most entries there, are not covered indirectly either - nothing under providers/*/tests references them. They parse live AWS Batch API responses, so the behaviour worth pinning down is the data_key renaming from the API's camelCase, the unknown=EXCLUDE policy that tolerates fields AWS adds over time, and the post_load hook that returns a BatchJob rather than a plain dict. Removes the module's entry from OVERLOOKED_TESTS in the same commit, as that list fails once a listed test file exists. related: apache#35442 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VKBUyiL28LCzdLsc3kqEoX
sokojh
force-pushed
the
add-tests-for-batch-boto-schema
branch
from
September 7, 2026 05:23
4681b2b to
0d0ba88
Compare
vincbeck
approved these changes
Sep 9, 2026
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
imrichardwu
pushed a commit
to imrichardwu/airflow
that referenced
this pull request
Sep 11, 2026
The three marshmallow schemas in boto_schema.py sit on the OVERLOOKED_TESTS allowlist and, unlike most entries there, are not covered indirectly either - nothing under providers/*/tests references them. They parse live AWS Batch API responses, so the behaviour worth pinning down is the data_key renaming from the API's camelCase, the unknown=EXCLUDE policy that tolerates fields AWS adds over time, and the post_load hook that returns a BatchJob rather than a plain dict. Removes the module's entry from OVERLOOKED_TESTS in the same commit, as that list fails once a listed test file exists. related: apache#35442
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
providers/amazon/.../executors/batch/boto_schema.pyis on theOVERLOOKED_TESTSallowlist in
airflow-core/tests/unit/always/test_project_structure.py, tracked bythe meta issue #35442.
Unlike most entries on that list, the three schemas in this module are not covered
indirectly either — no file under any
providers/*/tests/directory referencesBatchSubmitJobResponseSchema,BatchJobDetailSchemaorBatchDescribeJobsResponseSchema. (For contrast,build_submit_kwargsin thesibling
batch_executor_config.pyis exercised fromtest_batch_executor.py,which is why this PR is scoped to
boto_schema.pyalone.)These schemas parse live AWS Batch API responses, so the behaviour worth pinning
down is the part that is easy to break silently: the
data_keymapping from theAPI's camelCase to Airflow's snake_case, the
unknown = EXCLUDEpolicy that letsthe schemas tolerate fields AWS adds over time, and the
@post_loadhook thatmakes
BatchJobDetailSchemareturn aBatchJobrather than a plain dict.How
Adds
providers/amazon/tests/unit/amazon/aws/executors/batch/test_boto_schema.pywith one test class per schema, covering for each:
jobId→job_idrenaming,submit_job/describe_jobsare dropped ratherthan raising,
ValidationError,BatchJobDetailSchemareturns aBatchJobinstance withstatus_reasonleft as
Nonewhen the API omits it,BatchDescribeJobsResponseSchemabuilds oneBatchJobper entry, acceptsan empty
jobslist, and propagates a nested job'sValidationError.No production code is touched.
What
13 tests added. The module's entry is removed from
OVERLOOKED_TESTSin the samePR, as that list fails when a listed test file exists.
Verified locally:
related: #35442
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines