Skip to content

[SPARK-39532][INFRA] Move checkout and sync steps into re-usable action#36924

Closed
EnricoMi wants to merge 2 commits into
apache:masterfrom
G-Research-Forks:branch-ci-reuse-checkout-and-sync
Closed

[SPARK-39532][INFRA] Move checkout and sync steps into re-usable action#36924
EnricoMi wants to merge 2 commits into
apache:masterfrom
G-Research-Forks:branch-ci-reuse-checkout-and-sync

Conversation

@EnricoMi

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Steps "Checkout Spark repository" and "Sync the current branch with the latest in Apache Spark" are defined 9 times in build-and-test.yml. This moves these two steps into a composite action that is than reused.

Why are the changes needed?

Simplifies workflow file and reduces code duplication.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Tested in fork CI.

@github-actions github-actions Bot added the INFRA label Jun 20, 2022
@EnricoMi

Copy link
Copy Markdown
Contributor Author

@HyukjinKwon here is some first simplification of the build-and-test workflow.

@EnricoMi EnricoMi changed the title [CI] Move checkout and sync steps into re-usable composite action [SPARK-39532][CI] Move checkout and sync steps into re-usable composite action Jun 20, 2022
@EnricoMi EnricoMi changed the title [SPARK-39532][CI] Move checkout and sync steps into re-usable composite action [SPARK-39532][CI] Move checkout and sync steps into re-usable action Jun 20, 2022
@EnricoMi EnricoMi force-pushed the branch-ci-reuse-checkout-and-sync branch from 1d9577d to 7bb761f Compare June 20, 2022 10:19
@HyukjinKwon HyukjinKwon changed the title [SPARK-39532][CI] Move checkout and sync steps into re-usable action [SPARK-39532][INFRA] Move checkout and sync steps into re-usable action Jun 20, 2022
@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@HyukjinKwon HyukjinKwon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice. LGTM if it works

git -c user.name='Apache Spark Test Account' -c user.email='sparktestacc@gmail.com' commit -m "Merged commit" --allow-empty

branding:
icon: 'check-circle'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick question: what is this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is defines the icon of the action for UI and marketplace purposes, for instance the blue circle with the triangle at https://github.com/marketplace/actions/checkout

Not sure if this is required, does not make much sense if action is not published.

@EnricoMi

Copy link
Copy Markdown
Contributor Author

This is not applicable to Apache Spark CI as it first checksout Spark repo, then the fork repo, while this action (until merged into upstream repo) only becomes available after the sync step, which is main part of the composite action.

In other words, this action cannot be executed until it has been merged into master. This makes it also difficult to contribute to.

@EnricoMi EnricoMi closed this Jun 20, 2022
@HyukjinKwon

Copy link
Copy Markdown
Member

uhoh, right. thanks for explanation though :-).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants