Skip to content
This repository was archived by the owner on Oct 22, 2026. It is now read-only.

Do not reload table schema when update catalog cache (#2667) - #2678

Merged
ti-chi-bot merged 4 commits into
pingcap:release-3.0from
ti-chi-bot:cherry-pick-2667-to-release-3.0
Apr 20, 2023
Merged

ti-chi-bot merged 4 commits into
pingcap:release-3.0from
ti-chi-bot:cherry-pick-2667-to-release-3.0

Conversation

@ti-chi-bot

Copy link
Copy Markdown
Member

This is an automated cherry-pick of #2667

What problem does this PR solve?

TiSpark will reload all table schemas when init catalogCache. It will also reload all table schemas once we call listTable or getTable with the invalid cache.

One user reports that TiSpark hangs with more than 10000+ table schemas.

What is changed and how it works?

Add a configuration spark.tispark.load_tables , default is true.
when set it to false, we will not load all tables when we reload the cache. table cache will only be loaded when the table is used.

Check List

Tests

  • I have tested it locally and it works.
  • User has tested and it works.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot

ti-chi-bot commented Apr 20, 2023 •

Copy link
Copy Markdown
Member Author

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • shiyuhang0
  • zhangyangyu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@shiyuhang0

Copy link
Copy Markdown
Contributor

/run-all-tests

shiyuhang0
shiyuhang0 previously approved these changes Apr 20, 2023
zhangyangyu
zhangyangyu previously approved these changes Apr 20, 2023
@shiyuhang0
shiyuhang0 dismissed stale reviews from zhangyangyu and themself via 7b069a8 April 20, 2023 05:50
@shiyuhang0

Copy link
Copy Markdown
Contributor

/run-all-tests

@shiyuhang0

Copy link
Copy Markdown
Contributor

/merge

@ti-chi-bot

Copy link
Copy Markdown
Member Author

This pull request has been accepted and is ready to merge.

DetailsCommit hash: fb66ea1

@ti-chi-bot
ti-chi-bot merged commit 8c17eef into pingcap:release-3.0 Apr 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants