Skip to content
This repository was archived by the owner on Oct 22, 2026. It is now read-only.

Do not reload table schema when update catalog cache (#2667) - #2677

Merged
ti-chi-bot[bot] merged 3 commits into
pingcap:release-3.2from
ti-chi-bot:cherry-pick-2667-to-release-3.2
Apr 27, 2023
Merged

ti-chi-bot[bot] merged 3 commits into
pingcap:release-3.2from
ti-chi-bot:cherry-pick-2667-to-release-3.2

Conversation

@ti-chi-bot

Copy link
Copy Markdown
Member

This is an automated cherry-pick of #2667

What problem does this PR solve?

TiSpark will reload all table schemas when init catalogCache. It will also reload all table schemas once we call listTable or getTable with the invalid cache.

One user reports that TiSpark hangs with more than 10000+ table schemas.

What is changed and how it works?

Add a configuration spark.tispark.load_tables , default is true.
when set it to false, we will not load all tables when we reload the cache. table cache will only be loaded when the table is used.

Check List

Tests

  • I have tested it locally and it works.
  • User has tested and it works.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot

ti-chi-bot commented Apr 20, 2023 •

Copy link
Copy Markdown
Member Author

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • xuanyu66
  • zhangyangyu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Details

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@shiyuhang0

Copy link
Copy Markdown
Contributor

/run-all-tests

@shiyuhang0

Copy link
Copy Markdown
Contributor

/run-all-tests

@shiyuhang0

Copy link
Copy Markdown
Contributor

/merge

@ti-chi-bot

ti-chi-bot Bot commented Apr 27, 2023

Copy link
Copy Markdown

This pull request has been accepted and is ready to merge.

DetailsCommit hash: 0c036cb

@ti-chi-bot
ti-chi-bot Bot merged commit 0947e43 into pingcap:release-3.2 Apr 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants