Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix](segmentload) should remove segment from segment cache if load segment failed #41608

Merged
merged 6 commits into from
Oct 10, 2024

Conversation

yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Oct 9, 2024

Proposed changes

  1. If the segment load failed, it is not recoverable, should remove it from segment cache and the query will create new segment again.
  2. Could not depend on remove segment from cache during load index failed, because the call once may throw exception and the error handle logic is not reached.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei
Copy link
Contributor Author

yiguolei commented Oct 9, 2024

run buildall

Copy link
Contributor

github-actions bot commented Oct 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.27% (9629/25838)
Line Coverage: 28.67% (79881/278594)
Region Coverage: 28.10% (41281/146929)
Branch Coverage: 24.72% (21033/85080)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cbc85af3e8cd0a9920cad2cc23b1e386328091d8_cbc85af3e8cd0a9920cad2cc23b1e386328091d8/report/index.html

@yiguolei
Copy link
Contributor Author

yiguolei commented Oct 9, 2024

run buildall

Copy link
Contributor

github-actions bot commented Oct 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

Comment on lines 919 to 922
auto status = [this]() {
return _inverted_index_file_reader_open.call(
[&] { return _open_inverted_index(); });
}();
Copy link
Contributor

@gavinchou gavinchou Oct 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto why not just

Suggested change
auto status = [this]() {
return _inverted_index_file_reader_open.call(
[&] { return _open_inverted_index(); });
}();
auto status = _inverted_index_file_reader_open.call([&] { return _open_inverted_index(); });

Defer defer([&]() { _footer_pb.reset(); });
return _create_column_readers(*_footer_pb);
});
auto status = [this]() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just

auto status = _create_column_readers_once_call.call(...);

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.30% (9639/25840)
Line Coverage: 28.68% (79919/278631)
Region Coverage: 28.10% (41331/147077)
Branch Coverage: 24.72% (21044/85144)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5aa513a58e75d289d62edba128976f0265cb5422_5aa513a58e75d289d62edba128976f0265cb5422/report/index.html

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yiguolei
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yiguolei
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.24% (9643/25893)
Line Coverage: 28.56% (79991/280122)
Region Coverage: 27.98% (41344/147751)
Branch Coverage: 24.60% (21054/85594)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c6456cf28bc87476d0d82505640d04ae80fdec7e_c6456cf28bc87476d0d82505640d04ae80fdec7e/report/index.html

@yiguolei yiguolei added the p0_c label Oct 10, 2024
Copy link
Member

@eldenmoon eldenmoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 10, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 8f7e106 into apache:master Oct 10, 2024
26 of 31 checks passed
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 10, 2024
…d segment failed (apache#41608)

## Proposed changes

1. If the segment load failed, it is not recoverable, should remove it
from segment cache and the query will create new segment again.
2. Could not depend on remove segment from cache during load index
failed, because the call once may throw exception and the error handle
logic is not reached.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Oct 10, 2024
…d segment failed (apache#41608)

## Proposed changes

1. If the segment load failed, it is not recoverable, should remove it
from segment cache and the query will create new segment again.
2. Could not depend on remove segment from cache during load index
failed, because the call once may throw exception and the error handle
logic is not reached.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit that referenced this pull request Oct 10, 2024
yiguolei added a commit that referenced this pull request Oct 11, 2024
## Proposed changes

Issue Number: close #xxx
introduced by #41608

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 11, 2024
…he#41685)

## Proposed changes

Issue Number: close #xxx
introduced by apache#41608

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit that referenced this pull request Oct 11, 2024
…) (#41697)

## Proposed changes

pick #41685
Issue Number: close #xxx
introduced by #41608

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024
…d segment failed (apache#41608)

## Proposed changes

1. If the segment load failed, it is not recoverable, should remove it
from segment cache and the query will create new segment again.
2. Could not depend on remove segment from cache during load index
failed, because the call once may throw exception and the error handle
logic is not reached.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 15, 2024
…d segment failed (apache#41608)

## Proposed changes

1. If the segment load failed, it is not recoverable, should remove it
from segment cache and the query will create new segment again.
2. Could not depend on remove segment from cache during load index
failed, because the call once may throw exception and the error handle
logic is not reached.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 15, 2024
…he#41685)

## Proposed changes

Issue Number: close #xxx
introduced by apache#41608

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 19, 2024
…d segment failed (apache#41608)

## Proposed changes

1. If the segment load failed, it is not recoverable, should remove it
from segment cache and the query will create new segment again.
2. Could not depend on remove segment from cache during load index
failed, because the call once may throw exception and the error handle
logic is not reached.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Oct 19, 2024
…he#41685)

## Proposed changes

Issue Number: close #xxx
introduced by apache#41608

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.3-merged p0_c reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants