Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update tests and test data for relevancy search functions #707

Merged
merged 21 commits into from
Jul 25, 2022
Merged

Update tests and test data for relevancy search functions #707

merged 21 commits into from
Jul 25, 2022

Conversation

Yury-Fridlyand
Copy link
Collaborator

Description

  • Reduce test data set (3.7k lines down to 60)
  • Update tests according to the new data set
  • Add new tests which validate additional parameters
  • Reorganize (rename, move) tests to have them stored under single standard

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

opensearch-trigger-bot bot and others added 20 commits April 19, 2022 18:10
@opensearch-trigger-bot @ylwu-amzn
bump ml-client to 2.0 (opensearch-project#568) (opensearch-project#569)
01af4b6 
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
(cherry picked from commit 0ae97ad)

Co-authored-by: Yaliang Wu <ylwu@amazon.com>
@opensearch-trigger-bot @vamsimanohar
2.0.0.0-rc1 release notes (opensearch-project#582) (opensearch-projec…
40f881e 
…t#585)

* 2.0.0.0-rc1 release notes

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit 4c7111a)

Co-authored-by: vamsi-amazon <99925918+vamsi-amazon@users.noreply.github.com>
@opensearch-trigger-bot @joshuali925
Remove explicit node version for workbench (opensearch-project#588) (o…
eb65d2d 
…pensearch-project#589)

Signed-off-by: Joshua Li <joshuali925@gmail.com>
(cherry picked from commit a7d4395)

Co-authored-by: Joshua Li <joshuali925@gmail.com>
@joshuali925 @vamsimanohar
Remove rc1 qualifier for 2.0 (opensearch-project#600)
6b37929 
* Remove rc1 qualifier for 2.0

Signed-off-by: Joshua Li <joshuali925@gmail.com>
(cherry picked from commit d71cddc)
@vamsimanohar
Release Notes 2.0.0.0
7e70f0b 
Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit 14fc606)
@MitchellGale
Merge branch 'main' of https://github.com/opensearch-project/sql into…
1317394 
… HEAD

� Conflicts:
�	build.gradle
�	workbench/opensearch_dashboards.json
�	workbench/package.json
@MitchellGale
Added test cases for using pipe in flags argument inside Simple_Query…
5432ba3 
…_String. Added implementation for simple_query_string to use multiple flags seperated by pipes.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale
Added test cases for using pipe in flags argument inside Simple_Query…
69bf216 
…_String. Added implementation for simple_query_string to use multiple flags seperated by pipes.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale
Removed extra new line.
475946f 
Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale
removed un-used imports in SimpleQueryStringQuery.java.
ad999c6 
Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale
Removed extra spaces in import group.
922471f 
Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale
Merge pull request #89 from Bit-Quill/dev-enumlikevaluessupportforfla…
478c27c 
…gsinSimpleQueryString

Adds support for enum like values in flag argument in Simple Query String
@MitchellGale
Merge branch 'main' of https://github.com/opensearch-project/sql into…
8b7e93c 
… Integ-enumLikeValuesdSupprtForFalsinSimpleQueryString
@Yury-Fridlyand
Some renamings to put tests under same order.
4cacdb1 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Rework on test: reduce data set, add new tests, etc.
17d8f87 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Update match_phrase and match_phrase_prefix tests.
00ae2c3 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Merge remote-tracking branch 'origin/Integ-enumLikeValuesdSupprtForFa…
6e1cd62 
…lsinSimpleQueryString' into dev-update-test-data-relevancy-search

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Update simple_query_string tests by adding complex cases for flags.
951a717 
Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Merge pull request #90 from Bit-Quill/dev-update-test-data-relevancy-…
2dcb002 
…search

Update test data for relevancy search integration tests
@Yury-Fridlyand
Merge remote-tracking branch 'upstream/main' into integ-update-test-d…
f638c40 
…ata-relevancy-search

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand Yury-Fridlyand requested a review from a team as a code owner July 22, 2022 03:43
@codecov-commenter
Copy link

codecov-commenter commented Jul 22, 2022 •
edited
Loading

Codecov Report

Merging #707 (109afe9) into main (592c58b) will decrease coverage by 31.98%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##               main     #707       +/-   ##
=============================================
- Coverage     94.74%   62.76%   -31.99%     
=============================================
  Files           283       10      -273     
  Lines          7676      658     -7018     
  Branches        560      118      -442     
=============================================
- Hits           7273      413     -6860     
+ Misses          349      192      -157     
+ Partials         54       53        -1
Flag Coverage Δ
query-workbench 62.76% <ø> (ø)
sql-engine ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ilter/lucene/relevance/SimpleQueryStringQuery.java
...nsearch/sql/expression/window/WindowFunctions.java
...h/sql/exception/ExpressionEvaluationException.java
...arch/sql/expression/aggregation/MaxAggregator.java
...sql/expression/aggregation/AggregatorFunction.java
...a/org/opensearch/sql/data/model/ExprTimeValue.java
...ch/sql/data/utils/NullsFirstExprValueOrdering.java
...arch/sql/expression/aggregation/AvgAggregator.java
...rg/opensearch/sql/data/model/ExprMissingValue.java
...a/org/opensearch/sql/data/model/ExprNullValue.java
... and 263 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

penghuo
penghuo approved these changes Jul 22, 2022
View reviewed changes
Copy link
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the change!

@Yury-Fridlyand
Merge remote-tracking branch 'upstream/main' into integ-update-test-d…
109afe9 
…ata-relevancy-search

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>
@joshuali925 joshuali925 merged commit 5c6fd72 into opensearch-project:main Jul 25, 2022
@Yury-Fridlyand Yury-Fridlyand deleted the integ-update-test-data-relevancy-search branch July 25, 2022 22:28
MitchellGale pushed a commit to Bit-Quill/opensearch-project-sql that referenced this pull request Jul 27, 2022
@Yury-Fridlyand @MitchellGale
Added support for enum like values in Simple Query String.
6c77e07 
Signed-off-by: Mitchell Gale <mitchellg@bitquilltech.com>

Adds support for multiple enum like values to be passed to simple query string.

Update tests and test data for relevancy search functions (opensearch-project#707)

Signed-off-by: Yury Fridlyand <yuryf@bitquilltech.com>

Added integration test for escape character in select statement.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

AOS-637: PoC remove escape characters

Signed-off-by: Andrew Carbonetto <andrewc@AndrewsMBP2022.local>

Fixed number of escape characters returned by plugin. Number returned is now expected number.

'I\'m' -> I'm
'I\\''m' -> I\'m
'I\\\'m' -> I\'m

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Addedd test for 'im' in literals.txt.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added unit test for unquoteText in StringUtils.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed old unescapeBackslashes from StringUtils which was replaced by unquoteText.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed * from import of Assertions in StringUtilsTest.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed indentation. Moved to 2 space indents to replace 4 space indents.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Put static imports before regular imports. Maintained alphabetical order in StringUtilsTest.java

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added case for double single quote which converts to a single single quote.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added new line at end of StringUtilsTest.java and added space between static and non-static imports

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Replaced statement lambda with expression lambda.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Allows for quotes(") and backticks(`) to be around strings in result.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added additional test cases to unquoteTest() in StringUtilsTest.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added additional test cases to unquoteTest() in StringUtilsTest.java. Added case for double, double quote being converted to single, double quote.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Made double single and double double quotes only reduce down to a single of their respective type when the string is quoted by the same type. Added some unit tests to test for that condition in StringUtilsTest.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed whitespace found before comma in StringUtilsTest.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added more tests. Removed formatting changes in OpenSearchExecutionEngine.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting ProjectOperator.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Resolved issue with overlapping replacement of quotes (double and single).

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Moved `replace` periods to new line.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Changed StringUtils.java to use String Builder. Updated some tests.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Changed over to fully use string builder and works with escape as well as double quotes.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed formatting of StringUtilsTest.java, removed code in comments in StringUtils.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Double quote implemented to convert to single quote when string is surrounded by same string. Consistent with MySQL standard.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Cleaned up redundant code from StringUtils.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Removed IT test added.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Moved whichquote inside unquote text. Removed large portion of code from if statement to break off for a return. Added unit test, removed unit tests for whichquote function. Removed whichquote function.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added some additional unit tests. Added branch for case of quoted by back tick to skip over String Builder. Fixed issue with strings passed to unquotetext being too small casuing array bounds access issue.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Fixed test case expected.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting changes to OpenSearchExecutionEngine.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Reverting changes to ProjectOperator.java.java.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added test cases in literals.txt.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>

Added back tick and moved String builder declaration down.

Signed-off-by: mitchellg <mitchellg@bitquilltech.com>
@MitchellGale MitchellGale mentioned this pull request Jul 28, 2022
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@penghuo penghuo penghuo approved these changes

@joshuali925 joshuali925 joshuali925 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Yury-Fridlyand @codecov-commenter @penghuo @joshuali925 @vamsimanohar @MitchellGale