Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactors partiql-tests-runner for multiple engines #1289

Merged
merged 3 commits into from
Dec 8, 2023

Conversation

rchowell
Copy link
Contributor

@rchowell rchowell commented Dec 8, 2023

Description

This PR updates the conformance test-runner for partiql-lang-kotlin to support multiple backends. The conformance test data is stored as Ion, so each backend needs toIon and fromIon methods for value loading.

Other Information

  • Updated Unreleased Section in CHANGELOG: [YES/NO]
    No, not a public API

  • Any backward-incompatible changes? [YES/NO]
    No, updates to tests only

  • Any new external dependencies? [YES/NO]
    No

  • Do your changes comply with the Contributing Guidelines
    and Code Style Guidelines? [YES/NO]
    Yes

License Information

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link

github-actions bot commented Dec 8, 2023

Conformance comparison report

Base (f8f6b2a) afd611b +/-
% Passing 92.33% 92.54% 0.21%
✅ Passing 5372 5384 12
❌ Failing 446 434 -12
🔶 Ignored 0 0 0
Total Tests 5818 5818 0

Number passing in both: 5372

Number failing in both: 434

Number passing in Base (f8f6b2a) but now fail: 0

Number failing in Base (f8f6b2a) but now pass: 12

The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see
  • equiv attribute value pair unpivot non-missing, compileOption: LEGACY
  • equiv attribute value pair unpivot missing, compileOption: LEGACY
  • notInPredicateSingleExpr, compileOption: LEGACY
  • tuple navigation with array notation without explicit CAST to string, compileOption: LEGACY
  • path on string, compileOption: LEGACY
  • tuple navigation missing attribute dot notation, compileOption: LEGACY
  • tuple navigation missing attribute array notation, compileOption: LEGACY
  • single source FROM with bag and AT clause, compileOption: LEGACY
  • single source FROM with scalar and AT clause, compileOption: LEGACY
  • single source FROM with tuple and AT clause, compileOption: LEGACY
  • single source FROM with absent value null and AT clause, compileOption: LEGACY
  • single source FROM with absent value missing and AT clause, compileOption: LEGACY

@codecov-commenter
Copy link

codecov-commenter commented Dec 8, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f8f6b2a) 72.20% compared to head (c0d4b9b) 72.20%.
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #1289   +/-   ##
=========================================
  Coverage     72.20%   72.20%           
  Complexity     2100     2100           
=========================================
  Files           222      222           
  Lines         16019    16019           
  Branches       2891     2891           
=========================================
  Hits          11567    11567           
  Misses         3635     3635           
  Partials        817      817           
Flag Coverage Δ
CLI 13.43% <ø> (ø)
EXAMPLES 80.28% <ø> (ø)
LANG 80.57% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rchowell rchowell marked this pull request as ready for review December 8, 2023 15:51
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this was an accidental addition. To mitigate this, these results should be placed in some ignored directory like build IMO. It doesn't need to be addressed with this PR, but this file should be removed.

Copy link
Member

@johnedquinn johnedquinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad for this refactoring and simplification. Thanks!

@rchowell rchowell merged commit 1c3c79a into main Dec 8, 2023
@rchowell rchowell deleted the test-runner-refactor branch December 8, 2023 19:34
alancai98 added a commit that referenced this pull request Dec 12, 2023
* Adds support for EXCLUDE in the SqlDialect

* fix typer/transfomrer

* Fix Timestamp Type Parsing Issue. (#1284)

* fix timestamp type parsing issue

* Refactors partiql-tests-runner for multiple engines (#1289)

* Adds int32 and int64 variants of date_add (#1291)

* Adds support for DISTINCT in the Planner

* Fix PlanCompiler output type for DISTINCT + ORDER BY (#1298)

* Fixes scan_indexed and adds PIVOT (#1297)

* Port filter_distinct fix to partiql-eval internal

---------

Co-authored-by: John Ed Quinn <johqunn@amazon.com>
Co-authored-by: Yingtao Liu <yliuuu@amazon.com>
Co-authored-by: yliuuuu <107505258+yliuuuu@users.noreply.github.com>
Co-authored-by: John Ed Quinn <40360967+johnedquinn@users.noreply.github.com>
Co-authored-by: R. C. Howell <RCHowell@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants