Rework on serialization and deserialization in pagination

[Yury-Fridlyand]

Supersedes Bit-Quill#252

Description

Rework PhysicalPlan tree serialization and deserialization. Make it scalable and extendable.
If a new plan should be serialized (a new feature is becoming supported by pagination) - it should properly implement SerializablePlan methods.
The plan tree will be recovered in deserialization the same way it was before serialization.
New mechanism can also skip plans from serialization - for example, it skips ResourceMonitorPlan and PaginateOperator.

This change is required for Pagination Phase 2 features.

Code in this PR is based on #1497/#1483.

Issues Resolved

N/A

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1498 (9f04686) into feature/pagination/P1 (b9cb0d0) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@                     Coverage Diff                     @@
##             feature/pagination/P1    #1498      +/-   ##
===========================================================
- Coverage                    98.51%   98.51%   -0.01%     
+ Complexity                    4013     3995      -18     
===========================================================
  Files                          361      363       +2     
  Lines                         9905     9879      -26     
  Branches                       643      633      -10     
===========================================================
- Hits                          9758     9732      -26     
  Misses                         142      142              
  Partials                         5        5              

Flag	Coverage Δ
sql-engine	98.51% <100.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...java/org/opensearch/sql/storage/StorageEngine.java	100.00% <ø> (ø)
...nsearch/data/value/OpenSearchExprValueFactory.java	100.00% <ø> (ø)
...opensearch/request/ContinuePageRequestBuilder.java	100.00% <ø> (ø)
.../opensearch/request/InitialPageRequestBuilder.java	100.00% <ø> (ø)
...rch/sql/opensearch/setting/OpenSearchSettings.java	100.00% <ø> (ø)
...ql/opensearch/storage/OpenSearchStorageEngine.java	100.00% <ø> (ø)
...ge/scan/OpenSearchIndexScanAggregationBuilder.java	100.00% <ø> (ø)
.../storage/scan/OpenSearchIndexScanQueryBuilder.java	100.00% <ø> (ø)
...nsearch/storage/script/ExpressionScriptEngine.java	100.00% <ø> (ø)
...ge/script/aggregation/AggregationQueryBuilder.java	100.00% <ø> (ø)
... and 14 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[GabeFernandez310]

😆

[MaxKsyunz]

Use try-with-resources (try (... gzip = new ...) { .. }) here so that gzip is closed even if exception is thrown.

Call to close will become unnecessary.

https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html

[Yury-Fridlyand]

This causes java.io.EOFException: Unexpected end of ZLIB input stream.

  protected String serialize(Serializable object) throws NotSerializableException {
    try {
      try (ByteArrayOutputStream output = new ByteArrayOutputStream();
          ObjectOutputStream objectOutput = new ObjectOutputStream(output)) {
        objectOutput.writeObject(object);
        objectOutput.flush();

        try (ByteArrayOutputStream out = new ByteArrayOutputStream();
            // GZIP provides 35-45%, lzma from apache commons-compress has few % better compression
            GZIPOutputStream gzip = new GZIPOutputStream(out) {
              {
                this.def.setLevel(Deflater.BEST_COMPRESSION);
              }
            }) {
          gzip.write(output.toByteArray());
          return HashCode.fromBytes(out.toByteArray()).toString();
        }
      }
    } catch (NotSerializableException e) {
      throw e;
    } catch (IOException e) {
      throw new IllegalStateException("Failed to serialize: " + object, e);
    }
  }

[MaxKsyunz]

Both these streams are Closable. Good place to use try-with-resources.