feat(arrow): merge main #560
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…GinX-THU#444) * use default meta config for mysql * for tests * restore tests * use props file for pg * restore tests * remove unused file * restore mysql test config --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
…GinX-THU#439) 把以前 parquet 没合并的内容补上
sql support const arithmetic expression
1. 增加生成自增列的语法 2. 增加KEY列的操作语法，包括：将特定列升级成KEY列，将KEY列降级成普通列 详见https://oxlh5mrwi0.feishu.cn/docx/Gd8pd1aMDoNMIexXBPJcaea6n0g
* feat(shared): retry connect * feat(jdbc): new prefix `jdbc:iginx:tsdb:` --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
IGinX-THU#456) * fix(MAX_HEAP_SIZE): set max heap siez max(min(1/2 ram, 1024MB), min(1/4 ram, 64GB)) * fix start_iginx.bat * fix(MAX_HEAP_SIZE): user-defined ratio * fix(MAX_HEAP_SIZE): user-defined ratio * fix(MAX_HEAP_SIZE, MIN_HEAP_SIZE): user-defined percentage * fix(MAX_HEAP_SIZE, MIN_HEAP_SIZE): user-defined percentage --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
Test connection before registering engine. When a new database is registered, check for existing databases with the same IP and port. If the connection fails and the existing database is read-only, delete the existing database.
* docs: add FAQ to quickstart to explain how to resolve problems The following artifacts could not be resolved: cn.edu.tsinghua.iginx:parquet-file * chore: fix `mvn clean install -U -Dmaven.test.skip=true` * build: fix session_py format --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
实现了规则组，将一系列相同的规则放到一个规则组中，以简化规则的设置和更改。 原有的Set Rules命令现在会更改规则组而不是规则 目前以下规则合并入规则组 一系列谓词下推规则 -> FilterPushDownRule ColumnPruningRule和FragmentPruningByPatternRule -> ColumnPruningRule 一系列ConstantFoldingRule -> ConstantFoldingRule 一系列DistinctEliminateRule -> DistinctEliminateRule 具体参考PR文档 https://oxlh5mrwi0.feishu.cn/wiki/OVt5wJxlRip2uQkiFS7c8IHNnzM 此外还删除了规则FilterJoinTransposeRule，这是一力之前写的示例规则，这里不保留了
* feat(sql): GROUP BY expr && ORDER BY expr 1.支持对GROUP BY和ORDER BY中的列使用RowToRow表达式 2.支持GROUP BY和ORDER BY中的列与SELECT子句中的别名进行匹配 Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
支持在filter里使用参数为表达式的函数 更新剩余TPC-H语句准备测试
Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
* fix: align chinese in console table * update test * format
Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
* ci: dbRunner startup only 1 instance in standalone-test * fix influxdb * fix influxdb * fix influxdb * fix influxdb
…inX-THU#523) * fix: find meta_properties_path from path or classpath
重要PR，使得一般情况下的Python库都可以载入
…GinX-THU#493) 将action中原本为打开pushdown和关闭pushdown改为“Optimize全打开”和“Optimzier全关闭”两种测试场景。 现在默认状态下config中的所有rule和enable pushdown都是开启状态。 修改了部分SQLSessionIT逻辑和结果，来适应rule全开启的状态。
* feat(transform): email notification configured by YAML * Update EmailNotifier.java
* 将 restful 端口从 6666 设为 7888 * 将 filesystem 默认端口从 6667 设为 6668 * 重新排序config文件
* test: make email-test more robust * test: make email-test more robust
Fix: When cancelling finished/closed/failed jobs, correct information is given (e.g. job with id: xxx has finished/closed/failed.) Add missed new line in result. When sql execution fails, transform job fails immediately and print correct error message & source in server log. TODO(new pr): On restart, unfinished scheduled transform jobs should be able to start. yaml job files' paths should be specified relative to the client, not the server. provide stopOnFailure switch.
* ci: add option to config timeout-minutes * format
* add ci to check whether py-session has been updated * update pysession --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
RemHero: 之前写在一起的原因是为了减小任务初始化的时间，现在因为初始化时间减小，完全可以把任务并发执行，这样DBCE的代码结构也更加清晰和易于维护，而且action的时间也能减小。同时，把任务并发执行也能减小不同阶段任务之间的影响，之前的逻辑会导致如果最后一个阶段出错需要很长时间才知道fix是否正确。目前这种改动测试逻辑上是等价的。需要注意的点就是，把之前每个阶段执行完后，针对清理每个数据库数据的代码逻辑需要进行相应的调整或者删除，这部分逻辑已经不需要了。 后续的normal IT测试的前提需要清空数据库数据。
[tests added] Add stopOnFailure switch in scheduled jobs. [tests added] Yaml path in sql are now relatived to client/session. [tests added] Restart scheduled jobs when IGinX restarts.
avoid flag label
…GinX-THU#551) * fix path in scripts and pemja version * format * replace `.github/actions/service` with env `SERVICE_DIR` #PR549的关联PR
Co-authored-by: Xu Yihao <48053143+Yihao-Xu@users.noreply.github.com>
为 tpch 添加warmup步骤 重构相关 action tpch 测试中查询语句使用 19a 替代 19，以启用 iotdb12 在下推情况下的测试 使用 19a 代替 19后， 无下推情况下 influxdb 的tpch测试卡死，故将查询并发限制为2
Add tests for python3.13t on: win & mac & linux iotdb & filesystem Note: The testAddStorageEngine test in PySessionIT is skipped because it relies on fastparquet module which does not support python 3.13 yet 顺便解决IoTDB启动时使用了非JDK8导致出错的问题 剩余一个3.13t测试在macOS、FS情况下出错的问题，在PR#555中解决
…" extra params (IGinX-THU#555) LOAD DATA语句支持导入100MB以上的大文件 ADD STORAGEENGINE以及ALTER STORAGEENGINE语句的额外参数的key和value改为用=连接 ——修正macos上IGinX启动的Java环境问题
…config.write.buffer.timeout to 1 seconds (IGinX-THU#556) 修正FileSystem实现中内存对删除的处理而导致的RestIT.testDelete错误 change default value of `data.config.write.buffer.timeout` --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
…#558) * fix(conf): set ColumnPruningRule=off in no-optimizer tests 未过的TPC-H测试将在下一个PR中通过。 --------- Co-authored-by: Yuqing Zhu <yuqing.zhu@139.com>
f2b64e8
into
IGinX-THU:feat/core/replace-rowstream-with-arrow
4 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.