YDB CLI 2.6.0

Released on September 7, 2023. To update to version 2.6.0, select the Downloads section.

Features:

• Added --path option to ydb workload tpch run, which contains the path to the directory with tables created by the ydb workload tpch init command.

• Added ydb workload transfer topic-to-table run command, which loads the database with read requests from topics and write requests to the table.

• Added the option --consumer-prefix in the commands ydb workload topic init, ydb workload topic run read|full, specifying prefixes of consumer names.

• Added the --partition-ids option in the ydb topic read command, which specifies a comma-separated list of topic partition identifiers to read from.

• Added support for CSV and TSV parameter formats in YQL query execution commands.

• The interactive mode of query execution has been redesigned. Added new interactive mode specific commands: SET, EXPLAIN, EXPLAIN AST. Added saving history between CLI launches and auto-completion of YQL queries.

• Added the command ydb config info, which outputs the current connection parameters without connecting to the database.

• Added the command ydb workload kv run mixed, which loads the database with write and read requests.

• The --percentile option in the ydb workload topic run write|read|full commands can now take floating point values.

• The default values for the --seconds and --warmup options in the ydb workload topic run write|read|full commands have been increased to 60 seconds and 5 seconds, respectively.

• Changed the default value for the --supported-codecs option to RAW in the ydb topic create and ydb topic consumer add commands.

Bug fixes:

• Fixed string loss when loading with the ydb import file json command.

• Fixed ignored statistics during the warm-up of commands ydb workload topic run write|read|full.

• Fixed incomplete statistics output in the ydb scripting yql and ydb yql commands.

• Fixed incorrect output of progress bar in ydb tools dump and ydb tools restore commands.

• Fixed loading large files with the header in the ydb import file csv|tsv command.

• Fixed hanging of the ydb tools restore --import-data command.

• Fixed error Unknown value Rejected when executing the ydb operation list build index command.

Full Changelog: CLI_2.5.0...CLI_2.6.0

YDB Server 23.2.12

Functionality:

• (Experimental) Implemented visibility of own changes. With this feature enabled you can read changed values from the current transaction, which has not been committed yet. This functionality also allows multiple modifying operations in one transaction on a table with secondary indexes. To enable this feature add enable_kqp_immediate_effects: true under table_service_config section into configuration file.
• (Experimental) Implemented read iterators. This feature allows to separate reads and computations. Read iterators allow datashards to increase read queries throughput. To enable this feature add enable_kqp_data_query_source_read: true under table_service_config section into configuration file.

Embedded UI:

• Navigation improvements:
  • Diagnostics and Development mode switches are moved to the left panel.
  • Every page has breadcrumbs.
  • Storage groups and nodes info are moved from left buttons to tabs on the database page.
• Query history and saved queries are moved to tabs over the text editor area in query editor.
• Info tab for scheme objects displays parameters using terms from CREATE or ALTER statements.
• Added column tables support.

Performance:

• For scan queries, you can now effectively search for individual rows using a primary key or secondary indexes. This can bring you a substantial performance gain in many cases. Similarly to regular queries, you need to explicitly specify its name in the query text using the VIEW keyword to use a secondary index.
• (Experimental) Added an option to give control of the system tablets of the database (SchemeShard, Coordinators, Mediators, SysViewProcessor) to its own Hive instead of the root Hive, and do so immediately upon creating a new database. Without this flag, the system tablets of the new database are created in the root Hive, which can negatively impact its load. Enabling this flag makes databases completely isolated in terms of load, that may be particularly relevant for installations, consisting from a roughly hundred or more databases. To enable this feature add alter_database_create_hive_first: true under feature_flags section into configuration file.

Bug fixes:

• Fixed a bug in the autoconfiguration of the actor system, resulting in all the load being placed on the system pool.
• Fixed a bug that caused full scanning when searching by prefix of the primary key using LIKE.
• Fixed bugs when interacting with datashard followers.
• Fixed bugs when working with memory in column tables.
• Fixed a bug in processing conditions for immediate transactions.
• Fixed a bug in the operation of iterator-based reads on datasharrd followers.
• Fixed a bug that caused cascading reinstallation of data delivery sessions to asynchronous indexes.
• Fixed bugs in the optimizer for scanning queries.
• Fixed a bug in the incorrect calculation of storage consumption by Hive after expanding the database.
• Fixed a bug that caused operations to hang on non-existent iterators.
• Fixed bugs when reading a range on a NOT NULL column.
• Fixed a bug in the replication of VDisks.
• Fixed a bug in the work of the run_interval option in TTL.

Full Changelog: 23.1.26...23.2.12

YDB CLI 2.5.0

Released on June 20, 2023. To update to version 2.5.0, select the Downloads section.

Features

• For the ydb import file command, a parameter --timeout has been added that specifies the time within which the operation should be performed on the server.
• Added a progress bar in commands ydb scheme rmdir --recursive and ydb import file.
• Added the command ydb workload kv run read-rows, which loads the database with requests to read rows using a new experimental API call ReadRows (implemented only in the main branch), which performs faster key reading than select.
• New parameters --warmup-time, --percentile, --topic have been added to the ydb workload topic, setting the test warm-up time, the percentile in the statistics output and the topic name, respectively.
• Added the ydb workload tpch command to run the TPC-H benchmark.
• Added the --ordered flag in the command ydb tools dump, which preserves the order by primary key in tables.

Performance

• The data loading speed in the ydb import file command has been increased by adding parallel loading. The number of threads is set by the new parameter --threads.
• A performance of the ydb import file json command has been increased by reducing the number of data copies.

Full Changelog: CLI_2.4.0...CLI_2.5.0

YDB CLI 2.4.0

Released on May 24, 2023. To update to version 2.4.0, select the Downloads section.

Features

• Added the ability to upload multiple files in parallel with the command ydb import file.
• Added support for deleting column tables for the command ydb scheme rmdir --recursive.
• Improved stability of the command ydb workload topic.

Full Changelog: CLI_2.3.0...CLI_2.4.0

YDB Server 23.1.26

Functionality:

• Added initial table scan when creating a CDC changefeed. Now, you can export all the data existing at the time of changefeed creation.
• Added atomic index replacement. Now, you can atomically replace one pre-defined index with another. This operation is absolutely transparent for your application. Indexes are replaced seamlessly, with no downtime.
• Added the audit log: Event stream including data about all the operations on YDB objects.

Performance:

• Improved formats of data exchanged between query stages. As a result, we accelerated SELECTs by 10% on parameterized queries and by up to 30% on write operations.
• Added autoconfiguring for the actor system pools based on the workload against them. This improves performance through more effective CPU sharing.
• Optimized the predicate logic: Processing of parameterized OR or IN constraints is automatically delegated to DataShard.
• (Experimental) For scan queries, you can now effectively search for individual rows using a primary key or secondary indexes. This can bring you a substantial gain in performance in many cases. Similarly to regular queries, to use a secondary index, you need to explicitly specify its name in the query text using the VIEW keyword.
• The query's computational graph is now cached at query runtime, reducing the CPU resources needed to build the graph.

Bug fixes:

• Fixed bugs in the distributed data warehouse implementation. We strongly recommend all our users to upgrade to the latest version.
• Fixed the error that occurred on building an index on NOT NULL columns.
• Fixed statistics calculation with MVCC enabled.
• Fixed errors with backups.
• Fixed the race condition that occurred at splitting and deleting a table with SDC.

Full Changelog: 22.5.10...23.1.26

YDB CLI 2.3.0

Released on May 5, 2023. To update to version 2.3.0, select the Downloads section.

What's new:

• Added interactive query execution mode. It can be launched using ydb yql command without arguments. This mode is experimental and is a subject to change.
• Added ydb table index rename command for atomic secondary index replacement or renaming.
• Added ydb workload topic command section that allows to run a workload of writes and reads to topics.
• Added --recursive option for ydb scheme rmdir command that allows to remove a directory recursively with all its content.
• Added topic and coordination node support for ydb scheme describe command.
• Added --commit option for ydb topic consumer command to commit offset for consumer.
• Added --columns option for ydb import file csv|tsv command to list column names in, instead of placing it into file header.
• Added --newline-delimited option for ydb import file csv|tsv command that confirms that there is no newline characters inside records which allows to read from several sections of a file simultaneously.

Bug fixes:

• Fixed a bug that caused executing the ydb import file command to consume too much memory and CPU.

Full Changelog: CLI_2.2.0...CLI_2.3.0

YDB CLI 2.2.0

Released on March 3, 2023. To update to version 2.2.0, select the Downloads section.

What's new:

• Fixed the error that didn't allow specifying supported compression algorithms when adding a topic consumer.
• Added support for streaming YQL scripts and queries based on parameters transferred via stdin.
• YQL query parameter values can now be transferred from a file.
• Password input requests are now output to stderr instead of stdout.
• You can now save the root CA certificate path in a profile.
• Added a global parameter named --profile-file to use the specified file as storage for profile settings.
• Added a new type of load testing: ydb workload clickbench.

Full Changelog: CLI_2.1.1...CLI_2.2.0

YDB Server 22.5.10

What's new:

• Added changefeed configuration settings to transfer additional info about changes to topic.
• Added support of rename for tables with TTL setting enabled.
• Added retention period setting for cdc-log.

BugFixes & Improvements:

• Fixed: Error inserting 0 rows via BulkUpsert.
• Fixed: Error when importing Date/DateTime columns from CSV.
• Fixed: Error importing data from CSV with line breaks in values.
• Fixed: Error importing data from CSV with empty values.
• Improved Query Processing performance (WorkerActor was replaced with SessionActor).
• DataShard compaction now starts immediately after split or merge operations.

Full Changelog: 22.4.44...22.5.10

YDB CLI 2.1.1

Released on December 30, 2022. To update to version 2.1.1, select the Downloads section.

Improvements:

• Added support for the --stats parameter of the ydb scheme describe command for column-oriented tables.
• Added support for Parquet files to enable their import with the ydb import command.
• Added support for additional logging and retries for the ydb import command.

Full Changelog: CLI_2.1.0...CLI_2.1.1

YDB Server 22.4.44

• Minor bugfix and improvements
• Fixed security issue (CVE-2022-28228)

Full Changelog: 22.4.31...22.4.44