This repository has been archived by the owner on Dec 16, 2022. It is now read-only.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary of build creation -
This PR includes changes from
release-7.0
branch till 8/20.And the following Slack specific patches
Note - Cherrypick keep deletevindexentries for scatter queries is not included since a proper fix has been merged.
Reverts from previous release are not applied.
Vitess 7 General Availability highlights only. (For full list see release notes)-
Incompatible Changes
This release includes the following changes which may result in incompatibilities when upgrading from a previous release. It is important that Vitess components are upgraded in the recommended order. This will change in the next release as documented in VEP-3.
For Slack, we are using
discoverygateway
so it doesn't change anything. We can rollouttabletgateway
after this deploy.For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP
VTTablet has been refactored to be generally more responsive to state changes and its environment: RFC: Change tabletmanager model vitessio/vitess#6131 tm revamp: remove most topo.ChangeType vitessio/vitess#6139 tm revamp: deprecate obsolete code vitessio/vitess#6241 tm revamp: remove calls to refreshTablet vitessio/vitess#6263 tabletmanager revamp: refactor initialization vitessio/vitess#6311 RFC: vttablet breaking changes vitessio/vitess#6345 tabletserver: code simplification vitessio/vitess#6348 vttablet: bring back demote_master_type vitessio/vitess#6384 tabletserver: improved state transitions with new stateManager vitessio/vitess#6396 vttablet: action lock needed to update mysql port vitessio/vitess#6432 vttablet refactor phase 2 vitessio/vitess#6461
Some initialization parameters are now mandatory: init_keyspace, init_shard and init_tablet_type. The flow where you could issue a vtctlclient InitTablet and then invoke vttablet without those init parameters is no longer supported.
For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP. We don't use
enable_replication_watcher
ordisable_active_reparents
For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP
For Slack, it should be a NOOP
Deprecations
The following vtctl commands have been deprecated. Deprecated commands will continue to work until the next release (8.0). Replace slave with replica in vtctl commands and flags vitessio/vitess#6428 RFC: vttablet breaking changes vitessio/vitess#6345 vttablet refactor phase 2 vitessio/vitess#6461
StartSlave -> Replaced with StartReplication
StopSlave -> Replaced with StopReplication
ChangeSlaveType -> Replaced with ChangeTabletType
InitTablet
UpdateTabletAddrs
Various vtctl commands had a flag called "-wait_slave_timeout". It has been replaced with “-wait_replicas_timeout”. Scripts using the old flag will continue to work until the next major release (8.0). Replace slave with replica in vtctl commands and flags vitessio/vitess#6428
Helm charts: The current helm charts have not been keeping up with the forward progress in vitess, and there has been no maintainer for it. Additionally, the operator has pulled ahead of what the helm charts could do. The operator can also be functionally more versatile than the helm charts because custom code can be written to manage complex workflows. To maintain focus and velocity in the project, we are deprecating the helm charts. RFC: Deprecating Helm vitessio/vitess#6439
Consul: The usage of Consul has been problematic for vitess users. The main difficulty comes from the fact that it's non-trivial for a multi-cell deployment. Due to this, we want to discourage the use of consul as a topo server, and are deprecating support for it. RFC: Deprecating Consul vitessio/vitess#6440
Thanks to @setassociative for this report -
---- Slack Changes ----
vitessio#6496 - rafael - Fixes error log
vitessio#6481 - doeg - Update user-facing terminology in the vtctld2 UI
vitessio#6473 - doeg - Add front-end build tooling + fix build error
vitessio#6467 - spark4 - Add more test cases for TestIgnoreMaxPayloadSizeDirective
vitessio#6430 - spark4 - Add query comment to ignore max memory row limit
vitessio#6405 - ajm188 - Update s3 retryer to correctly fallback to the default retryer
vitessio#6395 - ajm188 - Fix segfault in backup retryer
vitessio#6375 - spark4 - Fix unit test
vitessio#6358 - ajm188 - Upgrade consul to 1.8.0 to reconcile k8s addition with consul upgrade
vitessio#6350 - ajm188 - Track BackupEngine and BackupStorageHandle errors together.
vitessio#6143 - spark4 - Add support for query payload limit
vitessio#6282 - rafael - Adds more keyspace, table and tablet_type to logs emitted to files
vitessio#6279 - zmagg - Add more labels to VReplicationSecondsBehindMaster
vitessio#6216 - rafael - Allow empty shards when validating schemas
vitessio#6226 - demmer - mysql rollback transactions should not bump query counts
vitessio#6189 - ajm188 - Upgrade pgzip to v1.2.4
vitessio#6182 - ajm188 - Retry "use of closed network connection" in s3 backup handle
---- Non Slack Changes ----
vitessio#6592 - aquarapid - Trivial copy-pasta comment fixup
vitessio#6583 - harshit-gangal - Backport: Close Idle reserved connections and Rollback Idle transactions: Release 7.0
vitessio#6565 - deepthi - region_sharding: working resharding example
vitessio#6587 - shlomi-noach - vtcompose/docker-compose: fix InitShardMaster
vitessio#6531 - sougou - vtgate routing cleanup: remove routeOptions
vitessio#6584 - shlomi-noach - vtcompose/docker-compose: fix InitShardMaster
vitessio#6552 - harshit-gangal - Close Idle reserved connections and Rollback Idle transactions
vitessio#6569 - timvaillancourt - Add realtime health stats to vtctld /api/keyspace/ks/tablets API
vitessio#6567 - deepthi - tablet_picker: keep trying to find a tablet until context expires
vitessio#6551 - systay - Correctly report AUTOCOMMIT status in network packets
vitessio#6571 - sougou - build: replace gogo proto with golang
vitessio#6576 - rohit-nayak-ps - Vtctl Workflow cmd: don't expect all shards to have workflow streams
vitessio#6580 - systay - improve make proto
vitessio#6572 - harshit-gangal - Forward Port 6554 to 7.0 from 6.0
vitessio#6573 - harshit-gangal - Forward Port 6554 to master from 7.0
vitessio#6556 - rohit-nayak-ps - Reverse workflow: update cells/tablet_types
vitessio#6581 - systay - Backport: Make sure to handle EXPLAIN on vttablet
vitessio#6579 - systay - Make sure to handle EXPLAIN on tablets
vitessio#6504 - systay - Backport: HandleSet in executor to handle multiple settings in single set statement
vitessio#6564 - sougou - tm: minor logging fix
vitessio#6563 - harshit-gangal - Backport of 6522 to release 7.0
vitessio#6522 - harshit-gangal - Remove shard session held in vtgate for reserved connection on connection failure
vitessio#6555 - harshit-gangal - backport vitessio#6514 to 7.0 release branch
vitessio#6505 - systay - Backport: Make sure lookup vindexes are queryable inside transaction
vitessio#6512 - deepthi - Add diagnostic logging to healthcheck.
vitessio#6550 - systay - Make sure to backtick schema names properly
vitessio#6517 - harshit-gangal - Lock Session Support
vitessio#6514 - harshit-gangal - Fix DDL Execution on reserved connection without in active transaction
vitessio#6543 - shlomi-noach - Named columns and convenience SQL query interface
vitessio#6533 - PrismaPhonic - Workflow: Add List-All Command
vitessio#6536 - deepthi - ensure tests for discoverygateway
vitessio#6535 - deepthi - Add diagnostic logging to healthcheck.
vitessio#6515 - sougou - vttablet: Open and Close healthStreamer
vitessio#6477 - Johnny-Three - add skip flag that can skip comparing source & destination schema when run splitdiff
vitessio#6519 - rohit-nayak-ps - VReplication and vstreamer metrics
vitessio#6525 - rohit-nayak-ps - Field event: add additional attributes
vitessio#6526 - aquarapid - Minor vindex fixes:
vitessio#6520 - deepthi - replace more uses of slave with replica/replication
vitessio#6513 - deepthi - 7.0.0 release notes
vitessio#6511 - aquarapid - Convert syslog events to normal log events
vitessio#6499 - systay - Make sure lookup vindexes are queryable inside transaction
vitessio#6490 - sougou - vttablet: create database if not present
vitessio#6509 - artemvovk - Mini: fix tablet hostname
vitessio#6501 - systay - handle boolean settings better
vitessio#6498 - PinZhang - Fixed typo
vitessio#6494 - harshit-gangal - Allow set to have scope for each setting
vitessio#6495 - harshit-gangal - HandleSet in executor to handle multiple settings in single set statement
vitessio#6493 - harshit-gangal - Allow enumeration in set system variables
vitessio#6488 - systay - Settings tweak backport
vitessio#6478 - harshit-gangal - [java] bump java version to 8.0.0-SNAPSHOT for next release
vitessio#6486 - systay - Add more system settings
vitessio#6484 - deepthi - Backport vitessio#6481
vitessio#6479 - harshit-gangal - [java] bump java version to 7.0 for release-7.0
vitessio#6410 - rohit-nayak-ps - Vtctl: Adding VExec and Workflow commands
vitessio#6470 - harshit-gangal - block lock function usage other than dual table
vitessio#6472 - systay - small refactor of SET plan building
vitessio#6461 - sougou - vttablet refactor phase 2
vitessio#6459 - harshit-gangal - System Variables support using Reserved Connections
vitessio#6370 - systay - Support locking functions
vitessio#6465 - rohit-nayak-ps - Vtctl Reshard Example: Added missing =
vitessio#6463 - systay - change how tracing query comments are handled
vitessio#6438 - rohit-nayak-ps - Vrepl Savepoint: handle savepoint during vreplication and in vstream api
vitessio#6448 - rohit-nayak-ps - Schema version table: create _vt db if required
vitessio#6418 - enisoc - examples/operator: Fix port-forward command.
vitessio#6460 - deepthi - Vtexplain: migrate to new healthcheck
vitessio#6454 - rohit-nayak-ps - TestQueryPlanCache: fix flakiness
vitessio#6456 - rohit-nayak-ps - Vreplication Cross-cell source: add support for cells in reshard
vitessio#6462 - arindamnayak - removed error check while verifying the replication
vitessio#6458 - deepthi - Unit test for scatter_conn using new HealthCheck
vitessio#6408 - arindamnayak - Implemented PITR-2
vitessio#6442 - deepthi - Multi-cell tablet picker
vitessio#6451 - systay - Close reserved connections when client hangs up
vitessio#6441 - systay - VTGate side of reserved connections
vitessio#6435 - rohit-nayak-ps - Schema tracking: initial schema insert
vitessio#6428 - deepthi - Replace slave with replica in vtctl commands and flags
vitessio#6445 - systay - Handle non-keyword tokens FORMAT, TREE and VITESS
vitessio#6396 - sougou - tabletserver: improved state transitions with new stateManager
vitessio#6414 - harshit-gangal - Savepoint Construct Implementation in VTGate
vitessio#6431 - harshit-gangal - Replace ExecuteBatch call with Execute in VTGate
vitessio#6437 - aquarapid - The warning code field should be called "Code", not "Type".
vitessio#6432 - sougou - vttablet: action lock needed to update mysql port
vitessio#6434 - harshit-gangal - updated parameter list in test to new method
vitessio#6402 - systay - Statistics for reserved connections
vitessio#6425 - hihihuhu - add a flag to indicate if a vindex handles null value
vitessio#6420 - hihihuhu - batch the lookup query when the column type is integral or binary
vitessio#6412 - harshit-gangal - Beginexecute prequeries
vitessio#6400 - zhijianli88 - Cleanup testing tempfiles
vitessio#6411 - rohit-nayak-ps - schema_version table: change schemax to longblob
vitessio#6413 - mroland91 - fixing log messages
vitessio#6422 - teejae - Materialize: Add copy option to strip constraints from DDL
vitessio#6404 - harshit-gangal - Allow Savepoint on VTTablet
vitessio#6374 - PrismaPhonic - Tablet Manager: Make WaitForPosition() accept either file:pos or GTID-based positions.
vitessio#6423 - rohit-nayak-ps - ReshardingWorkflow: rename interface to address linter suggestion
vitessio#6416 - deepthi - linter fixes, remove staticcheck hook because golangci-lint includes it
vitessio#6415 - sougou - test: faster unit tests
vitessio#6406 - sougou - fix: golangci-lint precheck
vitessio#6399 - rohit-nayak-ps - Schema Version Table: change ddl column to blob
vitessio#6392 - deepthi - rename and deprecate RPCs, rename vars and files
vitessio#6403 - dependabot[bot] - build(deps): bump log4j2.version from 2.13.0 to 2.13.3 in /java
vitessio#6385 - Akilan1999 - Golangci-lint Workflow and hooks ( Excluding handling errors)
vitessio#6351 - harshit-gangal - Add support for Union All
vitessio#6397 - rohit-nayak-ps - VStream Client: send current position events immediately
vitessio#6377 - shlomi-noach - MiniVitess: self contained vitess cluster, automatically resolving and self-configuring to meet external MySQL cluster
vitessio#6313 - harshit-gangal - Reserve Connection: tablet server implementation
vitessio#6233 - inexplicable - exclude
DestinationKeyspaceID,DestinationKeyspaceIDs
from the plan cache key prefixvitessio#6383 - tomkrouper - Add
-rename_tables
to DropSourcesvitessio#6393 - shlomi-noach - make docker_local, docker/local/run.sh utility
vitessio#6384 - sougou - vttablet: bring back demote_master_type
vitessio#6380 - deepthi - delete RPCs that were deprecated in 6.0
vitessio#6379 - morgo - Remove references to slave where possible
vitessio#6371 - aquarapid - Add XOR operator support.
vitessio#6368 - systay - go mod tidy
vitessio#6360 - shlomi-noach - orchestrator repo moved under openark/ org
vitessio#6348 - sougou - tabletserver: code simplification
vitessio#6311 - sougou - tabletmanager revamp: refactor initialization
vitessio#6202 - sverch - Make it easier to test multiple endpoints with aya
vitessio#6364 - Akilan1999 - Fixed lint errors from golangci-lint
vitessio#6363 - deepthi - more dollar sign tests
vitessio#6355 - harshit-gangal - Parsing support for SAVEPOINT, ROLLBACK TO SAVEPOINT, and RELEASE SAVEPOINT Statements
vitessio#6342 - systay - Allow parenthesized selects
vitessio#6357 - sougou - config: fix heartbeat interval regression
vitessio#6353 - carsonoid - Only check for k3s binary on Linux
vitessio#6341 - rohit-nayak-ps - VSCopy Test: Mods/bug fixes/test logging to make test repeatable/debuggable
vitessio#6338 - deepthi - go mod tidy
vitessio#6339 - sougou - tests: deflake TestConsolidation
vitessio#6284 - rohit-nayak-ps - VReplication: allow foreign key constraints during copy
vitessio#6296 - PrismaPhonic - Find Errant GTIDs
vitessio#6328 - systay - Make sure to protect insert lookup queries from bind var name clashes (master)
vitessio#6325 - rohit-nayak-ps - VSCopy Flaky Test: sync event log access, minor cleanup
vitessio#6303 - harshit-gangal - Reserve connection protobuf changes
vitessio#6317 - harshit-gangal - Bump Netty to 4.1.48.Final and tcnative to 2.0.30.Final
vitessio#6314 - harshit-gangal - fix the check of planvalue in changedvindexupdate
vitessio#5680 - kalfonso - Add slim Docker file for functional testing using vttestserver
vitessio#6230 - Hellcatlk - Add unit test for ResolveIPv4Addrs
vitessio#6280 - hs0210 - Add unit test for func stringMapToString.
vitessio#6289 - dweitzman - Fix labels for opentsdb with single-label histograms
vitessio#6298 - luisfmcalado - Fix prepared statements in column specs
vitessio#6232 - rohit-nayak-ps - VStream Copy: streaming vevents from the beginning
vitessio#6310 - nilium - Add "processlist" to non-reserved keywords
vitessio#6309 - aquarapid - Rework error messages on parse errors when reloading table ACL config
vitessio#6283 - rohit-nayak-ps - TestMessage: account for jitter
vitessio#6308 - harshit-gangal - fix udv: return null type for non-existing udv
vitessio#6307 - Hellcatlk - Fix go-lint warning: package comment should be of the form 'Package trace'
vitessio#6304 - systay - Do not rewrite the type description for CAST
vitessio#6301 - systay - Allow charset introducers for vindex columns
vitessio#6261 - deepthi - fix flag docs to be more readable
vitessio#6297 - adsr - Add flag to catch and ignore SIGPIPE
vitessio#6302 - deepthi - vtcombo: use new gateway/healthcheck
vitessio#6299 - systay - Rollback auto-committable transactions when query fails
vitessio#6293 - deepthi - show vitess_tablets should show new master's timestamp after reparenting
vitessio#6263 - sougou - tm revamp: remove calls to refreshTablet
vitessio#6291 - systay - Make it possible to rewrite function calls inside function calls
vitessio#6217 - PrismaPhonic - Add relay log position and file-based positions to SlaveStatus.
vitessio#6249 - aquarapid - Fix issue vitessio#6240. PR vitessio#6130 did not take the header length offset into
vitessio#6278 - sougou - vttablet: fix deadlock in schema engine
vitessio#6276 - enisoc - Rebuild SrvVSchema during InitTablet.
vitessio#6273 - systay - Remove unnecessary comment
vitessio#6275 - deepthi - CI test for region_sharding example
vitessio#6270 - systay - Make multi line inserts safer against bind var collisions
vitessio#6244 - deepthi - Replica transactions
vitessio#6265 - dkhenry - Adding new options to vtcombo to provision mysql
vitessio#6268 - rohit-nayak-ps - Schema Engine Notifier: fix race between schema change broadcasts and…
vitessio#6241 - sougou - tm revamp: deprecate obsolete code
vitessio#6257 - systay - Cleaned up simple stuff around scatter con
vitessio#6259 - rohit-nayak-ps - Reverting breaking change relating to filter regular expressions in vitessio#6242
vitessio#6242 - teejae - Refactor tabletmanager table filtering logic
vitessio#6251 - teejae - Materialize: Use ApplySchema for bulk schema change
vitessio#6256 - systay - Simplify vtgate executor
vitessio#6250 - sougou - yaml: use float64 for durations
vitessio#6253 - Johnny-Three - Optimize Watch_replication_Stream parameter description
vitessio#6252 - systay - Test that vtexplain works with healthcheck
vitessio#6254 - deepthi - fix path to client.go
vitessio#6243 - teejae - Add context to MysqlDaemon schema functions
vitessio#6164 - rohit-nayak-ps - VStreamer: Schema versioning
vitessio#6237 - aquarapid - Allow 'binary' for character_set_results; it's equivalent to null, which
vitessio#6239 - systay - Add DO statements
vitessio#6245 - enisoc - mysqlctl: Fix connection leak in edge case of killConnection().
vitessio#6248 - deepthi - vtexplain: use DiscoveryGateway with LegacyHealthCheck
vitessio#6209 - deepthi - region sharding example
vitessio#6215 - inexplicable - SIGHUP support for tls config update
vitessio#6139 - sougou - tm revamp: remove most topo.ChangeType
vitessio#6155 - deepthi - Healthcheck: new simpler implementation
vitessio#4696 - dweitzman - Disable foreign key checks when preparing existing schema for preflight schema change tests
vitessio#6225 - harshit-gangal - Support uppercase user defined variables
vitessio#6224 - systay - For default() - make sure to escape columns when necessary
vitessio#6222 - sougou - vindexes: ignore_nulls option for lookups
vitessio#6176 - systay - Refactoring of transaction handling in tabletserver
vitessio#6207 - teejae - Vtctl Materialize optimizations
vitessio#6201 - rohit-nayak-ps - Ext Connector Test: Fix race conditions while expecting queries
vitessio#6204 - systay - Small fixes
vitessio#6205 - adsr - Fix panic with
select database()
in OLAP mode (vitessio#6200)vitessio#6203 - harshit-gangal - Get cached vschema from executor
vitessio#6192 - systay - Add tests for SHOW COLUMNS
vitessio#6187 - aquarapid - Update CLI help with some missing flags and correct SwitchWrites
vitessio#6186 - systay - Fix tokenizer to not choke on special comments
vitessio#6184 - enisoc - go/mysql: Fix MariadbGTIDSet multi-domain support.
vitessio#6185 - enisoc - go/mysql: Fix TLS test for Go 1.13+.
vitessio#6179 - harshit-gangal - Add support for Mysql row_count() function
vitessio#6154 - sougou - examples: using planetscale's vitess-operator
vitessio#6178 - morgo - Change local example to use DropSources
vitessio#6175 - systay - Added fuzzer
vitessio#5189 - aquarapid - Allow sending back stale shard name data if requested,…
vitessio#6137 - aquarapid - Add new debug handlers to set block and mutex profiling
vitessio#6169 - teejae - Change all external links in vitess to use https
vitessio#6167 - harshit-gangal - enable set statement test
vitessio#6168 - sougou - helm: simplified for examples
vitessio#6151 - Johnny-Three - support drop vindex
vitessio#6163 - harshit-gangal - make setsysvar map for first time
vitessio#6159 - morgo - Fix regression in release script
vitessio#6152 - systay - Added planning logic for IN and NOT IN when values are null
vitessio#6157 - harshit-gangal - Fix all com_init_db and mysql client related issue with use
vitessio#6149 - harshit-gangal - Building Block for Set Passthrough in VTGate
vitessio#6150 - systay - Simplified code
vitessio#6103 - sougou - vrepl: import from multiple external sources
vitessio#6146 - systay - Use 'SelectNone' for all equality against null
vitessio#6145 - harshit-gangal - Fix owned table name formatting and duplicate error code in consistent lookup
vitessio#6138 - derekperkins - vitessdriver: allow overriding the driver name
vitessio#6135 - deepthi - print MasterTermStartTime in ListAllTablets and
show vitess_tablets
vitessio#6140 - systay - Make sure to handle types no matter casing
vitessio#6132 - harshit-gangal - Implements Send Primitive's GetFields
vitessio#6136 - sougou - parser: allow '$' in identifiers
vitessio#6133 - systay - Create better plans for NULL comparisons
vitessio#6130 - keyurdg - Reduce the number of packets sent from vttablet to MySQL
vitessio#6128 - zhangzhihui1 - etcd2topo/server_test: add err check
vitessio#6120 - Hellcatlk - Add unit test for function in go/sync2/atomic.go
vitessio#6110 - systay - Add EXPLAIN format for Vitess plans
vitessio#6127 - aquarapid - Replace Sprintf by string concat in normalizer, reduces