Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enable customizing mydumper options in CRD backup #2407

Merged
merged 16 commits into from
May 11, 2020
Merged

enable customizing mydumper options in CRD backup #2407

merged 16 commits into from
May 11, 2020

Conversation

onlymellb
Copy link
Contributor

@onlymellb onlymellb commented May 8, 2020 •
edited
Loading

What problem does this PR solve?

This PR resolve #2391

What is changed and how does it work?

Support specifying mydumper options in the backup which can be used during backup with mydumper.
The mydumper options can refer to this doc

backup CR example:

---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
  name: demo1-backup-s3
  namespace: test1
spec:
  from:
    host: demo1-tidb
    port: 4000
    user: root
    secretName: backup-demo1-tidb-secret
  s3:
    provider: aws
    bucket: test1-demo1
    region: us-west-2
    secretName: aws-secret
  mydumper:
    options:
    - --threads=16
    - --rows=10000
    - --skip-tz-utc
  storageClassName: local-storage
  storageSize: 1Gi

Check List

Tests

  • Unit test
  • E2E test
  • Stability test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has Go code change

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Does this PR introduce a user-facing change?:

Support customizing `mydumper` options in Backup CR

DanielZhangQD
DanielZhangQD reviewed May 9, 2020
View reviewed changes
cmd/backup-manager/app/util/util.go Outdated
Comment on lines 141 to 153
defaultOptions := []string{
"--long-query-guard=3600",
"--tidb-force-priority=LOW_PRIORITY",
"--verbose=3",
"--compress-protocol",
"--threads=16",
"--rows=10000",
"--skip-tz-utc",
}
defaultTableRegexOptions := []string{
"--regex",
"^(?!(mysql|test|INFORMATION_SCHEMA|PERFORMANCE_SCHEMA|METRICS_SCHEMA|INSPECTION_SCHEMA))",
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move these to the var definitions at the beginning of the file?

cmd/backup-manager/app/util/util.go Show resolved Hide resolved
cmd/backup-manager/app/util/util.go Show resolved Hide resolved
pkg/apis/pingcap/v1alpha1/types.go Outdated Show resolved Hide resolved
luolibin and others added 6 commits May 9, 2020 13:01
DanielZhangQD
DanielZhangQD approved these changes May 9, 2020
View reviewed changes
Copy link
Contributor

@DanielZhangQD DanielZhangQD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DanielZhangQD
Copy link
Contributor

/run-e2e-tests

@onlymellb
Copy link
Contributor Author

/run-e2e-tests

@onlymellb onlymellb requested a review from weekface May 11, 2020 02:52
@onlymellb onlymellb merged commit a5b4482 into pingcap:master May 11, 2020
@onlymellb onlymellb deleted the customizing-mydumper-options branch May 11, 2020 05:33
@cofyc
Copy link
Contributor

cofyc commented May 11, 2020

should this be cherry-picked in 1.1?

@onlymellb onlymellb mentioned this pull request May 12, 2020
Merged
3 tasks
@cofyc
Copy link
Contributor

cofyc commented May 13, 2020

/run-cherry-picker

1 similar comment
@DanielZhangQD
Copy link
Contributor

/run-cherry-picker

sre-bot pushed a commit to sre-bot/tidb-operator that referenced this pull request May 13, 2020
@sre-bot
enable customizing mydumper options in CRD backup (pingcap#2407)
e15e046 
* enable customizing mydumper options in CRD backup

* add unit tests
@sre-bot sre-bot mentioned this pull request May 13, 2020
Merged
@sre-bot
Copy link
Contributor

sre-bot commented May 13, 2020

cherry pick to release-1.1 in PR #2453

sre-bot added a commit that referenced this pull request May 13, 2020
@sre-bot
enable customizing mydumper options in CRD backup (#2407) (#2453)
ad25153
@sre-bot sre-bot mentioned this pull request May 15, 2020
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DanielZhangQD DanielZhangQD DanielZhangQD approved these changes

@weekface weekface weekface approved these changes

@LinuxGit LinuxGit Awaiting requested review from LinuxGit

@shuijing198799 shuijing198799 Awaiting requested review from shuijing198799

Assignees
No one assigned
Labels
needs-cherry-pick-1.1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Enable customizing mydumper options in CRD backup
5 participants
@onlymellb @DanielZhangQD @cofyc @sre-bot @weekface