Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync pd and tikv configmap in controller #1330

Merged
merged 2 commits into from
Dec 13, 2019
Merged

sync pd and tikv configmap in controller #1330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
98c61ee
Sync pd and tikv configmap in controller
aylei Dec 11, 2019
a5820bf
Merge branch 'master' into fix-1263
aylei Dec 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion manifests/crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -923,7 +923,11 @@ spec:
target store.
format: int64
type: integer
max-store-down-time: {}
max-store-down-time:
description: MaxStoreDownTime is the max duration after
which a store will be considered to be down if it hasn't
reported heartbeats.
type: string
merge-schedule-limit:
description: MergeScheduleLimit is the max coexist merge
schedules.
Expand Down Expand Up @@ -994,6 +998,8 @@ spec:
description: TsoSaveInterval is the interval to save timestamp.
type: string
type: object
configUpdateStrategy:
type: string
replicas:
format: int32
type: integer
Expand Down Expand Up @@ -2192,6 +2198,8 @@ spec:
type: integer
type: object
type: object
configUpdateStrategy:
type: string
maxFailoverCount:
format: int32
type: integer
Expand Down
17 changes: 15 additions & 2 deletions pkg/apis/pingcap/v1alpha1/openapi_generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

52 changes: 3 additions & 49 deletions pkg/apis/pingcap/v1alpha1/pd_config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@
package v1alpha1

import (
"fmt"
"strconv"
"time"

"github.com/pingcap/log"
"github.com/pingcap/pd/pkg/metricutil"
"github.com/pingcap/pd/pkg/typeutil"
Expand Down Expand Up @@ -101,13 +97,13 @@ type PDConfig struct {
Replication *PDReplicationConfig `toml:"replication,omitempty" json:"replication,omitempty"`

// +optional
Namespace map[string]PDNamespaceConfig `json:"namespace,omitempty"`
Namespace map[string]PDNamespaceConfig `toml:"namespace,omitempty" json:"namespace,omitempty"`

// +optional
PDServerCfg *PDServerConfig `toml:"pd-server,omitempty" json:"pd-server,omitempty"`

// +optional
ClusterVersion string `json:"cluster-version,omitempty"`
ClusterVersion string `toml:"cluster-version,omitempty" json:"cluster-version,omitempty"`

// QuotaBackendBytes Raise alarms when backend size exceeds the given quota. 0 means use the default quota.
// the default size is 2GB, the maximum is 8GB.
Expand Down Expand Up @@ -250,7 +246,7 @@ type PDScheduleConfig struct {
// MaxStoreDownTime is the max duration after which
// a store will be considered to be down if it hasn't reported heartbeats.
// +optional
MaxStoreDownTime Duration `toml:"max-store-down-time,omitempty" json:"max-store-down-time,omitempty"`
MaxStoreDownTime string `toml:"max-store-down-time,omitempty" json:"max-store-down-time,omitempty"`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BurntSushi/toml do not respect MarshalText, use string instead

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BurntSushi/toml seems not maintained anymore...

we can use https://github.com/pelletier/go-toml and implement toml.MarshalTOML() (like MarshalJSON())

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://play.golang.org/p/_SfokqsaCPo it works

Copy link
Contributor Author

@aylei aylei Dec 13, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's exactly what I've thought.

Actually I've tried go-toml( and naoina/toml), it turns out that go-toml does not support interface{} (necessary for schemaless configuration, e.g. Pump). Also, PD and TiDB uses BurntSushi/toml so there is little verifying work, seems like that BurntSushi/toml is not an ideal but the only choice

// LeaderScheduleLimit is the max coexist leader schedules.
// +optional
LeaderScheduleLimit *uint64 `toml:"leader-schedule-limit,omitempty" json:"leader-schedule-limit,omitempty"`
Expand Down Expand Up @@ -369,45 +365,3 @@ type PDServerConfig struct {
// +optional
UseRegionStorage *bool `toml:"use-region-storage,omitempty" json:"use-region-storage,string,omitempty"`
}

// Duration is a wrapper of time.Duration for TOML and JSON.
// Copied from pingcap typeutil, add marshal to TOML support
type Duration struct {
time.Duration
}

// NewDuration creates a Duration from time.Duration.
func NewDuration(duration time.Duration) Duration {
return Duration{Duration: duration}
}

// MarshalJSON returns the duration as a JSON string.
func (d *Duration) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`"%s"`, d.String())), nil
}

// UnmarshalJSON parses a JSON string into the duration.
func (d *Duration) UnmarshalJSON(text []byte) error {
s, err := strconv.Unquote(string(text))
if err != nil {
return err
}
duration, err := time.ParseDuration(s)
if err != nil {
return err
}
d.Duration = duration
return nil
}

// UnmarshalText parses a TOML string into the duration.
func (d *Duration) UnmarshalText(text []byte) error {
var err error
d.Duration, err = time.ParseDuration(string(text))
return err
}

// MarshalText marshal duration to a TOML string
func (d *Duration) MarshalText() ([]byte, error) {
return []byte(fmt.Sprintf(`"%s"`, d.String())), nil
}
16 changes: 16 additions & 0 deletions pkg/apis/pingcap/v1alpha1/tidbcluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,22 @@ func (tc *TidbCluster) TiDBConfigUpdateStrategy() ConfigUpdateStrategy {
return s
}

func (tc *TidbCluster) PDConfigUpdateStrategy() ConfigUpdateStrategy {
s := tc.Spec.PD.ConfigUpdateStrategy
if string(s) == "" {
s = ConfigUpdateStrategyInPlace
}
return s
}

func (tc *TidbCluster) TiKVConfigUpdateStrategy() ConfigUpdateStrategy {
s := tc.Spec.TiKV.ConfigUpdateStrategy
if string(s) == "" {
s = ConfigUpdateStrategyInPlace
}
return s
}

func (tc *TidbCluster) PDIsAvailable() bool {
lowerLimit := tc.Spec.PD.Replicas/2 + 1
if int32(len(tc.Status.PD.Members)) < lowerLimit {
Expand Down
6 changes: 6 additions & 0 deletions pkg/apis/pingcap/v1alpha1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,9 @@ type PDSpec struct {
Service *ServiceSpec `json:"service,omitempty"`
StorageClassName string `json:"storageClassName,omitempty"`

// +optional
ConfigUpdateStrategy ConfigUpdateStrategy `json:"configUpdateStrategy,omitempty"`

// Config is the Configuration of pd-servers
Config *PDConfig `json:"config,omitempty"`
}
Expand All @@ -200,6 +203,9 @@ type TiKVSpec struct {
StorageClassName string `json:"storageClassName,omitempty"`
MaxFailoverCount int32 `json:"maxFailoverCount,omitempty"`

// +optional
ConfigUpdateStrategy ConfigUpdateStrategy `json:"configUpdateStrategy,omitempty"`

// Config is the Configuration of tikv-servers
Config *TiKVConfig `json:"config,omitempty"`
}
Expand Down
17 changes: 0 additions & 17 deletions pkg/apis/pingcap/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions pkg/controller/tidbcluster/tidb_cluster_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ func NewController(
svcControl,
podControl,
certControl,
typedControl,
setInformer.Lister(),
svcInformer.Lister(),
podInformer.Lister(),
Expand All @@ -140,6 +141,7 @@ func NewController(
setControl,
svcControl,
certControl,
typedControl,
setInformer.Lister(),
svcInformer.Lister(),
podInformer.Lister(),
Expand Down
Loading