Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dm-operator: support discovery dm-master service in current discovery service #3098

Merged
merged 36 commits into from
Aug 19, 2020

Conversation

lichunzhu
Copy link
Contributor

@lichunzhu lichunzhu commented Aug 12, 2020

What problem does this PR solve?

#2868
Support discovery dm-master service in current discovery service

What is changed and how does it work?

  1. add newdm handler
  2. add interface to decode dm master info

Check List

Tests

  • Unit test
  • E2E test

Code changes

  • Has Go code change

Does this PR introduce a user-facing change?:

NONE

@lichunzhu lichunzhu changed the title Discovery support dm dm-operator: make tidb discovery support dm Aug 12, 2020
@lichunzhu lichunzhu added the status/PTAL PR needs to be reviewed label Aug 12, 2020
@lichunzhu lichunzhu changed the title dm-operator: make tidb discovery support dm dm-operator: support discovery dm-master service in current discovery service Aug 12, 2020
@codecov-commenter
Copy link

codecov-commenter commented Aug 12, 2020

Codecov Report

Merging #3098 into master will increase coverage by 0.12%.
The diff coverage is 56.70%.

@@            Coverage Diff             @@
##           master    #3098      +/-   ##
==========================================
+ Coverage   43.12%   43.24%   +0.12%     
==========================================
  Files         156      159       +3     
  Lines       16924    17073     +149     
==========================================
+ Hits         7298     7384      +86     
- Misses       9001     9062      +61     
- Partials      625      627       +2     
Flag Coverage Δ
#unittest 43.24% <56.70%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@@ -44,6 +46,7 @@ func NewServer(pdControl pdapi.PDControlInterface, cli versioned.Interface, kube
func (s *server) registerHandlers() {
ws := new(restful.WebService)
ws.Route(ws.GET("/new/{advertise-peer-url}").To(s.newHandler))
ws.Route(ws.GET("/newdm/{advertise-peer-url}").To(s.newDMHandler))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to add a parameter to /new/{advertise-peer-url} API for dm? e.g. service=dm

newHandler and newDMHandler are similar, we can avoid duplicate code if we can.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 609e797

cofyc
cofyc previously approved these changes Aug 13, 2020
cli: cli,
pdControl: pdControl,
masterControl: masterControl,
clusters: map[string]*clusterInfo{},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialize dmClusters here?

return "", err
}
keyName := fmt.Sprintf("%s/%s", ns, dcName)
// TODO: the replicas should be the total replicas of pd sets.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// TODO: the replicas should be the total replicas of pd sets.
// TODO: the replicas should be the total replicas of dm master set.

pkg/discovery/discovery.go Show resolved Hide resolved
result, err = s.discovery.DiscoverDM(advertisePeerURL)
default:
klog.Errorf("invalid register-type %s", registerType)
if err := resp.WriteError(http.StatusInternalServerError, err); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set err to include the invalid register-type %s message? It is nil here, right?

return nil, fmt.Errorf("unable to list members info from dm-master, err: %s", listMemberResp.Msg)
}
if len(listMemberResp.ListMemberResp) != 1 {
return nil, fmt.Errorf("invalid list members resp: %s", body)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use string(body)?


// MasterClientURL builds the url of master client
func MasterClientURL(namespace Namespace, clusterName string, scheme string) string {
return fmt.Sprintf("%s://%s-master.%s:2379", scheme, clusterName, string(namespace))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2379 should be updated?

}
klog.Infof("dm advertisePeerUrl is: %s", advertisePeerUrl)
strArr := strings.Split(advertisePeerUrl, ".")
if len(strArr) != 4 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IF we require the namespace to be the same as we have checked in L140, we can just take pd-0.pd-peer as the URL to decrease the DNS queries.

Comment on lines 81 to 84
klog.Errorf("invalid register-type %s", registerType)
if err := resp.WriteError(http.StatusInternalServerError, fmt.Errorf("invalid register-type %s", registerType)); err != nil {
klog.Errorf("failed to writeError: %v", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.Errorf("invalid register-type %s", registerType)
if err := resp.WriteError(http.StatusInternalServerError, fmt.Errorf("invalid register-type %s", registerType)); err != nil {
klog.Errorf("failed to writeError: %v", err)
}
err = fmt.Errorf("invalid register-type %s", registerType)
klog.Errorf("%v", err)
if werr := resp.WriteError(http.StatusInternalServerError, err); werr != nil {
klog.Errorf("failed to writeError: %v", werr)
}

DanielZhangQD
DanielZhangQD previously approved these changes Aug 19, 2020
Copy link
Contributor

@DanielZhangQD DanielZhangQD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ti-srebot
ti-srebot previously approved these changes Aug 19, 2020
cofyc
cofyc previously approved these changes Aug 19, 2020
@lichunzhu lichunzhu dismissed stale reviews from cofyc, ti-srebot, and DanielZhangQD via b7b0c31 August 19, 2020 06:38
Copy link
Contributor

@DanielZhangQD DanielZhangQD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DanielZhangQD DanielZhangQD merged commit 10941eb into pingcap:master Aug 19, 2020
@lichunzhu lichunzhu deleted the discoverySupportDM branch August 19, 2020 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/LGT2 status/PTAL PR needs to be reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants