Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch code refactoring #49818

Closed
wants to merge 57 commits into from
Closed

Conversation

dincamihai
Copy link
Contributor

@dincamihai dincamihai commented Sep 28, 2018

What does this PR do?

This started from RFC#0002 but I realized later that it is not required for the implementation of the RFC.
Another PR will address the implementation.

This PR is a gradual refactoring.
For the reviewer: It is easier to review the PR commit by commit to understand how I got to the end result. Thanks!

Tests written?

Not yet

@dincamihai dincamihai requested a review from a team as a code owner September 28, 2018 15:39
@ghost ghost self-requested a review September 28, 2018 15:39
@dincamihai dincamihai changed the title [WIP] Extract Batch arguments preparation to separate functions [WIP] Implementation of RFC#0002 - batch mode publishing Sep 28, 2018
if 'password' in kwargs:
eauth['password'] = kwargs.pop('password')
if 'token' in kwargs:
eauth['token'] = kwargs.pop('token')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you did not wrote this thing and just moved, but let's help it here:

for opt in ['eauth', 'username', 'password', 'token']:
    if opt in kwargs:
        eauth[opt] = kwargs.pop(opt)

if 'gather_job_timeout' in kwargs:
opts['gather_job_timeout'] = kwargs['gather_job_timeout']
if 'batch_wait' in kwargs:
opts['batch_wait'] = int(kwargs['batch_wait'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering why batch_wait needed to be int()'ed here. I'd rather add it to the config, same as timeout etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And also shrink this too in for opt... loop, rather three time if.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@isbm this could all be good ideas but I don't want to incorporate them at this point because I'm trying to do a refactoring here. The refactoring has already a high chance of introducing you bugs and adding additional changes would raise it even more.
Let's try to keep all these ideas for after I am done with the refactoring and let's do them in the following iteration. Agreed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, WP, right. Yeah, we can make it better with the follow-up then.

@rallytime rallytime requested a review from cachedout October 1, 2018 14:14
@cachedout
Copy link
Contributor

Hi @dincamihai Just waiting on one response to feedback from you here and then I think we can move forward with this. Thanks!

@rallytime
Copy link
Contributor

@dincamihai Did you have a chance to come back to this?

@dincamihai
Copy link
Contributor Author

@rallytime I'm working on it. I'll try to respond the questions above now.

@dincamihai
Copy link
Contributor Author

@rallytime just a note
I was keeping this PR as WIP because it is just a small refactoring done as the first step for implementing that async-batch RFC.
I am also fine if you want to merge this now. I hope to produce a follow-up PR soon.

@rallytime
Copy link
Contributor

@dincamihai I think it's nice to iterate on smaller things when we can, so if it's alright with you, let's get this in and you can submit your other work at a later date.

@cachedout
Copy link
Contributor

I think the last thing we need before we can get this in is an addition to the release notes. @dincamihai can you take care of that?

@dincamihai
Copy link
Contributor Author

hello @cachedout sorry for the late reply.
This was just a small internal refactoring without changing the functionality. Does this qualify for an entry in the release notes?

By the way, I have prepared the second part of refactorings:
https://github.com/saltstack/salt/compare/develop...dincamihai:batch-refactoring-part2?expand=1

I have not created a PR yet because I'm not sure how to approach it given it is so big.

@dincamihai dincamihai changed the title [WIP] Implementation of RFC#0002 - batch mode publishing Batch code refactoring Nov 16, 2018
@dincamihai
Copy link
Contributor Author

@cachedout I have pushed those commits as part of this PR since they build on top of the other already included (and not yet merged) commits.

@dincamihai dincamihai mentioned this pull request Nov 21, 2018
@cachedout
Copy link
Contributor

After the tests run this should be ready to go.

@Ch3LL Ch3LL requested a review from dwoz June 20, 2019 13:54
@dwoz
Copy link
Contributor

dwoz commented Dec 27, 2019

@dincamihai This needs to be rebased to the master branch

@waynew
Copy link
Contributor

waynew commented Mar 10, 2020

@dincamihai merges to Salt are also now requiring tests. Are you still interested in driving this to completion?

@waynew
Copy link
Contributor

waynew commented Apr 14, 2020

@dincamihai do you need any help with this? Are you still planning to drive it to completion?

@sagetherage sagetherage added the develop Pointing to develop branch; needs rebase label May 23, 2020
@sagetherage
Copy link
Contributor

@dincamihai looks like we have asked you rebase this against master and added tests. We have not heard from you in some time. We may consider this PR abandoned and close it soon.

@sagetherage sagetherage added this to the Blocked milestone May 26, 2020
@sagetherage
Copy link
Contributor

Reviewed in Grooming 2021-APR-05 considering closing

@sagetherage
Copy link
Contributor

can you please rebase against the master branch? we are no longer taking changes against develop.

Discussed in Grooming 2021-APR-19 have requested rebase with no response, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
develop Pointing to develop branch; needs rebase Reviewers-Assigned
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants