Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extensible background build discarders #4368

Merged
merged 12 commits into from
Feb 14, 2020
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.Extension;
import hudson.model.AsyncPeriodicWork;
import hudson.model.Job;
import hudson.model.TaskListener;
import org.kohsuke.accmod.Restricted;
import org.kohsuke.accmod.restrictions.NoExternalUse;

import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
* Background task actually running background build discarders.
*
* @see GlobalBuildDiscarderConfiguration
* @see GlobalBuildDiscarderStrategy
*/
@Restricted(NoExternalUse.class)
@Extension
public class BackgroundGlobalBuildDiscarder extends AsyncPeriodicWork {
private static final Logger LOGGER = Logger.getLogger(BackgroundGlobalBuildDiscarder.class.getName());

public BackgroundGlobalBuildDiscarder() {
super("Periodic background build discarder"); // TODO i18n
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the TODO required for merge?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. I'm not sure we're even localizing these.

}

@Override
protected void execute(TaskListener listener) throws IOException, InterruptedException {
for (Job job : Jenkins.get().allItems(Job.class)) {
processJob(listener, job);
}
}

public static void processJob(TaskListener listener, Job job) {
listener.getLogger().println("Processing " + job.getFullName());
GlobalBuildDiscarderConfiguration.get().getConfiguredBuildDiscarders().forEach(strategy -> {
String displayName = strategy.getDescriptor().getDisplayName();
listener.getLogger().println("Offering " + job.getFullName() + " to " + displayName);
if (strategy.isApplicable(job)) {
listener.getLogger().println(job.getFullName() + " accepted by " + displayName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly concerned that this might be excessive logging on very large instances

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but it's all going to a rotated set of log files, so will never fill up a disk (except when it's tiny to begin with):

if (f.isFile()) {
if ((lastRotateMillis + logRotateMillis < System.currentTimeMillis())
|| (logRotateSize > 0 && f.length() > logRotateSize)) {
lastRotateMillis = System.currentTimeMillis();
File prev = null;
for (int i = 5; i >= 0; i--) {
File curr = i == 0 ? f : new File(f.getParentFile(), f.getName() + "." + i);
if (curr.isFile()) {
if (prev != null && !prev.exists()) {
if (!curr.renameTo(prev)) {
logger.log(getErrorLoggingLevel(), "Could not rotate log files {0} to {1}",
new Object[]{curr, prev});
}
} else {
if (!curr.delete()) {
logger.log(getErrorLoggingLevel(), "Could not delete log file {0} to enable rotation",
curr);
}
}
}
prev = curr;
}
}

try {
strategy.apply(job);
} catch (Exception ex) {
listener.error("An exception occurred when executing " + displayName + ": " + ex.getMessage());
LOGGER.log(Level.WARNING, "An exception occurred when executing " + displayName, ex);
}
}
});
}

@Override
public long getRecurrencePeriod() {
return HOUR;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.Extension;
import hudson.ExtensionList;
import hudson.util.DescribableList;
import net.sf.json.JSONObject;
import org.jenkinsci.Symbol;
import org.kohsuke.accmod.Restricted;
import org.kohsuke.accmod.restrictions.NoExternalUse;
import org.kohsuke.stapler.StaplerRequest;

import java.io.IOException;
import java.util.Collections;

/**
* Global configuration UI for background build discarders
*
* @see GlobalBuildDiscarderStrategy
* @see BackgroundGlobalBuildDiscarder
*/
@Restricted(NoExternalUse.class)
@Extension @Symbol("buildDiscarders")
public class GlobalBuildDiscarderConfiguration extends GlobalConfiguration {
public static GlobalBuildDiscarderConfiguration get() {
return ExtensionList.lookupSingleton(GlobalBuildDiscarderConfiguration.class);
}

private final DescribableList<GlobalBuildDiscarderStrategy, GlobalBuildDiscarderStrategyDescriptor> configuredBuildDiscarders =
new DescribableList<>(this, Collections.singletonList(new JobGlobalBuildDiscarderStrategy()));

private Object readResolve() {
configuredBuildDiscarders.setOwner(this);
return this;
}

public DescribableList<GlobalBuildDiscarderStrategy, GlobalBuildDiscarderStrategyDescriptor> getConfiguredBuildDiscarders() {
return configuredBuildDiscarders;
}

@Override
public boolean configure(StaplerRequest req, JSONObject json) throws FormException {
try {
configuredBuildDiscarders.rebuildHetero(req, json, GlobalBuildDiscarderStrategyDescriptor.all(), "configuredBuildDiscarders");
return true;
} catch (IOException x) {
throw new FormException(x, "configuredBuildDiscarders");
}
}
}
51 changes: 51 additions & 0 deletions core/src/main/java/jenkins/model/GlobalBuildDiscarderListener.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.Extension;
import hudson.model.Job;
import hudson.model.Run;
import hudson.model.listeners.RunListener;
import hudson.util.LogTaskListener;
import org.kohsuke.accmod.Restricted;
import org.kohsuke.accmod.restrictions.NoExternalUse;

import java.util.logging.Level;
import java.util.logging.Logger;

/**
* Run background build discarders on an individual job once a build is finalized
*/
@Extension
@Restricted(NoExternalUse.class)
public class GlobalBuildDiscarderListener extends RunListener<Run> {

private static final Logger LOGGER = Logger.getLogger(GlobalBuildDiscarderListener.class.getName());

@Override
public void onFinalized(Run run) {
Job job = run.getParent();
BackgroundGlobalBuildDiscarder.processJob(new LogTaskListener(LOGGER, Level.FINE), job);
}
}
81 changes: 81 additions & 0 deletions core/src/main/java/jenkins/model/GlobalBuildDiscarderStrategy.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.ExtensionPoint;
import hudson.model.AbstractDescribableImpl;
import hudson.model.Job;
import hudson.model.Run;

import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
* Extension point for global background build discarders.
*
* @see BackgroundGlobalBuildDiscarder
* @see GlobalBuildDiscarderConfiguration
* @see JobGlobalBuildDiscarderStrategy
*/
public abstract class GlobalBuildDiscarderStrategy extends AbstractDescribableImpl<GlobalBuildDiscarderStrategy> implements ExtensionPoint {
private static final Logger LOGGER = Logger.getLogger(GlobalBuildDiscarderStrategy.class.getName());

/**
* Returns true if and only if this strategy applies to the given job.
* @param job
* @return true if and only if this strategy applies to the given job.
*/
public abstract boolean isApplicable(Job<?, ?> job);

/**
* Applies this build discarder strategy to the given job, i.e. delete builds based on this strategy's configuration.
*
* The default implementation calls {@link #apply(Run)} on each build.
*
* @param job
* @throws IOException
* @throws InterruptedException
*/
public void apply(Job<? extends Job, ? extends Run> job) throws IOException, InterruptedException {
for (Run<? extends Job, ? extends Run> run : job.getBuilds()) {
try {
apply(run);
} catch (IOException ex) {
LOGGER.log(Level.WARNING, "Failed to delete " + run.getFullDisplayName(), ex);
}
}
}

/**
* Applies this build discarder strategy to the given run, i.e. delete builds based on this strategy's configuration.
*
* @param run
* @throws IOException
* @throws InterruptedException
*/
public void apply(Run<?, ?> run) throws IOException, InterruptedException {
// no-op by default
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.DescriptorExtensionList;
import hudson.model.Descriptor;

/**
* {@link Descriptor} for {@link BackgroundGlobalBuildDiscarder}.
*/
public abstract class GlobalBuildDiscarderStrategyDescriptor extends Descriptor<GlobalBuildDiscarderStrategy> {

public static DescriptorExtensionList<GlobalBuildDiscarderStrategy, GlobalBuildDiscarderStrategyDescriptor> all() {
return Jenkins.get().getDescriptorList(GlobalBuildDiscarderStrategy.class);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* The MIT License
*
* Copyright 2019 Daniel Beck
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/
package jenkins.model;

import hudson.Extension;
import hudson.model.Job;
import org.jenkinsci.Symbol;
import org.kohsuke.accmod.Restricted;
import org.kohsuke.accmod.restrictions.NoExternalUse;
import org.kohsuke.stapler.DataBoundConstructor;

import javax.annotation.Nonnull;
import java.io.IOException;

/**
* Periodically call a job's configured build discarder in the background.
*/
@Restricted(NoExternalUse.class)
public class JobGlobalBuildDiscarderStrategy extends GlobalBuildDiscarderStrategy {

@DataBoundConstructor
public JobGlobalBuildDiscarderStrategy() {
// required for data binding
}

@Override
public boolean isApplicable(Job<?, ?> job) {
return job.getBuildDiscarder() != null;
}

@Override
public void apply(Job<?, ?> job) throws IOException, InterruptedException {
job.logRotate();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Job is a non-final, public, abstract class, and logRotate() is a non-final, public method on Job - and neither are annotated with @NoExternalUse.

It is not guaranteed that job.logRotate() will actually use the build discarder that was checked in isApplicable(...). This may cause the documentation around this step to fail to align with what actually happens.

Is there a reason to prefer job.logRotate() over job.getBuildDiscarder().perform( job )?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I think the expectation is that #logRotate() not be overridden, but it's not enforced, true.

I am unsure which is the more correct option.

#logRotate() is called by Run#execute(RunExecution), so an argument can be made that this is the expected way to rotate logs, and if the job overrides it in a way that differs from what getBuildDiscarder().perform(…) does, #logRotate() should be take precedence.

OTOH, the way this feature is explained on the UI would indicate we want getBuildDiscarder().perform(…) instead. Obviously, much of that is by necessity.

}

@Extension @Symbol("jobBuildDiscarder")
public static class DescriptorImpl extends GlobalBuildDiscarderStrategyDescriptor {
@Nonnull
@Override
public String getDisplayName() {
return Messages.JobGlobalBuildDiscarderStrategy_displayName();
}
}
}
Loading