Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuration of HTTP timeouts for maven repositories #3951

Merged
merged 8 commits into from
Jun 19, 2024

Conversation

ammachado
Copy link
Contributor

@ammachado ammachado commented Jan 27, 2024

What's changed?

Changed maven downloader to allow configuration of connection/read timeouts

What's your motivation?

https://maven.apache.org/guides/mini/guide-http-settings.html#connection-timeouts

Anything in particular you'd like reviewers to focus on?

No

Anyone you would like to review specifically?

No

Have you considered any alternatives or workarounds?

No

Any additional context

N/A

Checklist

  • I've added unit tests to cover both positive and negative cases
  • I've read and applied the recipe conventions and best practices
  • I've used the IntelliJ IDEA auto-formatter on affected files

@ammachado ammachado force-pushed the maven-downloader-timeouts branch from 1c9d5b4 to 976f48c Compare January 27, 2024 05:48
@ammachado ammachado marked this pull request as ready for review January 29, 2024 13:28
@ammachado ammachado force-pushed the maven-downloader-timeouts branch from 976f48c to 3aaf687 Compare January 29, 2024 13:28
Comment on lines +109 to +121
/**
* Constructor required by {@link org.openrewrite.gradle.marker.GradleProject}.
*
* @deprecated Use {@link #MavenRepository(String, String, String, String, boolean, String, String, Duration, Duration, Boolean)}
*/
@Deprecated
@JsonIgnore
public MavenRepository(
@Nullable String id, String uri, @Nullable String releases, @Nullable String snapshots, boolean knownToExist,
@Nullable String username, @Nullable String password, @Nullable Boolean deriveMetadataIfMissing
) {
this(id, uri, releases, snapshots, knownToExist, username, password, null, null, deriveMetadataIfMissing);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used by an external project, org.openrwrite.gradle.tooling:model.

@ammachado ammachado force-pushed the maven-downloader-timeouts branch from 3aaf687 to fef3f86 Compare January 31, 2024 14:10
@timtebeek
Copy link
Contributor

Nice addition @ammachado ! I'm tagging @sambsnyd for an additional review as we're adding fields to tree classes, and he reworked parts of the Maven repository handling recently.

Also copying the details on the failed test to save others a click

MavenPomDownloaderTest > centralIdOverridesDefaultRepository() FAILED
    java.lang.AssertionError: 
    Expecting elements:
      [MavenRepository(id=local, uri=file:///home/runner/.m2/repository, releases=true, snapshots=true, knownToExist=true, username=null, *** connectTimeout=null, readTimeout=null, deriveMetadataIfMissing=false),
        MavenRepository(id=central, uri=https://repo.maven.apache.org/maven2, releases=true, snapshots=false, knownToExist=true, username=null, *** connectTimeout=null, readTimeout=null, deriveMetadataIfMissing=true)]
    to be exactly 1 times URI https://internalartifactrepository.yourorg.com/
        at org.openrewrite.maven.internal.MavenPomDownloaderTest.centralIdOverridesDefaultRepository(MavenPomDownloaderTest.java:115)

@timtebeek timtebeek requested a review from sambsnyd February 1, 2024 09:22
@timtebeek timtebeek added the enhancement New feature or request label Feb 1, 2024
@ammachado
Copy link
Contributor Author

Nice addition @ammachado ! I'm tagging @sambsnyd for an additional review as we're adding fields to tree classes, and he reworked parts of the Maven repository handling recently.

Also copying the details on the failed test to save others a click

MavenPomDownloaderTest > centralIdOverridesDefaultRepository() FAILED
    java.lang.AssertionError: 
    Expecting elements:
      [MavenRepository(id=local, uri=file:///home/runner/.m2/repository, releases=true, snapshots=true, knownToExist=true, username=null, *** connectTimeout=null, readTimeout=null, deriveMetadataIfMissing=false),
        MavenRepository(id=central, uri=https://repo.maven.apache.org/maven2, releases=true, snapshots=false, knownToExist=true, username=null, *** connectTimeout=null, readTimeout=null, deriveMetadataIfMissing=true)]
    to be exactly 1 times URI https://internalartifactrepository.yourorg.com/
        at org.openrewrite.maven.internal.MavenPomDownloaderTest.centralIdOverridesDefaultRepository(MavenPomDownloaderTest.java:115)

As a suggestion, can you consider publishing the test reports on a failed build? Perhaps using a custom action for it (for example, https://github.com/marketplace/actions/junit-report-action).

@timtebeek
Copy link
Contributor

As a suggestion, can you consider publishing the test reports on a failed build? Perhaps using a custom action for it (for example, https://github.com/marketplace/actions/junit-report-action).

Yes I'd been eyeing https://github.com/gradle/actions/tree/main/setup-gradle#build-reporting earlier today, as part of our upgrade in https://github.com/openrewrite/gh-automation/ . It's been a pain point of mine as well to have to go into the logs.

@timtebeek timtebeek self-requested a review February 6, 2024 23:21
@timtebeek timtebeek removed their request for review March 15, 2024 09:37
@timtebeek timtebeek self-requested a review May 4, 2024 17:01
@sambsnyd
Copy link
Member

What's the motivation here? Does this solve a problem you are running into?

@timtebeek timtebeek removed their request for review June 13, 2024 17:17
@cjohnsto88
Copy link

Not my PR but I am watching as I also have an issue that would be solved with this.

During execution the plugin downloads maven-metadata.xml files. My company's internal proxy of Maven Central is very slow for downloads of these files. I cannot get the wider and obvious root cause fixed. The files do download eventually, however in the current implementation there is no way to set the timeout as the custom code to download the maven-metadata.xml files in the plugin does not respect the <configuration> declarations for a <server> specified here.

@ammachado
Copy link
Contributor Author

Not my PR but I am watching as I also have an issue that would be solved with this.

During execution the plugin downloads maven-metadata.xml files. My company's internal proxy of Maven Central is very slow for downloads of these files. I cannot get the wider and obvious root cause fixed. The files do download eventually, however in the current implementation there is no way to set the timeout as the custom code to download the maven-metadata.xml files in the plugin does not respect the <configuration> declarations for a <server> specified here.

That's exactly the issue that originated this change.

@sambsnyd sambsnyd merged commit 52c7703 into openrewrite:main Jun 19, 2024
1 check passed
@timtebeek
Copy link
Contributor

Thanks a lot @ammachado ! Should help to conform to the user's configured session, as we've seen more variation in artifact repository performance than we had expected folks would tolerate in their day to day work. 🙃

Comment on lines +354 to +357
<configuration>
<connectTimeout>40000</connectTimeout>
<readTimeout>50000</readTimeout>
</configuration>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having another look at https://maven.apache.org/guides/mini/guide-http-settings.html#connection-timeouts
Over there it seems there's two options

A. an older <timeout> directly in <configuration>

<settings>
  <servers>
    <server>
      <id>my-server</id>
      <configuration>
        <timeout>6000</timeout> <!-- milliseconds -->
      </configuration>
    </server>
  </servers>
</settings>

B. a newer <connectionTimeout> and <readTimeout> nested inside <httpConfiguration><put>

<settings>
  <servers>
    <server>
      <id>my-server</id>
      <configuration>
        <httpConfiguration>
          <put>
            <connectionTimeout>10000</connectionTimeout> <!-- milliseconds -->
          </put>
        </httpConfiguration>
      </configuration>
    </server>
  </servers>
</settings>

In this PR

We seem to have added connectTimeout (as opposed to connectionTimeout, as well as readTimeout, but neither of those nested in the <httpConfiguration> + <put> elements I'd expect from the Maven docs.

In practice I think these fields won't be set when reading a Maven settings file from disk in style A or B above. Should we make any adjustments here @ammachado ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crankydillo pushed a commit to crankydillo/rewrite that referenced this pull request Jul 11, 2024
…ite#3951)

* Allow configuration of HTTP timeouts for maven repositories

* Applying server connection timeouts to `MavenRepositoryMirror`.

* Reverting unnecessary changes

* Increasing timeouts due to flaky connections

* Reusing repository settings when using profiles

---------

Co-authored-by: Tim te Beek <tim@moderne.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants