Skip to content

Commit

Permalink
[MNGSITE-438] Add guide for Large Scale Centralized Deployments
Browse files Browse the repository at this point in the history
This closes #223
  • Loading branch information
philsttr authored and michael-o committed Jan 1, 2021
1 parent f4d6829 commit 0b33ee0
Show file tree
Hide file tree
Showing 3 changed files with 246 additions and 0 deletions.
243 changes: 243 additions & 0 deletions content/apt/guides/mini/guide-large-scale-centralized-deployments.apt
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
------
Guide to Large Scale Centralized Deployments
------
Phil Clay
------
2021-01-01
------

~~ Licensed to the Apache Software Foundation (ASF) under one
~~ or more contributor license agreements. See the NOTICE file
~~ distributed with this work for additional information
~~ regarding copyright ownership. The ASF licenses this file
~~ to you under the Apache License, Version 2.0 (the
~~ "License"); you may not use this file except in compliance
~~ with the License. You may obtain a copy of the License at
~~
~~ http://www.apache.org/licenses/LICENSE-2.0
~~
~~ Unless required by applicable law or agreed to in writing,
~~ software distributed under the License is distributed on an
~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~~ KIND, either express or implied. See the License for the
~~ specific language governing permissions and limitations
~~ under the License.

~~ NOTE: For help with the syntax of this file, see:
~~ http://maven.apache.org/doxia/references/apt-format.html

Guide to Large Scale Centralized Deployments

This guide covers a simple optimized approach to using a
{{{../../repository-management.html}repository manager}}
in a large organization with hundreds/thousands of Maven projects.

The pillars of this approach are:

[[1]] Use a centralized {{{../../repository-management.html}repository manager}}.

* Maven clients should download needed artifacts from the repository manager.

* Maven clients should upload proprietary artifacts to the repository manager.

[[2]] Configure the location to download/upload artifacts in Maven <<<settings.xml>>> files,
rather than in <<<pom.xml>>> files.

[[3]] Centrally manage the <<<settings.xml>>> files, and distribute them via automation.


* {Repository Manager Layout}

Repository managers generally have at least three types of repositories:

[hosted] contains artifacts uploaded to the repository manager

[proxy] proxies a remote repository and caches artifacts

[virtual] aggregates several repositories into one


The simplest way to organize repositories within a repository manager is
to have a single virtual repository that aggregates:

* a proxy repository for each public repository to mirror. (For example: Maven Central)

* a hosted repository for releases

* a hosted repository for snapshots

* a hosted repository that can contain both releases and snapshots
(Only needed if some projects are still using Maven Deploy Plugin \< 2.8.
See {{{Managing_Uploads_to_the_Repository_Manager}Managing Uploads to the Repository Manager}} for more info.)

Separate hosted repositories are generally used for releases and snapshots
due to the need for different artifact retention policies.

The following sections describe how to configure Maven clients to:

* {{{Managing_Downloads_from_the_Repository_Manager}Download}} artifacts from the virtual repository.

* {{{Managing_Uploads_to_the_Repository_Manager}Upload}} artifacts to one of the hosted repositories.

* {Managing Downloads from the Repository Manager}

All artifacts used by Maven projects in the organization should be downloaded
from the single virtual repository of the repository manager.

Maven can be instructed to download artifacts from the repository manager's virtual repository
by defining a mirror in the Maven <<<settings.xml>>> file as described in the
{{{./guide-mirror-settings.html}Guide to Mirror Settings}}.

Example: To download artifacts from the corporate repository manager's <<<maven-virtual>>> repository:

+-----+
<settings>
...
<mirrors>
<!-- Mirror all external repositories via the Corporate Repository Manager's Maven virtual repository -->
<mirror>
<id>corp-repository-manager</id>
<name>Corporate Repository Manager</name>
<mirrorOf>external:*</mirrorOf>
<url>https://corp-repository-manager-host/maven-virtual</url>
</mirror>
</mirrors>
...
</settings>
+-----+

* {Managing Uploads to the Repository Manager}

All proprietary artifacts produced by Maven projects in the organization should be uploaded
to the repository manager's hosted repositories.

The {{{../../plugins/maven-deploy-plugin}Maven Deploy Plugin}} can be instructed to upload artifacts
to the repository manager's repositories by defining the <<<alt*DeploymentRepository>>> properties
in the Maven <<<settings.xml>>> file.
When these properties are defined, the Maven Deploy Plugin's {{{../../plugins/maven-deploy-plugin/deploy-mojo.html}deploy}}
goal uses them instead of the <<<\<distributionManagement\>>>> section of <<<pom.xml>>> files
to determine where to upload artifacts.

Defining the upload destination of artifacts in <<<settings.xml>>> files rather than
in the <<<\<distributionManagement\>>>> section of <<<pom.xml>>> files allows the destinations
to be centrally managed, which simplifies maintenance if the destinations need to change.
In other words, rather than changing a huge number of <<<pom.xml>>> files,
you just need to change {{{Settings_File_Locations}relatively few}}
<<<settings.xml>>> files if/when the distribution locations need to change.

The ability to specify separate alternate deployment repositories for releases and snapshots
via the <<<altReleaseDeploymentRepository>>> and <<<altSnapshotDeploymentRepository>>> properties,
respectively, was added in Maven Deploy Plugin 2.8.
To get the most out of the approach defined in this document, all projects should use Maven Deploy Plugin \>=2.8.
If some projects are still using an older version of Maven Deploy Plugin (\>=2.3 and \<2.8), then
specify a single alternate deployment repository via the <<<altDeploymentRepository>>> property
that points to a repository capable of containing both releases and snapshots.

Typically, only continuous integration servers are allowed to upload artifacts to the repository manager.
Therefore, these settings should only be specified in <<<settings.xml>>> files on continuous integration servers,
and should not be in <<<settings.xml>>> files on developer machines.
Alternatively, if you want developers to be able to upload artifacts to the repository manager,
then include these properties in the <<<settings.xml>>> files used by developers.

Example: To upload artifacts to one of the corporate repository manager's hosted repositories:

+-----+
<settings>
...
<profiles>
<profile>

<id>corp-repository-manager</id>

<properties>
<!--
For Maven Deploy Plugin >= 2.8, deploy snapshots to this repository instead of the
distributionManagement snapshotRepository from project pom.xml files.
-->
<altSnapshotDeploymentRepository>corp::default::https://corp-repository-manager-host/maven-snapshots</altSnapshotDeploymentRepository>

<!--
For Maven Deploy Plugin >= 2.8, deploy releases to this repository instead of the
distributionManagement repository from project pom.xml files.
-->
<altReleaseDeploymentRepository>corp::default::https://corp-repository-manager-host/maven-releases</altReleaseDeploymentRepository>

<!--
Only needed if some projects are still using Maven Deploy Plugin >=2.3 and < 2.8,
which is the case if projects are using the default version of Maven Deploy Plugin in maven 3.x.
For Maven Deploy Plugin >=2.3 and < 2.8, deploy both releases and snapshots to this repository
instead of the repositories mentioned in distributionManagement from project pom.xml files.
-->
<altDeploymentRepository>corp::default::https://corp-repository-manager-host/maven-combined</altDeploymentRepository>
</properties>

<repositories>
<repository>
<id>corp</id>
<!--
This URL is overridden by the corp-repository-manager mirror above.
Some misbehaving tools might complain if they can't resolve the host specified here.
If you encounter this problem, use the same URL as the corp-repository-manager mirror.
-->
<url>https://ignored</url>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>

<pluginRepositories>
<pluginRepository>
<id>corp</id>
<!--
This URL is overridden by the corp-repository-manager mirror above.
Some misbehaving tools might complain if they can't resolve the host specified here.
If you encounter this problem, use the same URL as the corp-repository-manager mirror.
-->
<url>https://ignored</url>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>

</profile>
</profiles>
<activeProfiles>
<activeProfile>corp-repository-manager</activeProfile>
</activeProfiles>
...
</settings>
+-----+


* {Settings File Locations}

Maven <<<settings.xml>>> files need to be available wherever Maven builds are performed, typically:

* on continuous integration servers, and

* on developer machines

Both locations should have the mirror settings mentioned in
{{{Managing_Downloads_from_the_Repository_Manager}Managing Downloads from the Repository Manager}}.

Typically, only continuous integration servers should have the deployment repository settings
mentioned in {{{Managing_Uploads_to_the_Repository_Manager}Managing Uploads to the Repository Manager}},
because only continuous integration servers should be allowed to upload to the repository manager.
Alternatively, if you want developers to be able to upload artifacts to the repository manager,
then include the deployment repository properties in the <<<settings.xml>>> files used by developers.

How the <<<settings.xml>>> files are stored and updated is beyond the scope of this document.
The general recommendation is to manage a few <<<settings.xml>>> files centrally,
and then use automation to distribute them to continuous integration servers and developer machines.
2 changes: 2 additions & 0 deletions content/apt/guides/mini/index.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@

* {{{$prefix/guide-default-execution-ids.html} Guide to Configuring Default Mojo Executions}}

* {{{$prefix/guide-large-scale-centralized-deployments.html} Guide to Large Scale Centralized Deployments}}

~~ ** Testing

~~ * {{{$prefix/guide-attached-tests.html}Reusable Test JARs}}
1 change: 1 addition & 0 deletions content/site.xml
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ under the License.
<item name="Alternative Wagon Providers" href="/guides/mini/guide-wagon-providers.html" />
<item name="Building JDK 1.4 Projects Using JDK 1.5" href="/guides/mini/guide-building-jdk14-on-jdk15.html" />
<item name="Configuring Default Mojo Executions" href="/guides/mini/guide-default-execution-ids.html" />
<item name="Large Scale Centralized Deployments" href="/guides/mini/guide-large-scale-centralized-deployments.html" />
</item>
<item name="Settings Reference" href="/settings.html" /> <!-- TODO: all references should be collected under another area instead/as well, reconsider -->
<item name="POM Reference" href="/pom.html" />
Expand Down

0 comments on commit 0b33ee0

Please sign in to comment.