Review documentation regarding self-managed package registry (elastic…

…#1287)

Starting on 7.16 we are going to officialy support self-managed package
registry deployments for air-gapped environments. Update documentation
referring to this feature.

docs/en/ingest-management/beats-agent-comparison.asciidoc

@@ -388,7 +388,7 @@ The following table shows a comparison of capabilities supported by {beats} and
 |{y}
 |{n}
 |{y}
-|The Integrations app requires a network connection to the {fleet-guide}/fleet-overview.html#package-registry-intro[Elastic Package Registry]. We are considering an https://github.com/elastic/integrations/issues/1178[on-prem version of EPR]. {fleet}-managed {agent}s require a connection to our artifact repository for agent binary upgrades. These are not required for standalone {agent}s or {beats}.
+|The Integrations app requires a network connection to the {fleet-guide}/fleet-overview.html#package-registry-intro[{package-registry}], or a {fleet-guide}/air-gapped.html#air-gapped-diy-epr[self-managed deployment of the {package-registry}]. {fleet}-managed {agent}s require a connection to our artifact repository for agent binary upgrades. These are not required for standalone {agent}s or {beats}.
 
 |Run without root on host
 |{y}

docs/en/ingest-management/fleet/air-gapped.asciidoc

@@ -0,0 +1,126 @@
+[[air-gapped]]
+= Air-gapped environments
+
+There are certain environments in which network traffic restrictions are mandatory. In these environments, the {kib} instance
+isn't able to reach the public {package-registry} endpoints, like https://epr.elastic.co/[epr.elastic.co], to download
+package metadata and content.
+
+There are two workarounds in this situation -- use a proxy server as network gateway to reach the public endpoints,
+or deploy your own instance of the {package-registry}.
+
+[discrete]
+[[air-gapped-proxy-server]]
+== Use a proxy server
+
+If you can route traffic to the public endpoint of the {package-registry} through a network gateway, there is a property in {kib} that
+can orchestrate to use a proxy server:
+
+[source,yaml]
+----
+xpack.fleet.registryProxyUrl: your-nat-gateway.corp.net
+----
+
+[discrete]
+[[air-gapped-diy-epr]]
+== Host your own {package-registry}
+
+If routing traffic through a proxy server is not an option, you can host your own {package-registry}.
+
+The {package-registry} can be deployed and hosted on-site using one of the
+available Docker images. These docker images include the {package-registry} and
+a selection of packages.
+
+There are different distributions available:
+
+* {version} (recommended): +docker.elastic.co/package-registry/distribution:{version}+ - Selection of packages from the production repository released with the {version} version of the {stack}.
+* production: `docker.elastic.co/package-registry/distribution:production` - Packages available in the production registry (https://epr.elastic.co).
+* staging: `docker.elastic.co/package-registry/distribution:staging` - Packages available in the staging registry (https://epr-staging.elastic.co). These packages may be pending of validation.
+* snapshot: `docker.elastic.co/package-registry/distribution:snapshot` - Packages under development.
+
+ifeval::["{release-state}"=="unreleased"]
+[WARNING]
+====
+Version {version} of the {package-registry} distribution has not yet been released.
+====
+endif::[]
+
+To update the distribution image, re-pull the image and then restart the docker container.
+
+Every distribution contains packages that can be used by different versions of
+the {stack}. The {package-registry} API exposes a {kib} version constraint that
+allows for filtering packages that are compatible with a particular version.
+
+NOTE: These steps use the standard Docker CLI, but you can create a Kubernetes manifest
+based on this information.
+These images can also be used with other container runtimes compatible with Docker images.
+
+1. Pull the Docker image from the public Docker registry:
++
+["source", "sh", subs="attributes"]
+----
+docker pull docker.elastic.co/package-registry/distribution:{version}
+----
++
+2. Save the Docker image locally:
++
+["source", "sh", subs="attributes"]
+----
+docker save -o package-registry-{version}.tar docker.elastic.co/package-registry/distribution:{version}
+----
++
+TIP: Check the image size to ensure that you have enough disk space. 
+
+3. Transfer the image to the air-gapped environment and load it:
++
+["source", "sh", subs="attributes"]
+----
+docker load -i package-registry-{version}.tar
+----
+
+4. Run the {package-registry}:
++
+["source", "sh", subs="attributes"]
+----
+docker run -it -p 8080:8080 docker.elastic.co/package-registry/distribution:{version}
+----
+
+5. (Optional) You can monitor the health of your {package-registry} with
+requests to the root path:
++
+["source", "sh", subs="attributes"]
+----
+docker run -it -p 8080:8080 \
+    --health-cmd "curl -f -L http://127.0.0.1:8080/" \
+    docker.elastic.co/package-registry/distribution:{version}
+----
+
+[discrete]
+[[air-gapped-diy-epr-kibana]]
+=== Connect {kib} to your hosted {package-registry}
+
+Use the `xpack.fleet.registryUrl` property in the {kib} config to set the URL of your hosted package registry. For example:
+
+[source,yaml]
+----
+xpack.fleet.registryUrl: "http://package-registry.corp.net:8080"
+----
+
+[discrete]
+[[air-gapped-tls]]
+=== TLS configuration of the {package-registry}
+
+You can configure the {package-registry} to listen on a secure HTTPS port using TLS.
+
+For example, given a key and a certificate pair available in `/etc/ssl`, you
+can start the {package-registry} listening on the 443 port using the following command:
+
+["source", "sh", subs="attributes"]
+----
+docker run -it -p 443:443 \
+  -v /etc/ssl/package-registry.key:/etc/ssl/package-registry.key:ro \
+  -v /etc/ssl/package-registry.crt:/etc/ssl/package-registry.crt:ro \
+  -e EPR_ADDRESS=0.0.0.0:443 \
+  -e EPR_TLS_KEY=/etc/ssl/package-registry.key \
+  -e EPR_TLS_CERT=/etc/ssl/package-registry.crt \
+  docker.elastic.co/package-registry/distribution:{version}
+----

docs/en/ingest-management/fleet/fleet-limitations.asciidoc

@@ -13,7 +13,7 @@ This feature has additional limitations at the current time:
 *   Support for a limited number of integrations (more coming soon)
 *   No output to {ls}, Kafka, or other remote clusters
 *   No proxy support for {fleet-server}
-*   Requires internet access for {kib} to download integration packages from the Elastic Package Registry
+*   Requires internet access or a <<air-gapped-diy-epr,self-managed instance of the {package-registry}>> for {kib} to download integration packages.
 *   {agent} requires internet access to perform binary upgrades from Fleet
 *   Limited support in the {fleet} app for advanced {beats} settings like multiline, processors, and so
 on

docs/en/ingest-management/index.asciidoc

@@ -50,6 +50,8 @@ include::integrations/upgrade-integration.asciidoc[leveloffset=+2]
 
 include::integrations/uninstall-integration.asciidoc[leveloffset=+2]
 
+include::fleet/air-gapped.asciidoc[leveloffset=+2]
+
 include::elastic-agent/install-elastic-agent.asciidoc[leveloffset=+1]
 
 include::elastic-agent/install-fleet-managed-elastic-agent.asciidoc[leveloffset=+2]

docs/en/ingest-management/integrations/integrations.asciidoc

@@ -45,3 +45,10 @@ too.
 |Remove an integration and its assets from {kib}.
 
 |===
+
+[NOTE]
+====
+The *Integrations* app in {kib} needs access to the public {package-registry} to
+discover integrations. If your deployment has network restrictions, you can
+{fleet-guide}/air-gapped.html#air-gapped-diy-epr[deploy your own self-managed {package-registry}].
+====

docs/en/ingest-management/overview.asciidoc

@@ -58,6 +58,9 @@ To do so, add the following setting to your `kibana.yml` file:
 xpack.fleet.registryProxyUrl: <your-proxy-address>
 ----
 
+You can find more information about running the {package-registry} in air-gapped
+environments in the section about {fleet-guide}/air-gapped.html[Air-gapped environments].
+
 [discrete]
 [[configuring-integrations]]
 == {agent} {integrations}

docs/en/integrations/air-gapped.asciidoc

@@ -1,96 +1,5 @@
 [[air-gapped]]
 = Air-gapped environments
 
-There are certain environments in which network traffic restrictions are mandatory. In these environments, the Kibana instance
-isn't able to reach the public Elastic Package Registry (EPR) endpoints, like https://epr.elastic.co/[epr.elastic.co], to download
-package metadata and content.
-
-There are two workarounds in this situation -- use a proxy server as network gateway to reach the public endpoints,
-or deploy your own instance of the Elastic Package Registry.
-
-[discrete]
-[[air-gapped-proxy-server]]
-== Use a proxy server
-
-If you can route traffic to the public endpoint of EPR through a network gateway, there is a property in Kibana that
-can orchestrate to use a proxy server:
-
-[source,yaml]
-----
-xpack.fleet.registryProxyUrl: your-nat-gateway.corp.net
-----
-
-For more information, see the {fleet-guide}/fleet-overview.html#package-registry-intro[Fleet and Elastic Agent Guide].
-
-[discrete]
-[[air-gapped-diy-epr]]
-== Host your own Elastic Package Registry
-
-beta::[]
-
-If routing traffic through a proxy server is not an option, you can host your own Elastic Package Registry.
-The Package Storage instance must be deployed and hosted on-site as a Docker container.
-Package Storage is a special distribution of the Package Registry which already includes packages.
-There are different distributions available:
-
-* production (recommended): `docker.elastic.co/package-registry/distribution:production` - stable, tested package revisions
-* staging: `docker.elastic.co/package-registry/distribution:staging` - package revisions ready for testing before release
-* snapshot: `docker.elastic.co/package-registry/distribution:snapshot` - package revisions updated on daily basis
-
-If you want to update the Package Storage image, you need to re-pull the image and restart docker container.
-
-Every distribution contains packages that can be used by different versions of the {stack}. As we adopted a continuous delivery pipeline for packages,
-we haven't introduced the box release approach so far (7.13.0, 7.14.0, etc.). The Package Registry API exposes a {kib} version constraint
-that allows for filtering packages that are compatible with particular stack version.
-
-NOTE: These steps use the standard Docker CLI, but it shouldn't be hard to transform them into Kubernetes descriptor file.
-Here is the k8s descriptor used by the e2e-testing project: https://github.com/elastic/e2e-testing/blob/k8s-deployment/cli/config/kubernetes/base/package-registry/[yaml files].
-
-1. Pull the Docker image from the public Docker registry:
-+
-[source,bash]
-----
-docker pull docker.elastic.co/package-registry/distribution:production
-----
-
-2. Save the Docker image locally:
-+
-[source,bash]
-----
-docker save -o epr.tar docker.elastic.co/package-registry/distribution:production
-----
-+
-TIP: please mind the image size, so you won't hit any capacity limit.
-
-3. Transfer the image to the air-gapped environment and load:
-+
-[source,bash]
-----
-docker load -i epr.tar
-----
-
-4. Run the Package Registry:
-+
-[source,bash]
-----
-docker run -it docker.elastic.co/package-registry/distribution:production
-----
-
-5. (Optional) Define the internal healthcheck for the service as:
-+
-[source,bash]
-----
-curl -f http://127.0.0.1:8080
-----
-
-[discrete]
-[[air-gapped-diy-epr-kibana]]
-=== Connect Kibana to the hosted Package Registry
-
-There is a dedicated property in the {kib} config to change the URL of the Package Registry's endpoint to a custom one.
-The example below connects to an internally hosted instance:
-
-[source,yaml]
-----
-xpack.fleet.registryUrl: "http://package-registry.corp.net:8080"
-----
+You can find the documentation about {fleet} and the {package-registry} in
+air-gapped environments in {fleet-guide}/air-gapped.html.

docs/en/integrations/what-is-an-integration.asciidoc

@@ -31,9 +31,9 @@ All packages must adhere to the <<package-spec,package specification>> -- a form
 
 . Publish the integration to the package registry
 +
-Once an integration (package) has been created, it needs to be built. Built integrations are stored in the {package-storage-repo-link}[Package Storage repository] and served up via the {package-registry-repo-link}[Elastic Package Registry (EPR)].
+Once an integration (package) has been created, it needs to be built. Built integrations are stored in the {package-storage-repo-link}[Package Storage repository] and served up via the {package-registry-repo-link}[{package-registry}].
 The {fleet} UI in {kib} connects to the {package-registry} and allows users to discover, install, and configure Elastic Packages.
-The EPR can also be run locally for testing purposes.
+The {package-registry} can also be {fleet-guide}/air-gapped.html#air-gapped-diy-epr[deployed on-premise in air-gapped environments].
 
 . Install the integration
 +