[docs] Add GCP settings (#16228) (#16339)

libbeat/docs/command-reference.asciidoc

@@ -399,15 +399,15 @@ ifeval::["{beatname_lc}"=="functionbeat"]
 Shows help for the `package` command.
 
 *`-o, --output`*::
-Specifies the full path to the zip file that will contain the package.
+Specifies the full path pattern to use when creating the packages. 
 
 {global-flags}
 
 *EXAMPLES*
 
 ["source","sh",subs="attributes"]
 -----
-{beatname_lc} package /path/to/file.zip
+{beatname_lc} package --output /path/to/folder/package-{{.Provider}}.zip
 -----
 
 [[remove-command]]

x-pack/functionbeat/docs/config-options.asciidoc → x-pack/functionbeat/docs/config-options-aws.asciidoc

@@ -1,9 +1,9 @@
 [id="configuration-{beatname_lc}-options"]
 [role="xpack"]
-== Configure functions
+== Configure AWS functions
 
 ++++
-<titleabbrev>Configure functions</titleabbrev>
+<titleabbrev>Configure AWS functions</titleabbrev>
 ++++
 
 {beatname_uc} runs as a function in your serverless environment.

x-pack/functionbeat/docs/config-options-gcp.asciidoc

@@ -0,0 +1,198 @@
+[id="configuration-{beatname_lc}-gcp-options"]
+[role="xpack"]
+== Configure Google Functions
+
+++++
+<titleabbrev>Configure Google functions</titleabbrev>
+++++
+
+beta[]
+
+{beatname_uc} runs as a Google Function on Google Cloud Platform (GCP).
+
+Before deploying {beatname_uc}, you need to configure one or more functions and
+specify details about the services that will trigger the functions.
+
+You configure the functions in the the +{beatname_lc}.yml+ configuration file.
+When you're done, you can <<deploy-to-cloud-provider,deploy the functions>>
+to your serverless environment.
+
+The following example configures two functions: `pubsub` and `storage`. The
+`pubsub` function collects log events from https://cloud.google.com/pubsub/[Google
+Pub/Sub]. The `storage` function collects log events from
+https://cloud.google.com/storage/[Google Cloud Storage]. Both functions in the
+example forward the events to {es}.
+
+["source","sh",subs="attributes"]
+----
+functionbeat.provider.gcp.location_id: "europe-west2"
+functionbeat.provider.gcp.project_id: "my-project-123456"
+functionbeat.provider.gcp.storage_name: "functionbeat-deploy"
+functionbeat.provider.gcp.functions:
+  - name: pubsub
+    enabled: true
+    type: pubsub
+    description: "Google Cloud Function for Pub/Sub"
+    trigger:
+      resource: "projects/_/pubsub/myPubSub"
+      #service: "pubsub.googleapis.com"
+  - name: storage
+    enabled: true
+    type: storage
+    description: "Google Cloud Function for Cloud Storage"
+    trigger:
+      resource: "projects/my-project/buckets/my-storage"
+      event_type: "google.storage.object.finalize"
+
+cloud.id: "MyESDeployment:SomeLongString=="
+cloud.auth: "elastic:mypassword"
+----
+
+[id="{beatname_lc}-gcp-options"]
+[float]
+=== Configuration options
+Specify the following options to configure the functions
+that you want to deploy to Google Cloud Platform (GCP).
+
+TIP: If you change the configuration after deploying the function, use
+the <<update-command,`update` command>> to update your deployment.
+
+[float]
+[id="{beatname_lc}-gcp-location_id"]
+==== `provider.gcp.location_id`
+
+The region where your GCP project is located.
+
+[float]
+[id="{beatname_lc}-gcp-project_id"]
+==== `provider.gcp.project_id`
+
+The ID of the GCP project where the function artifacts will be deployed. See the
+https://cloud.google.com/about/locations/[Google Cloud Function documentation]
+to verify that Cloud Functions are supported in the region you specify.
+
+[float]
+[id="{beatname_lc}-gcp-storage_name"]
+==== `provider.gcp.storage_name`
+
+The name of the Google Cloud storage bucket where the function artifacts will be
+deployed. If the bucket doesn't exist, it will be created, if you have the
+correct project permissions (`storage.objects.create`).
+
+[float]
+[id="{beatname_lc}-gcp-functions"]
+==== `functionbeat.provider.gcp.functions`
+A list of functions that are available for deployment.
+
+[float]
+[id="{beatname_lc}-gcp-name"]
+===== `name`
+
+A unique name for the Google function.
+
+[float]
+[id="{beatname_lc}-gcp--type"]
+===== `type`
+
+The type of GCP service to monitor. For this release, the supported types
+are:
+
+[horizontal]
+`pubsub`:: Collect log events from Google Pub/Sub.
+`storage`:: Collect log events from Google Cloud storage buckets. 
+
+[float]
+[id="{beatname_lc}-gcp-description"]
+===== `description`
+
+A description of the function. This description is useful when you are running
+multiple functions and need more context about how each function is used.
+
+[float]
+[id="{beatname_lc}-gcp-memory-size"]
+==== `memory_size`
+
+The maximum amount of memory to allocate for this function. 
+The default is `256MB`.
+
+[float]
+[id="{beatname_lc}-gcp-timeout"]
+==== `timeout`
+
+The execution timeout in seconds. If the function does not finish in time,
+it is considered failed and terminated. The default is `60s`. Increase this
+value if you see timeout messages is the Google Stackdriver logs.
+
+[float]
+[id="{beatname_lc}-gcp-service_account_email"]
+==== `service_account_email`
+
+The email of the service account that the function will assume as its identity.
+The default is {projectid}@appspot.gserviceaccount.com.email.
+
+[float]
+[id="{beatname_lc}-gcp-labels"]
+==== `labels`
+
+One or more labels to apply to the function. A label is a key-value pair that
+helps you organize your Google Cloud resources.
+
+[float]
+[id="{beatname_lc}-gcp-vpc_connector"]
+==== `vpc_connector`
+
+A VPC connector that the function can connect to when sending requests to
+resources in your VPC network.
+
+Use the format `projects/*/locations/*/connectors/*` or a fully qualified
+URI.
+
+[float]
+[id="{beatname_lc}-gcp-maximum_instances"]
+==== `maximum_instances`
+
+The maximum instances that can be running at the same time. The default is
+unlimited.
+
+[float]
+[id="{beatname_lc}-gcp-triggers"]
+===== `trigger`
+
+The trigger that will cause the function to execute.
+
+* If `type` is `pubsub`, specify the name of the Pub/Sub topic to watch for
+messages.
+
+* If `type` is `storage`, specify the Cloud Storage bucket to watch for object
+events. For `event_type`, specify the type of object event that will trigger the
+function. See the https://cloud.google.com/functions/docs/calling/storage[Google Cloud
+docs] for a list of available event types.
+
+[float]
+[id="{beatname_lc}-gcp-keep_null"]
+==== `keep_null`
+
+If `true`, fields with null values will be published in the output document. By
+default, `keep_null` is `false`.
+
+[float]
+[id="{beatname_lc}-gcp-fields"]
+==== `fields`
+
+Optional fields that you can specify to add additional information to the
+output. Fields can be scalar values, arrays, dictionaries, or any nested
+combination of these.
+
+[float]
+[id="{beatname_lc}-gcp-processors"]
+==== `processors`
+
+Define custom processors for this function. For example, you can specify a
+dissect processor to tokenize a string:
+
+[source,yaml]
+----
+processors:
+  - dissect:
+      tokenizer: "%{key1} %{key2}"
+----

x-pack/functionbeat/docs/configuring-howto.asciidoc

@@ -13,6 +13,7 @@ include::{libbeat-dir}/shared-configuring.asciidoc[]
 The following topics describe how to configure {beatname_uc}:
 
 * <<configuration-{beatname_lc}-options>>
+* <<configuration-{beatname_lc}-gcp-options>>
 * <<configuration-general-options>>
 * <<configuring-internal-queue>>
 * <<configuring-output>>
@@ -31,7 +32,9 @@ The following topics describe how to configure {beatname_uc}:
 
 --
 
-include::./config-options.asciidoc[]
+include::./config-options-aws.asciidoc[]
+
+include::./config-options-gcp.asciidoc[]
 
 include::./general-options.asciidoc[]
 

x-pack/functionbeat/docs/deploying.asciidoc

@@ -4,11 +4,8 @@
 
 After configuring {beatname_uc} and defining cloud functions for the services
 you want to monitor, deploy the functions to your cloud provider. To do this,
-you can:
-
-* <<manager-deployment,Use the {beatname_uc} manager>> (good for getting
-  started), 
-* Or <<own-deployment,use your own deployment infrastructure>>.
+you can use the {beatname_uc} manager (good for getting started), or use your
+own deployment infrastructure.
 
 [[manager-deployment]]
 ==== Use the {beatname_uc} manager
@@ -18,8 +15,9 @@ when you don't have your own deployment infrastructure or process in place.
 
 During deployment, the {beatname_uc} manager:
 
-* Exports an AWS CloudFormation template. You can inspect the template by
-running the <<export-command,`export`>> command.
+* Exports a function template to use for deployment. For AWS, it exports an
+{cloudformation-ref} template. For Google Cloud, it exports a YAML configuration
+file. To inspect the template, run the <<export-command,`export`>> command.
 * Creates a zip package that includes the function code and +{beatname_lc}.yml+
 config file.
 * Uploads the package to the specified cloud provider.
@@ -38,8 +36,8 @@ provider:
 +
 `BEAT_STRICT_PERMS=false`:: This setting makes the function skip the ownership
 check on the configuration file.
-`ENABLED_FUNCTIONS=function-name-1,function-name-2`:: Specifies a comma-
-separated list of functions that are enabled in the configuration file. For
+`ENABLED_FUNCTIONS=function-name-1,function-name-2`:: Specifies a
+comma-separated list of functions that are enabled in the configuration file. For
 example, to package functions called `my-kinesis` and `my-cloudwatch-logs`, run:
 +
 *linux and mac*:
@@ -67,26 +65,51 @@ archive. For example:
 +
 ["source","sh",subs="attributes"]
 ----------------------------------------------------------------------
-./{beatname_lc} -v -e -d "*" package --output /path/to/file.zip
+./{beatname_lc} -v -e -d "*" package --output /path/to/folder/package-{{.Provider}}.zip
 ----------------------------------------------------------------------
 +
 *win:*
 +
 ["source","sh",subs="attributes"]
 ----------------------------------------------------------------------
-.{backslash}{beatname_lc}.exe -v -e -d "*" package --output /path/to/file.zip
+.{backslash}{beatname_lc}.exe -v -e -d "*" package --output /path/to/folder/package-{{.Provider}}.zip
 ----------------------------------------------------------------------
 +
-This command generates a deployment package (called `file.zip` in the example)
-that contains:
+For `--output` specify a full path pattern.
++
+The `package` command generates deployment packages for each provider specified
+in the configuration. Each package contains:
 +
-* a binary, called `functionbeat-aws`, that contains the function code
+* a binary with the function code
 * the `functionbeat.yml` config file
 
 . If certificates are required, add the cert files to the zip package under the
-same path as the configured +{beatname_lc}.yml+ file. 
+same path as the configured +{beatname_lc}.yml+ file.
+
+. Export a function template to use for deployment:
++
+*linux and mac:*
++
+["source","sh",subs="attributes"]
+----------------------------------------------------------------------
+./{beatname_lc} export function FUNCTION_NAME
+----------------------------------------------------------------------
++
+*win:*
++
+["source","sh",subs="attributes"]
+----------------------------------------------------------------------
+.{backslash}{beatname_lc}.exe export function FUNCTION_NAME
+----------------------------------------------------------------------
++
+{beatname_uc} writes the template to stdout. For AWS functions, it writes an
+{cloudformation-ref} tempalte. For Google Cloud, it writes a YAML configuration
+file.
+
+. Modify the template to work with your infrastructure.
 
 . Deploy the package, using the infrastructure and automation supported by your
-cloud provider. For example, to deploy the package to AWS,
-<<export-cloudformation-template,export the AWS CloudFormation template>>
-managed by {beatname_uc}, and modify it to work with your infrastructure. 
+cloud provider, for example, {cloudformation-ref} or
+https://cloud.google.com/deployment-manager[Google Cloud Deployment manager].
++
+For more information about deployment, see your cloud provider's documentation.

x-pack/functionbeat/docs/export-cloudformation-template.asciidoc

@@ -1,6 +1,6 @@
 [[export-cloudformation-template]]
 [role="xpack"]
-=== Export AWS CloudFormation template
+=== Export AWS CloudFormation template  
 
 You can use {beatname_uc} to export an {cloudformation-ref} template then use
 the template with automation software to deploy {beatname_uc} code to your cloud