It’s a technical release to conclude the migration of UCC UI into this repository and we also happen to release 1 feature.
+apiVersion from globalConfig.json and bump schemaVersion (6c22704)additional_packaging.py file¶To extend the build process, you can create a additional_packaging.py file in the same file level where you have your globalConfig file.
This file should have the additional_packaging function, which accepts add-on name as its only argument.
See the following example for proper usage:
+ucc-gen finishes all its necessary steps.ucc-gen feature that has not been implemented.We can use this feature to map each field with meaningful value to display in the table. For example, the category field contains 1, 2, and 4 values, but when those values are displayed, the user might get confused as those values do not signify the meaning of their mapping. To avoid this confusion, the user can map each field with meaningful value as shown in the following example:
+{
+ "name": "account",
+ "title": "Account",
+ "table": {
+ "header": [
+ {
+ "field": "name",
+ "label": "Name"
+ },
+ {
+ "field": "key_id",
+ "label": "Key ID"
+ },
+ {
+ "field": "category",
+ "label": "Region Category",
+ "mapping": {
+ "1": "Global",
+ "2": "US Gov",
+ "4": "China"
+ }
+ }
+ ],
+ "actions": [
+ "edit",
+ "delete"
+ ]
+ },
+ "entity": [
+ {
+ "field": "name",
+ "label": "Name",
+ "type": "text",
+ "required": true,
+ },
+ {
+ "field": "key_id",
+ "label": "Key ID",
+ "type": "text",
+ },
+ {
+ "field": "category",
+ "label": "Region Category",
+ "type": "singleSelect",
+ "required": true,
+ "defaultValue": 1,
+ "options": {
+ "disableSearch": true,
+ "autoCompleteFields": [
+ {
+ "label": "Global",
+ "value": 1
+ },
+ {
+ "label": "GovCloud",
+ "value": 2
+ },
+ {
+ "label": "China",
+ "value": 4
+ }
+ ]
+ }
+ }
+ ]
+}
+This is how it looks in the UI:
+
It is possible to extend the default behaviour of the UCC-generated REST handlers.
+For example, if your add-on requires an API key to operate and you want to validate this API key during its creation, you might want to use the custom REST handlers to achieve that.
+++Note:
+ucc-genwill not override the REST handler code if you use the correct file name under thebinfolder.
See the following example of how it can be done. It contains a REST handler for creating an organization, with the organization_id and organization_api_key fields, which are not custom ones, generated by ucc-gen. But, also in the example, there is a custom handler, CustomRestHandler class which has additional steps for configuration creation and edit operations, specifically handleEdit and handleCreate methods:
import import_declare_test
+
+import logging
+
+from splunktaucclib.rest_handler import admin_external, util
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+from splunktaucclib.rest_handler.endpoint import (
+ RestModel,
+ SingleModel,
+ field,
+ validator,
+)
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ "organization_id",
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.AllOf(
+ validator.String(
+ max_len=50,
+ min_len=1,
+ ),
+ validator.Pattern(
+ regex=r"""^\d+$""",
+ ),
+ ),
+ ),
+ field.RestField(
+ "organization_api_key",
+ required=True,
+ encrypted=True,
+ default=None,
+ validator=validator.AllOf(
+ validator.String(
+ max_len=50,
+ min_len=1,
+ ),
+ validator.Pattern(
+ regex=r"""^[a-z0-9]+$""",
+ ),
+ ),
+ ),
+]
+model = RestModel(fields, name=None)
+
+
+endpoint = SingleModel(
+ "addon_name_organization", model, config_name="organization"
+)
+
+
+def _validate_organization(organization_id, organization_api_key):
+ # Some code to validate the API key.
+ # Should return nothing if the configuration is valid.
+ # Should raise an exception splunktaucclib.rest_handler.error.RestError if the configuration is not valid.
+ ...
+
+
+class CustomRestHandler(AdminExternalHandler):
+ def __init__(self, *args, **kwargs):
+ AdminExternalHandler.__init__(self, *args, **kwargs)
+
+ def handleList(self, confInfo):
+ AdminExternalHandler.handleList(self, confInfo)
+
+ def handleEdit(self, confInfo):
+ _validate_organization(
+ self.payload.get("organization_id"),
+ self.payload.get("organization_api_key"),
+ )
+ AdminExternalHandler.handleEdit(self, confInfo)
+
+ def handleCreate(self, confInfo):
+ _validate_organization(
+ self.payload.get("organization_id"),
+ self.payload.get("organization_api_key"),
+ )
+ AdminExternalHandler.handleCreate(self, confInfo)
+
+ def handleRemove(self, confInfo):
+ AdminExternalHandler.handleRemove(self, confInfo)
+
+
+if __name__ == "__main__":
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=CustomRestHandler,
+ )
+++UCC 5.18.0 natively supports custom REST handlers for the modular inputs.
+
One common scenario is to delete a checkpoint after you delete an +input in the Inputs page. Otherwise, users may face strange consequences if +they create an input with the same name as the input that was deleted, and this +newly created input will reuse the old checkpoint, because the names of +the inputs are the same. We would like to avoid this situation in the add-on.
+This can be done without a need to modify the REST handler code
+automatically by running ucc-gen.
See the following automatically generated REST handler code for a modular input REST +handler:
+import import_declare_test
+
+from splunktaucclib.rest_handler.endpoint import (
+ field,
+ validator,
+ RestModel,
+ DataInputModel,
+)
+from splunktaucclib.rest_handler import admin_external, util
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+import logging
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ 'interval',
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.Pattern(
+ regex=r"""^\-[1-9]\d*$|^\d*$""",
+ )
+ ),
+
+ field.RestField(
+ 'disabled',
+ required=False,
+ validator=None
+ )
+
+]
+model = RestModel(fields, name=None)
+
+
+
+endpoint = DataInputModel(
+ 'example_input_one',
+ model,
+)
+
+
+if __name__ == '__main__':
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=AdminExternalHandler,
+ )
+New file needs to be created in the bin folder of the add-on. Let’s call it
+splunk_ta_uccexample_delete_checkpoint_rh.py (name can be different).
And put the following content into the file.
+import import_declare_test
+
+from splunktaucclib.rest_handler.admin_external import AdminExternalHandler
+
+
+class CustomRestHandlerDeleteCheckpoint(AdminExternalHandler):
+ def __init__(self, *args, **kwargs):
+ AdminExternalHandler.__init__(self, *args, **kwargs)
+
+ def handleList(self, confInfo):
+ AdminExternalHandler.handleList(self, confInfo)
+
+ def handleEdit(self, confInfo):
+ AdminExternalHandler.handleEdit(self, confInfo)
+
+ def handleCreate(self, confInfo):
+ AdminExternalHandler.handleCreate(self, confInfo)
+
+ def handleRemove(self, confInfo):
+ # Add your code here to delete the checkpoint!
+ AdminExternalHandler.handleRemove(self, confInfo)
+Then, in globalConfig file you need to change the behaviour of the UCC to reuse +the REST handler that was just created.
+{
+ "name": "example_input_one",
+ "restHandlerModule": "splunk_ta_uccexample_delete_checkpoint_rh", <----- new field
+ "restHandlerClass": "CustomRestHandlerDeleteCheckpoint", <----- new field
+ "entity": [
+ "..."
+ ],
+ "title": "Example Input One"
+}
+After ucc-gen command is executed again, the generated REST handler for this
+input will be changed to the following.
import import_declare_test
+
+from splunktaucclib.rest_handler.endpoint import (
+ field,
+ validator,
+ RestModel,
+ DataInputModel,
+)
+from splunktaucclib.rest_handler import admin_external, util
+from splunk_ta_uccexample_delete_checkpoint_rh import CustomRestHandlerDeleteCheckpoint # <----- changed
+import logging
+
+util.remove_http_proxy_env_vars()
+
+
+fields = [
+ field.RestField(
+ 'interval',
+ required=True,
+ encrypted=False,
+ default=None,
+ validator=validator.Pattern(
+ regex=r"""^\-[1-9]\d*$|^\d*$""",
+ )
+ ),
+
+ field.RestField(
+ 'disabled',
+ required=False,
+ validator=None
+ )
+
+]
+model = RestModel(fields, name=None)
+
+
+
+endpoint = DataInputModel(
+ 'example_input_one',
+ model,
+)
+
+
+if __name__ == '__main__':
+ logging.getLogger().addHandler(logging.NullHandler())
+ admin_external.handle(
+ endpoint,
+ handler=CustomRestHandlerDeleteCheckpoint, # <----- changed
+ )
+This feature allows us to pass broarder description on Input and Configuration page displayed under main description.
+| Property | +Type | +Description | +
|---|---|---|
| create | +object | +Warning object definition for create form | +
| edit | +object | +Warning object definition for edit form | +
| clone | +object | +Warning object definition for clone form | +
| config | +object | +Warning object definition for config form | +
| Property | +Type | +Description | +
|---|---|---|
| message* | +string | +Text used for that description, you can put \n to add a breakline | +
| alwaysDisplay | +boolean | +Force warning to be always displayed, even after input changes. Default value is false. | +
"warning": {
+ "create": {
+ "message": "Some warning for create form",
+ "alwaysDisplay": true
+ },
+ "edit": {
+ "message": "Some warning for edit form "
+ },
+ "clone": {
+ "message": "Some warning for clone form"
+ }
+},
+This feature allows dynamic loading options for the singleSelect and the multipleSelect fields when the options for that field depend on other fields’ values. It loads options via an API call to the endpoint mentioned in endpointUrl under options when any dependencies field is updated and all required dependencies fields are non-null.
All non-required dependencies fields can be of any type, but all required dependencies fields should only be of single-select type.
+All dependencies fields’ values are added to the endpoint URL as query parameters.
+{
+ "field": "bucket_name",
+ "label": "S3 Bucket",
+ "type": "singleSelect",
+ "required": true,
+ "options": {
+ "disableonEdit": true,
+ "dependencies": [
+ "aws_account",
+ "aws_iam_role",
+ "aws_s3_region",
+ "private_endpoint_enabled",
+ "sts_private_endpoint_url",
+ "s3_private_endpoint_url"
+ ],
+ "endpointUrl": "splunk_ta_aws/splunk_ta_aws_s3buckets"
+ }
+}
+++ + + + + + +Note: When using the text type field, add debounce using the custom hook to reduce the number of API calls.
+
Using this functionality, the Inputs page form can be divided into distinct sections, each comprising relevant fields. If the isExpandable property is set to true in the global configuration file, the group will be in the collapsible panel type.
The groups will be displayed at the bottom of the form.
+label displays the title of a specific group.fields specifies the list of fields in a group. All fields must be present in the entity.options:isExpandable can be used to hide/show fields of the group. The default value is false.expand can be used to show all fields of the group while opening the form. The default value is false.{
+ "name": "aws_config",
+ "title": "Config",
+ "groups": [
+ {
+ "label": "AWS Input Configuration",
+ "options": {
+ "isExpandable": false
+ },
+ "fields": [
+ "name",
+ "aws_account"
+ ]
+ },
+ {
+ "label": "Splunk-related Configuration",
+ "options": {
+ "isExpandable": false
+ },
+ "fields": [
+ "sourcetype",
+ "index"
+ ]
+ },
+ {
+ "label": "Advanced Settings",
+ "options": {
+ "expand": false,
+ "isExpandable": true
+ },
+ "fields": [
+ "polling_interval"
+ ]
+ }
+ ],
+ "entity": []
+},
+This is how it looks in the UI:
+
UCC allows you to add Auth support in the configuration page. In UCC, OAuth2.0 of the Authorization Code Flow grant type is used. It only supports the standard parameters specified in RFCP749 for obtaining an authorization code.
Auth can be used inside the entity tag. Use type: "oauth" in the entity list and specify the options next to the type: "oauth".
type field value must be oauth.options:
auth_type must be present. It can have either [“basic”, “oauth”] (If we want basic and oauth both support) or [“oauth”] (If we want oauth support only).basic must be present only if the auth_type is [“basic”].oauth will have a list of fields for you to add in the oauth authentication flow. In the given example, it is client_id, client_secret, redirect_url, scope, and endpoint.client_id is the client id for applying auth to your app or apps.client_secret is the client secret for applying auth to your app or apps.redirect_url will show the redirect url, which needs to be put in the app’s redirect url.endpoint will be the endpoint for you to build oauth support. For example, for salesforce, it will either be “login.salesforce.com”, “test.salesforce.com”, or any other custom endpoint.endpoint_authorize specifies the endpoint used for authorization, for example, login.salesforce.com.endpoint_token specifies the endpoint used for the token acqusition, for example, api.login.salesforce.com.auth_code_endpoint must be present and its value should be the endpoint value for getting the auth_code using the app. If the url to get the auth_code is https://login.salesforce.com/services/oauth2/authorize, then this will have the value /services/oauth2/authorize.access_token_endpoint must be present and its value should be the endpoint value for getting the ccess_token using the auth_code received. If the url to get the access token is https://login.salesforce.com/services/oauth2/token, then it will have the value /services/oauth2/token.auth_label allows the user to have the custom label for the Auth Type dropdown.oauth_popup_width is the width in pixels of the pop-up window that will open for oauth authentication (Optional, defaults to 600).oauth_popup_height is the height in pixels of the pop-up window that will open for oauth authentication (Optional, defaults to 600).oauth_timeout is the timeout in seconds for oauth authentication (Optional, defaults to 180 seconds).oauth_state_enabled is used to include the state for oauth authentication (default value is false). auth_endpoint_token_access_type is an optional parameter that is mapped into the value of the token_access_type query param in the authorisation url.
The fields allowed in the basic and oauth fields are the following:
+oauth_field should be kept as it is and without any change.label can be changed if the user wants to change the label of the field in UI.field must keep it as it is for mandatory fields as mentioned before. help can be changed the if user wants to change the help text displayed below the field.encrypted should be true if the user wants that particular field encrypted, otherwise, there is no need to have this parameter.required specifies whether the field is required or not. The default value is true.defaultValue is the initial input value (string, number, or boolean).options:placeholder: The placeholder for the field.disableonEdit: When the form is in edit mode, the field becomes unable to be edited. The default value is false.enable: The enable property sets whether a field is enabled or not. The default value is true.++[!WARNING] +The Placeholder attribute is deprecated and will be removed in one of the following versions. Instead, using the “help” attribute is recommended.
+
"configuration": {
+ "title": "Configurations",
+ "description": "Configure your servers and templates.",
+ "tabs": [
+ {
+ "name": "account",
+ "title": "Account",
+ "entity": [
+ {
+ "field": "name",
+ "label": "Name",
+ "type": "text",
+ "required": true,
+ "help": "Enter a unique name for each Crowdstrike falcon host account.",
+ },
+ {
+ "type": "oauth",
+ "field": "oauth",
+ "label": "Not used",
+ "options": {
+ "auth_type": [
+ "basic",
+ "oauth"
+ ],
+ "basic": [
+ {
+ "oauth_field": "username",
+ "label": "User Name",
+ "field": "username",
+ "help": "Enter Account name."
+ },
+ {
+ "oauth_field": "password",
+ "label": "Password",
+ "field": "password",
+ "encrypted": true,
+ "help": "Enter Password."
+ },
+ {
+ "oauth_field": "security_token",
+ "label": "Securtiy Token",
+ "field": "security_token",
+ "encrypted": true,
+ "help": "Enter Security Token."
+ }
+ ],
+ "oauth": [
+ {
+ "oauth_field": "client_id",
+ "label": "Client Id",
+ "field": "client_id",
+ "help": "Enter Client Id."
+ },
+ {
+ "oauth_field": "client_secret",
+ "label": "Client Secret",
+ "field": "client_secret",
+ "encrypted": true,
+ "help": "Enter Client Secret."
+ },
+ {
+ "oauth_field": "redirect_url",
+ "label": "Redirect url",
+ "field": "redirect_url",
+ "help": "Please add this redirect URL in your app."
+ },
+ {
+ "oauth_field": "scope",
+ "label": "Scope",
+ "field": "scope",
+ "help": "Enter the scope for the authorization code with ',' separating each scope.",
+ "required": false
+ },
+ {
+ "oauth_field": "endpoint",
+ "label": "Endpoint",
+ "field": "endpoint",
+ "help": "Enter Endpoint"
+ }
+ ],
+ "auth_label": "Auth Type",
+ "oauth_popup_width": 600,
+ "oauth_popup_height": 600,
+ "oauth_timeout": 180,
+ "auth_code_endpoint": "/services/oauth2/authorize",
+ "access_token_endpoint": "/services/oauth2/token",
+ "auth_endpoint_token_access_type": "offline"
+ }
+ }
+ ],
+ }
+ ]
+}
+This is how the Add Account modal looks after adding the above code to the globalConfig.json file:
+This is how Add Account looks when the auth_type is basic:
+
This is how Add Account looks when the auth_type is oauth:
+
This feature allows you to download and unpack libraries with appropriate binaries for the indicated operating system during the build process. +To do this, you need to expand the meta section in the global configuration with the os-dependentLibraries field. This field takes the following attributes:
+| Property | +Type | +Description | +default value | +
|---|---|---|---|
| name* | +string | +is the name of the library we want to download. | +- | +
| version* | +string | +is the specific version of the given library. | +- | +
| dependencies | +boolean | +(Optional) is the parameter which determines whether the --no-deps flag will be used when installing the package from pip. When the value is set to true, the library will be installed along with all its dependencies. When the value is set to false (default) {name}={version}, it must be present in packages requirements.txt. |
+false | +
| platform* | +string | +is the platform for downloading the specified library. The value depends on the available wheels for a given library, for example, for this wheel, cryptography-41.0.5-cp37-abi3-manylinux_2_28_x86_64.whl, the platform is manylinux_2_28_x86_64. | +- | +
| python_version* | +string | +is the python version compatible with the library. | +- | +
| target* | +string | +is the Path where the selected library will be unpacked. | +- | +
| os* | +string | +is the name of the operating system which the library is intended for. Using this parameter, an appropriate insert into sys.path will be created. It takes 3 values windows, linux, and darwin. | +- | +
Generally, the wheel name convention is
{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl.
+For example, for this particular library,
grpcio-1.54.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
,
+your pip parameters are:
and your pip command should look like this:
+pip install --no-deps --platform manylinux_2_17_x86_64 --python-version 37 --target your/path/to/target --only-binary=:all: grpcio==1.54.2
A dot in the platform part indicates that a given distribution supports several platforms. +In this case, “.” in manylinux_2_17_x86_64.manylinux2014_x86_64 means this distribution supports both manylinux_2_17_x86_64 and manylinux2014_x86_64.
+For more informations, see .whl and manylinux platform.
+ ...
+ "meta": {
+ "name": "<TA name>",
+ "restRoot": "<restRoot>",
+ "version": "<TA version>",
+ "displayName": "<TA display name>",
+ "schemaVersion": "<schema version>",
+ "os-dependentLibraries": [
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "platform": "manylinux2014_x86_64",
+ "python_version": "37",
+ "os": "linux",
+ "target": "3rdparty/linux"
+ },
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "platform": "win_amd64",
+ "python_version": "37",
+ "os": "windows",
+ "target": "3rdparty/windows"
+ },
+ {
+ "name": "cryptography",
+ "version": "41.0.5",
+ "dependencies": true,
+ "platform": "manylinux2014_x86_64",
+ "python_version": "37",
+ "os": "linux",
+ "target": "3rdparty/linux_with_deps"
+ },
+ {
+ "name": "cffi",
+ "version": "1.15.1",
+ "platform": "win_amd64",
+ "python_version": "37",
+ "os": "windows",
+ "target": "3rdparty/windows"
+ }
+ ]
+ }
+Running the build for the above configuration will result in the creation of the following structure:
+output
+ └──<TA>
+ ├── bin
+ ...
+ └── lib
+ └── 3rdparty
+ ├── linux
+ │ ├── cryptography
+ │ └── cryptography-41.0.5.dist-info
+ ├── linux_with_deps
+ │ ├── _cffi_backend.cpython-37m-x86_64-linux-gnu.so
+ │ ├── cffi
+ │ ├── cffi-1.15.1.dist-info
+ │ ├── cryptography
+ │ ├── cryptography-41.0.5.dist-info
+ │ ├── pycparser
+ │ └── pycparser-2.21.dist-info
+ └── windows
+ ├── _cffi_backend.cp37-win_amd64.pyd
+ ├── cffi
+ ├── cffi-1.15.1.dist-info
+ ├── cryptography
+ └── cryptography-41.0.5.dist-info
+During the build process, a python script “import_declare_test.py” will be created in output/ta_name/bin to manipulate system paths.
+In each input using the specified libraries, this script must be imported.
+Currently, three operating systems are supported: Windows, Linux, and Darwin.
+If, for development purposes, there is a need to create other custom manipulations on sys.path,
+create your own script called “import_declare_test.py” and place it in the package/bin folder.
+This way, when building the TA, the default script will be replaced with the one created by the developer.
+The default script for this configuration will look like this:
import os
+import sys
+import re
+from os.path import dirname
+
+ta_name = 'demo_addon_for_splunk'
+pattern = re.compile(r'[\\/]etc[\\/]apps[\\/][^\\/]+[\\/]bin[\\/]?$')
+new_paths = [path for path in sys.path if not pattern.search(path) or ta_name in path]
+new_paths.insert(0, os.path.join(dirname(dirname(__file__)), "lib"))
+new_paths.insert(0, os.path.sep.join([os.path.dirname(__file__), ta_name]))
+sys.path = new_paths
+
+bindir = os.path.dirname(os.path.realpath(os.path.dirname(__file__)))
+libdir = os.path.join(bindir, "lib")
+platform = sys.platform
+if platform.startswith("linux"):
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/linux_with_deps"))
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/linux"))
+if platform.startswith("win"):
+ sys.path.insert(0, os.path.join(libdir, "3rdparty/windows"))
+This feature allows you to pass a Javascript function as a string to apply customized validation to form data.
+By using this approach, you can write custom JavaScript code where you can write your business logic, and validating can return error messages which will be displayed at the top of the form.
+This custom javascript function has a parameter, (for example, dataDict), which contains the form data object.
+This function will be called after all validators have validated the data form.
+{
+ "name": "example_input_one",
+ "title": "Example Input One",
+ "entity": [],
+ "options": {
+ "saveValidator": "function start_data_validator(dataDict) { const provided_datetime = new Date(dataDict['start_date']).getTime(); const current_datetime = new Date().getTime(); if (provided_datetime > current_datetime) { return 'Start date should not be in future'; }}"
+ }
+}
+This feature allows us to pass a broader description on the Input and Configuration pages displayed under main description.
+| Property | +Type | +Description | +
|---|---|---|
| text* | +string | +is text used for the description, you can put \n to add a breakline. | +
| links | +object | +enables including links inside description | +
| Property | +Type | +Description | +
|---|---|---|
| slug* | +string | +is used to identify the place for the link to appear. Put it inside the text, surrounded by 2 squared brackets. | +
| link* | +string | +is the link to be used. | +
| linkText* | +string | +is the text to be inserted, instead of slug. | +
{
+ "name": "example_input_one",
+ "title": "Example Input One",
+ "entity": [],
+ "subDescription": {
+ "text": "Ingesting data from to Splunk Cloud?\nRead our [[blogPost]] to learn more about Data Manager and it's availability on your Splunk Cloud instance.",
+ "links": [
+ {
+ "slug": "blogPost",
+ "link": "https://splk.it/31oy2b2",
+ "linkText": "blog post"
+ }
+ ]
+ }
+}
+