[receiver/expvarreceiver] Overall structure for new expvarreceiver (#…

…9747)

* initial configs and skeleton for expvarreceiver

* make goporto

* add changelog entry

* add jamesmoessis and moviestoreguy as codeowners for expvarreceiver

* correct versions

* add expvarreceiver to versions.yaml

* switch to using scraperhelper for config settings

* go mod tidy

* Update receiver/expvarreceiver/Makefile

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>

* cleanup todos

* squash the http client config as per review

* add changelog entry

* update collector dependency version

Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com>

.github/CODEOWNERS

@@ -120,6 +120,7 @@ receiver/couchdbreceiver/                            @open-telemetry/collector-c
 receiver/dockerstatsreceiver/                        @open-telemetry/collector-contrib-approvers @rmfitzpatrick
 receiver/dotnetdiagnosticsreceiver/                  @open-telemetry/collector-contrib-approvers @pmcollins @davmason
 receiver/elasticsearchreceiver/                      @open-telemetry/collector-contrib-approvers @djaglowski @binaryfissiongames
+receiver/expvarreceiver/                             @open-telemetry/collector-contrib-approvers @jamesmoessis @MovieStoreGuy
 receiver/filelogreceiver/                            @open-telemetry/collector-contrib-approvers @djaglowski
 receiver/fluentforwardreceiver/                      @open-telemetry/collector-contrib-approvers @dmitryax
 receiver/googlecloudpubsubreceiver/                  @open-telemetry/collector-contrib-approvers @alexvanboxel

CHANGELOG.md

@@ -8,6 +8,8 @@
 
 ### 🚀 New components 🚀
 
+- `expvarreceiver`: Initial work for a receiver designed to scrape `memstats` from Golang applications. (#9747)
+
 ### 💡 Enhancements 💡
 
 ### 🧰 Bug fixes 🧰

receiver/expvarreceiver/Makefile

@@ -0,0 +1 @@
+include ../../Makefile.Common

receiver/expvarreceiver/README.md

@@ -0,0 +1,48 @@
+# Expvar Receiver
+
+An Expvar Receiver scrapes metrics from [expvar](https://pkg.go.dev/expvar), 
+which exposes data in JSON format from an HTTP endpoint. The metrics are 
+extracted from the `expvar` variable [memstats](https://pkg.go.dev/runtime#MemStats), 
+which exposes various information about the Go runtime.
+
+> :construction: This receiver is in development and incomplete. It should not be used yet.
+
+## Configuration 
+
+### Default
+
+By default, without any configuration, a request will be sent to `http://localhost:8080/debug/vars` 
+every 60 seconds. The default configuration is achieved by the following:
+
+```yaml
+receivers:
+  expvar:
+```
+
+### Customising
+
+The following can be configured:
+- Configure the HTTP client for scraping the expvar variables. The full set of
+  configuration options for the client can be found in the core repo's
+  [confighttp](https://github.com/open-telemetry/opentelemetry-collector/tree/main/config/confighttp#client-configuration).
+  - defaults: 
+    - `endpoint = http://localhost:8080/debug/vars` 
+    - `timeout = 3s`
+- `collection_interval` - Configure how often the metrics are scraped.
+  - default: 1m
+- `metrics` - Enable or disable metrics by name.
+
+### Example configuration
+
+```yaml
+receivers:
+  expvar:
+    endpoint: "http://localhost:8000/custom/path"
+    timeout: 1s
+    collection_interval: 30s
+    metrics:
+      - name: example_metric.enabled
+        enabled: true
+      - name: example_metric.disabled
+        enabled: false
+```

receiver/expvarreceiver/config.go

@@ -0,0 +1,54 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expvarreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/expvarreceiver"
+
+import (
+	"fmt"
+	"net/url"
+
+	"go.opentelemetry.io/collector/config"
+	"go.opentelemetry.io/collector/config/confighttp"
+	"go.opentelemetry.io/collector/receiver/scraperhelper"
+)
+
+type Config struct {
+	scraperhelper.ScraperControllerSettings `mapstructure:",squash"`
+	HTTP                                    *confighttp.HTTPClientSettings `mapstructure:",squash"`
+	MetricsConfig                           []MetricConfig                 `mapstructure:"metrics"`
+}
+
+type MetricConfig struct {
+	Name    string `mapstructure:"name"`
+	Enabled bool   `mapstructure:"enabled"`
+}
+
+var _ config.Receiver = (*Config)(nil)
+
+func (c *Config) Validate() error {
+	if c.HTTP == nil {
+		return fmt.Errorf("must specify http_client configuration when using expvar receiver")
+	}
+	u, err := url.Parse(c.HTTP.Endpoint)
+	if err != nil {
+		return fmt.Errorf("endpoint is not a valid URL: %v", err)
+	}
+	if u.Host == "" {
+		return fmt.Errorf("host not found in HTTP endpoint")
+	}
+	if u.Scheme != "http" && u.Scheme != "https" {
+		return fmt.Errorf("scheme must be 'http' or 'https', but was '%s'", u.Scheme)
+	}
+	return nil
+}

receiver/expvarreceiver/config_test.go

@@ -0,0 +1,68 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expvarreceiver
+
+import (
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/component/componenttest"
+	"go.opentelemetry.io/collector/config"
+	"go.opentelemetry.io/collector/config/confighttp"
+	"go.opentelemetry.io/collector/service/servicetest"
+)
+
+func TestLoadConfig(t *testing.T) {
+	factories, err := componenttest.NopFactories()
+	assert.NoError(t, err)
+
+	factory := NewFactory()
+	factories.Receivers[typeStr] = factory
+	cfg, err := servicetest.LoadConfigAndValidate(filepath.Join("testdata", "config.yaml"), factories)
+
+	require.NoError(t, err)
+	require.NotNil(t, cfg)
+	assert.Equal(t, 2, len(cfg.Receivers))
+
+	// Validate default config
+	expectedCfg := factory.CreateDefaultConfig().(*Config)
+	expectedCfg.SetIDName("default")
+
+	assert.Equal(t, expectedCfg, cfg.Receivers[config.NewComponentIDWithName(typeStr, "default")])
+
+	// Validate custom config
+	expectedCfg = factory.CreateDefaultConfig().(*Config)
+	expectedCfg.SetIDName("custom")
+	expectedCfg.CollectionInterval = time.Second * 30
+	expectedCfg.HTTP = &confighttp.HTTPClientSettings{
+		Endpoint: "http://localhost:8000/custom/path",
+		Timeout:  time.Second * 5,
+	}
+	expectedCfg.MetricsConfig = []MetricConfig{
+		{
+			Name:    "example_metric.enabled",
+			Enabled: true,
+		},
+		{
+			Name:    "example_metric.disabled",
+			Enabled: false,
+		},
+	}
+
+	assert.Equal(t, expectedCfg, cfg.Receivers[config.NewComponentIDWithName(typeStr, "custom")])
+}

receiver/expvarreceiver/factory.go

@@ -0,0 +1,60 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expvarreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/expvarreceiver"
+
+import (
+	"context"
+	"fmt"
+	"time"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/config"
+	"go.opentelemetry.io/collector/config/confighttp"
+	"go.opentelemetry.io/collector/consumer"
+	"go.opentelemetry.io/collector/receiver/scraperhelper"
+)
+
+const (
+	typeStr         = "expvar"
+	defaultEndpoint = "http://localhost:8000/debug/vars"
+	defaultTimeout  = 3 * time.Second
+)
+
+func NewFactory() component.ReceiverFactory {
+	return component.NewReceiverFactory(
+		typeStr,
+		newDefaultConfig,
+		component.WithMetricsReceiver(newMetricsReceiver))
+}
+
+func newMetricsReceiver(
+	ctx context.Context,
+	settings component.ReceiverCreateSettings,
+	rCfg config.Receiver,
+	metrics consumer.Metrics,
+) (component.MetricsReceiver, error) {
+	return nil, fmt.Errorf("not implemented")
+}
+
+func newDefaultConfig() config.Receiver {
+	return &Config{
+		ScraperControllerSettings: scraperhelper.NewDefaultScraperControllerSettings(typeStr),
+		HTTP: &confighttp.HTTPClientSettings{
+			Endpoint: defaultEndpoint,
+			Timeout:  defaultTimeout,
+		},
+		MetricsConfig: []MetricConfig{},
+	}
+}

receiver/expvarreceiver/go.mod

@@ -0,0 +1,45 @@
+module github.com/open-telemetry/opentelemetry-collector-contrib/receiver/expvarreceiver
+
+go 1.17
+
+require (
+	github.com/stretchr/testify v1.7.1
+	go.opentelemetry.io/collector v0.51.0
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/felixge/httpsnoop v1.0.2 // indirect
+	github.com/go-logr/logr v1.2.3 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/golang/snappy v0.0.4 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/klauspost/compress v1.15.3 // indirect
+	github.com/knadh/koanf v1.4.1 // indirect
+	github.com/mitchellh/copystructure v1.2.0 // indirect
+	github.com/mitchellh/mapstructure v1.5.0 // indirect
+	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/rs/cors v1.8.2 // indirect
+	go.opencensus.io v0.23.0 // indirect
+	go.opentelemetry.io/collector/pdata v0.51.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.32.0 // indirect
+	go.opentelemetry.io/otel v1.7.0 // indirect
+	go.opentelemetry.io/otel/metric v0.30.0 // indirect
+	go.opentelemetry.io/otel/trace v1.7.0 // indirect
+	go.uber.org/atomic v1.9.0 // indirect
+	go.uber.org/multierr v1.8.0 // indirect
+	go.uber.org/zap v1.21.0 // indirect
+	golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
+	golang.org/x/sys v0.0.0-20220128215802-99c3d69c2c27 // indirect
+	golang.org/x/text v0.3.7 // indirect
+	google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa // indirect
+	google.golang.org/grpc v1.46.0 // indirect
+	google.golang.org/protobuf v1.28.0 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
+)