Wip pre deployment bug cleanup

README.md

@@ -51,41 +51,94 @@ version.
 
 ##### Service Architecture
 
-The `perseus` service exposes a gRPC API, along with JSON/REST mappings using the [gRPC Gateway](https://github.com/grpc-ecosystem/grpc-gateway)
-project for exposure to web-based consumers.  Both endpoints, plus a very basic web UI for testing,
-are served on a single port using [cmux](https://github.com/soheilhy/cmux).
+The `perseus` service exposes a gRPC API, along with JSON/REST mappings using the [gRPC Gateway](https://github.com/grpc-ecosystem/grpc-gateway) project for exposure to web-based consumers via a set of paths under `/api/v1/*`.  Both endpoints, plus a very basic web UI for testing, are served on a single port using [cmux](https://github.com/soheilhy/cmux).
 
-##### Service Operations
+In addition to the interactive endpoints, the service also exports an HTTP health check at `/healthz/` and basic Prometheus metrics at `/metrics/`.  For debugging and troubleshooting, the service supports retrieving [Go `pprof` data](https://pkg.go.dev/net/http/pprof) via HTTP at `/debug/pprof*`.
 
-Updates to the graph are made by submitting a request with the name and version of a module, along
-with a list of its dependencies (name and version).  This request is converted into a set of "edges"
-in the graph linking the name/version pairs.  Since tagged module versions are frozen in Go, any
-existing data for the submitted module is first removed.
+#### Running the Service
 
-Example update:
+For simplicity, we publish a pre-built Docker image (based on a `scratch` base) to the GitHub Container Registry when each release is tagged.  The image runs `perseus server` and exposes port 31138, which you can map to whatever is appropriate for your environment.  You will need to provide the URL location of the Perseus database, along with a valid username and password to connect, via environment variables.  The default database name is "perseus", but you can override that by also providing a `DB_NAME` environment variable.
 
-    curl --request POST http://localhost/api/v1/update-module-dependencies?module_name=github.com/example/foo&version=v1.2.3
-       --header 'Content-Type: application/json'
-       --data-raw '{"dependencies":[{"name":"github.com/rs/zerolog","versions:["v1.27.0"]}, {"name":"golang.org/x/text","versions":["v0.0.0-...."]}'
+    > docker run --rm -e DB_ADDR=... -e DB_USER=... -e DBPASS=... -p 31138:31138 ghcr.io/crowdstrike/perseus:latest
 
-Queries consist of a target module, an optional version, and a direction (ancestors or descendants).
-An "ancestor" query returns the set of modules that the target module depends on and a "descendant"
-query returns the set of modules that depend on the target.  In either case, if no version is specified
-for the target, the most recent known version is used.
+This example uses the `latest` tag but you should always reference a specific version for stability.
 
-Example query:
+We also generate pre-built binaries for Windows, Linux, and Mac that can be downloaded from [the releases page](https://github.com/CrowdStrike/perseus/releases).
 
-    curl --request GET http://localhost/api/v1/module-dependencies?module_name=github.com/example/foo&version=v1.2.3&direction=descendants
+#### The `perseus` CLI
 
-The format of the response is either:
+The `perseus` binary is both the server and a CLI tool for interacting with it.  The `update`
+and `query` commands are the workhorses as a client.  The Perseus server address can be specified
+by either setting the `PERSEUS_SERVER_ADDR` environment variable or passing it directly to the CLI
+using the `--server-addr` flag.
 
-- a JSON document containing the graph of dependencies as nested objects
-- a DOT file describing the graph
-- a rendered PNG or SVG image of the graph
+`perseus update` analyzes a Go module, on disk or available via public Go module proxies, and adds it
+to the Perseus graph.  For a module on disk that is a Git repository, the CLI will try to infer the
+version by looking at the Git tags on the current commit.  If there is exactly 1 module version tag,
+that version will be used.  If there is no semver tag or there are multiple, you will need to specify
+a version.
 
-#### Updating the Graph
+    # process the current version of the perseus module on disk
+    > perseus update --path ~/code/github.com/CrowdStrike/perseus
 
-The `perseus` binary is both the server and a CLI tool for interacting with it.  When used as a CLI, `perseus update ...` extracts the direct dependencies of a specified Go module and updates the Perseus graph.  By scripting executions of `perseus update` across the codebase, including as a CI step when new module versions are released, the Perseus graph can be incrementally grown to include all of the information necessary to query both ancestors and descendants of any particular version of a module.
+    # process a specific version of example/foo
+    > perseus update --path ~/code/github.com/example/foo -version v1.2.3
+
+For public modules a version must always be specified.
+
+    > perseus update --module github.com/example/foo --version v1.2.3
+
+Once you have data in your graph, `perseus query` is the way to retrieve it.  There are 4 available
+sub-commands: `list-modules`, `list-module-versions`, `ancestors`, and `descendants`.
+
+The first two commands return modules and versions based on glob pattern matches:
+
+    # list all modules under the github.com/example organization along with the highest version
+    > perseus query list-modules 'github.com/example/*' --list
+    Module                  Version
+    github.com/example/foo  v1.2.0
+    github.com/example/bar  v1.1.0
+    github.com/example/baz  v1.17.23
+
+    # list all versions of github.com/example/foo using an explicit format
+    > perseus query list-module-versions github.com/example/foo --format 'module {{.Path}} has version {{.Version}}.'
+    module github.com/example/foo has version v1.2.0
+    module github.com/example/foo has version v1.1.0
+    ...
+
+The `ancestors` and `descendants` commands walk the graph to return dependency trees, either what a specified version of a module depends on or what modules depend on it.
+
+    # show the modules that v1.2.0 of github.com/example/foo depends on as a tabular list
+    > perseus query ancestors github.com/example/foo@v1.2.0 --list
+    Dependency                                            Direct
+    github.com/pkg/errors@v0.9.1                          true
+    golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c  true
+    ...
+
+    # show the modules that depend on v0.9.1 of github.com/pkg/errors as nested JSON, using jq to format it nicely
+    > perseus query descendants github.com/pkg/errors@v0.9.1 --json | jq .
+    {
+        "module": {
+            "Path":    "github.com/pkg/errors",
+            "Version": "v0.9.1"
+        },
+        "deps": [
+            {
+                "module": {
+                    "Path":    "github.com/example/foo",
+                    "Version": "v1.2.0"
+                },
+                ...
+            }
+        ]
+    }
+
+In addition to text-based results using `--json`, `--list` or `--format`, the `ancestor` and `descendant` commands also supports outputting DOT directed graphs using the `--dot` flag.
+
+    # generate an SVG image of the dependency graph for the highest version of github.com/example/foo
+    > perseus query ancestors github.com/example/foo --dot | dot -Tsvg -o ~/foo_deps.svg
+
+The depth of the tree can be controlled by the `--max-depth` flag, with a default of 4 hops.
 
 <hr/>
 

client_config.go

@@ -36,7 +36,7 @@ func withServerAddress(addr string) clientOption {
 func readClientConfigEnv() []clientOption {
 	var opts []clientOption
 
-	if addr := os.Getenv("SERVER_ADDR"); addr != "" {
+	if addr := os.Getenv("PERSEUS_SERVER_ADDR"); addr != "" {
 		opts = append(opts, withServerAddress(addr))
 	}
 

go.mod

@@ -5,10 +5,12 @@ go 1.18
 require (
 	github.com/Masterminds/squirrel v1.5.3
 	github.com/go-git/go-git/v5 v5.6.0
+	github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
 	github.com/grpc-ecosystem/grpc-gateway/v2 v2.15.2
 	github.com/jackc/pgx/v4 v4.18.1
 	github.com/jmoiron/sqlx v1.3.5
 	github.com/mattn/go-isatty v0.0.17
+	github.com/prometheus/client_golang v1.14.0
 	github.com/soheilhy/cmux v0.1.5
 	github.com/spf13/cobra v1.6.1
 	github.com/spf13/pflag v1.0.5
@@ -28,6 +30,8 @@ require (
 	github.com/Microsoft/go-winio v0.5.2 // indirect
 	github.com/ProtonMail/go-crypto v0.0.0-20221026131551-cf6655e29de4 // indirect
 	github.com/acomagu/bufpipe v1.0.3 // indirect
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/cloudflare/circl v1.1.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/emirpasic/gods v1.18.1 // indirect
@@ -39,8 +43,12 @@ require (
 	github.com/kevinburke/ssh_config v1.2.0 // indirect
 	github.com/mattn/go-colorable v0.1.12 // indirect
 	github.com/mattn/go-runewidth v0.0.13 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
 	github.com/pjbgf/sha1cd v0.3.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/prometheus/client_model v0.3.0 // indirect
+	github.com/prometheus/common v0.37.0 // indirect
+	github.com/prometheus/procfs v0.8.0 // indirect
 	github.com/rivo/uniseg v0.2.0 // indirect
 	github.com/sergi/go-diff v1.1.0 // indirect
 	github.com/skeema/knownhosts v1.1.0 // indirect