From d3fd08b9f9919269803feb8dfc16365d5201babc Mon Sep 17 00:00:00 2001 From: Ben Moss Date: Thu, 3 Jan 2019 12:29:26 -0500 Subject: [PATCH 1/2] Create KEP for Windows Node Support Adds a KEP covering Windows support and a sig-windows directory for it to live in. --- .../0000-20190103-windows-node-support.md | 112 ++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 keps/sig-windows/0000-20190103-windows-node-support.md diff --git a/keps/sig-windows/0000-20190103-windows-node-support.md b/keps/sig-windows/0000-20190103-windows-node-support.md new file mode 100644 index 00000000000..77355c1251d --- /dev/null +++ b/keps/sig-windows/0000-20190103-windows-node-support.md @@ -0,0 +1,112 @@ +--- +kep-number: 0 +title: Windows node support +authors: + - "@benmoss" + - "@astrieanna" +owning-sig: sig-windows +participating-sigs: + - sig-architecture +reviewers: + - TBD +approvers: + - TBD +editor: TBD +creation-date: 2018-11-29 +last-updated: 2019-01-03 +status: provisional +--- + +# Windows node support + + +## Table of Contents + + * [Windows node support](#windows-node-support) + * [Table of Contents](#table-of-contents) + * [Summary](#summary) + * [Motivation](#motivation) + * [Goals](#goals) + * [Non-Goals](#non-goals) + * [Proposal](#proposal) + * [What works today](#what-works-today) + * [What will work eventually](#what-will-work-eventually) + * [What will never work (without underlying OS changes)](#what-will-never-work-without-underlying-os-changes) + * [Relevant resources/conversations](#relevant-resourcesconversations) + * [Risks and Mitigations](#risks-and-mitigations) + * [Graduation Criteria](#graduation-criteria) + * [Implementation History](#implementation-history) + * [Other references](#other-references) + + +## Summary + +There is strong interest in the community for adding support for workloads running on Microsoft Windows. This is non-trivial due to the significant differences in the implementation of Windows from the Linux-based OSes that have so far been supported by Kubernetes. + + +## Motivation + +Windows-native workloads still account for a significant portion of the enterprise software space. While containerization technologies emerged first in the UNIX ecosystem, Microsoft has made investments in recent years to enable support for containers in its Windows OS. As users of Windows increasingly turn to containers as the preferred abstraction for running software, the Kubernetes ecosystem stands to benefit by becoming a cross-platform cluster manager. + +### Goals + +- Enable users to run nodes on Windows servers +- Document the differences and limitations compared to Linux +- Test results added to testgrid to prevent regression of functionality + +### Non-Goals + +- Adding Windows support to all projects in the Kubernetes ecosystem (Cluster Lifecycle, etc) + +## Proposal + +As of 29-11-2018 much of the work for enabling Windows nodes has already been completed. Both `kubelet` and `kube-proxy` have been adapted to work on Windows Server, and so the first goal of this KEP is largely already complete. + +### What works today +- Windows-based containers can be created by kubelet, [provided the host OS version matches the container base image](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/version-compatibility) + - ConfigMap, Secrets: as environment variables or volumes + - Resource limits + - Pod & container metrics +- Pod networking with [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md), [OVN-Kubernetes](https://github.com/openvswitch/ovn-kubernetes), [two CNI meta-plugins](https://github.com/containernetworking/plugins), [Flannel](https://github.com/coreos/flannel) and [Calico](https://github.com/projectcalico/calico) +- Dockershim CRI +- Many[1] of the e2e conformance tests when run with [alternate Windows-based images](https://hub.docker.com/r/e2eteam/) which are being moved to [kubernetes-sigs/windows-testing](https://www.github.com/kubernetes-sigs/windows-testing) +- Persistent storage: FlexVolume with [SMB + iSCSI](https://github.com/Microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows), and in-tree AzureFile and AzureDisk providers + +1 This list should be available at https://k8s-testgrid.appspot.com/sig-windows but this test setup is not currently working. https://k8s-testgrid.appspot.com/google-windows#windows-prototype is also running against a Windows cluster. + +### What will work eventually +- `kubectl port-forward` hasn't been implemented due to lack of an `nsenter` equivalent to run a process inside a network namespace. +- CRIs other than Dockershim: CRI-containerd support is forthcoming + + +### What will never work (without underlying OS changes) +- Certain Pod functionality + - Privileged containers + - Reservations are not enforced by the OS, but overprovisioning could be blocked with `--enforce-node-allocatable=pods` (pending: tests needed) +- CSI plugins, which require privileged containers +- [Some parts of the V1 API](https://github.com/kubernetes/kubernetes/issues/70604) +- Overlay networking support in Windows Server 1803 is not fully functional using the `win-overlay` CNI plugin. Specifically service IPs do not work on Windows nodes. This is currently specific to `win-overlay` - other CNI plugins (OVS, AzureCNI) work. + +### Relevant resources/conversations + +- [sig-architecture thread](https://groups.google.com/forum/#!topic/kubernetes-sig-architecture/G2zKJ7QK22E) +- [cncf-k8s-conformance thread](https://lists.cncf.io/g/cncf-k8s-conformance/topic/windows_conformance_tests/27913232) +- [kubernetes/enhancements proposal](https://github.com/kubernetes/features/issues/116) + + +### Risks and Mitigations + +**Second class support**: Kubernetes contributors are likely to be thinking of Linux-based solutions to problems, as Linux remains the primary OS supported. Keeping Windows support working will be an ongoing burden potentially limiting the pace of development. + +**User experience**: Users today will need to use some combination of taints and node selectors in order to keep Linux and Windows workloads separated. In the best case this imposes a burden only on Windows users, but this is still less than ideal. + +## Graduation Criteria + + +## Implementation History + + + +## Other references + +[Past release proposal for v1.12/13](https://docs.google.com/document/d/1YkLZIYYLMQhxdI2esN5PuTkhQHhO0joNvnbHpW68yg8/edit#) From 2ca30d12b35ef238afb23145da7d9910448b67b6 Mon Sep 17 00:00:00 2001 From: Leah Hanson Date: Wed, 9 Jan 2019 18:00:39 -0500 Subject: [PATCH 2/2] Add owners file Signed-off-by: Ben Moss --- keps/sig-windows/OWNERS | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 keps/sig-windows/OWNERS diff --git a/keps/sig-windows/OWNERS b/keps/sig-windows/OWNERS new file mode 100644 index 00000000000..78470a912b9 --- /dev/null +++ b/keps/sig-windows/OWNERS @@ -0,0 +1,6 @@ +reviewers: + - sig-windows-leads +approvers: + - sig-windows-leads +labels: + - sig/windows