From 17fb66c02b8308000750a47a972675d2e0646900 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Wed, 6 Jul 2022 15:50:47 +0200 Subject: [PATCH 1/2] Add proposal: live-migration-for-bridged-pod-network.md Signed-off-by: Andrei Kvapil --- .../live-migration-for-bridged-pod-network.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 design-proposals/live-migration-for-bridged-pod-network.md diff --git a/design-proposals/live-migration-for-bridged-pod-network.md b/design-proposals/live-migration-for-bridged-pod-network.md new file mode 100644 index 00000000..959e9f0d --- /dev/null +++ b/design-proposals/live-migration-for-bridged-pod-network.md @@ -0,0 +1,52 @@ +# Overview +This feature enables support of live-migrate for VMs with pod network in bridge and macvtap modes. + +## Motivation +Masquerade and slirp are not so performant compared to bridge and macvtap modes. +We want to use the less latency modes with the opportunity to live-migrate the virtual machines. + +## Goals +Provide a live-migration feature for VMs running with pod netowork connected in bridged mode and macvtap mode. + +## Non Goals +The live-migration of virtual machines between the pods with different MAC address will invoke NIC reattaching procedure. This might affect some applications which are binding to the specific interface inside the VM. + +The live-migration of virtual machines between the pods with same MAC address will invoke the procedure to link down and up for the VM to renew DHCP lease with IP address and routes inside the VM. This is less destructive operation, but still may affect some workloads. + +## Definition of Users +Everyone who use bridge to bind pod network may want to live-migrate created VMs. + +## User Stories +* As a user / admin, I want to have an opportunity for live-migration of a VM with bridged pod-network. + +## Repos +- [KubeVirt](https://github.com/kubevirt/kubevirt) + +# Design + +To add two additional methods into live-migration procedure: +- If MAC address changed: detach / attach interface after live migration to have correct MAC address set +- If MAC address is not changed: link down / link up interface to force VM request new IP address and routes from DHCP + +## API Examples +There are no API changes from the user side. + +## Scalability +I don't see any scalability issues. + +## Update/Rollback Compatibility +This change adds new logic also for multus connected networks. +Because, multus can also be used to bind standard CNIs (eg. flannel), which does allow to preserve IP over the nodes. +The live-migration of such VMs were not handle network reconfiguration before, now it will be handled by the same procedure. + +## Functional Testing Approach +- Create two VMs: client and server in bridge mode +- Wait for lunch +- Get IP address for server VM +- Run ping from client to server +- live-migrate client VM +- Run ping from client to server + +# Implementation Phases +Implementation is already prepared as single pull request: +https://github.com/kubevirt/kubevirt/pull/7768 From f8fc6175f3f2586e5c44a7bf1ba775676ed554c2 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Fri, 8 Jul 2022 17:31:55 +0200 Subject: [PATCH 2/2] Apply suggestions from code review Co-authored-by: Miguel Duarte Barroso Co-authored-by: Ryan Hallisey Signed-off-by: Andrei Kvapil --- .../live-migration-for-bridged-pod-network.md | 28 +++++++++++-------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/design-proposals/live-migration-for-bridged-pod-network.md b/design-proposals/live-migration-for-bridged-pod-network.md index 959e9f0d..14a55248 100644 --- a/design-proposals/live-migration-for-bridged-pod-network.md +++ b/design-proposals/live-migration-for-bridged-pod-network.md @@ -1,17 +1,19 @@ -# Overview -This feature enables support of live-migrate for VMs with pod network in bridge and macvtap modes. +# Live migration for bridged pod network + +Author: Andrei Kvapil \ + +## Overview +This feature enables support of live-migrate for VMs with pod network in bridge mode. ## Motivation -Masquerade and slirp are not so performant compared to bridge and macvtap modes. +Masquerade and slirp are not so performant compared to bridge mode. We want to use the less latency modes with the opportunity to live-migrate the virtual machines. ## Goals -Provide a live-migration feature for VMs running with pod netowork connected in bridged mode and macvtap mode. +Provide a live-migration feature for VMs running with pod network connected in bridged mode. ## Non Goals -The live-migration of virtual machines between the pods with different MAC address will invoke NIC reattaching procedure. This might affect some applications which are binding to the specific interface inside the VM. - -The live-migration of virtual machines between the pods with same MAC address will invoke the procedure to link down and up for the VM to renew DHCP lease with IP address and routes inside the VM. This is less destructive operation, but still may affect some workloads. +Live migration with network reconfiguration is not seamless, sometimes it can take some time for applying a new settings. ## Definition of Users Everyone who use bridge to bind pod network may want to live-migrate created VMs. @@ -24,10 +26,15 @@ Everyone who use bridge to bind pod network may want to live-migrate created VMs # Design -To add two additional methods into live-migration procedure: +To add a new feature gate `NetworkAwareLiveMigration`, which enables two additional methods in live-migration procedure: - If MAC address changed: detach / attach interface after live migration to have correct MAC address set + + The live-migration of virtual machines between the pods with different MAC address will invoke NIC reattaching procedure. This may affect some applications which are binding to the specific interface inside the VM. + - If MAC address is not changed: link down / link up interface to force VM request new IP address and routes from DHCP + The live-migration of virtual machines between the pods with same MAC address will invoke the procedure to link down and up for the VM to renew DHCP lease with IP address and routes inside the VM. This is less destructive operation, but still may affect some workloads listening on particular IP addresses. + ## API Examples There are no API changes from the user side. @@ -36,12 +43,11 @@ I don't see any scalability issues. ## Update/Rollback Compatibility This change adds new logic also for multus connected networks. -Because, multus can also be used to bind standard CNIs (eg. flannel), which does allow to preserve IP over the nodes. -The live-migration of such VMs were not handle network reconfiguration before, now it will be handled by the same procedure. +When using multiple NICs, there are some CNIs that didn't work with live-migration. Now these CNIs will work. ## Functional Testing Approach - Create two VMs: client and server in bridge mode -- Wait for lunch +- Wait for launch - Get IP address for server VM - Run ping from client to server - live-migrate client VM