diff --git a/LICENSE b/LICENSE index 34a85897c882f..81899d0fee17d 100644 --- a/LICENSE +++ b/LICENSE @@ -248,6 +248,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt. (BSD 3 License) d3 v5.16.0 (https://d3js.org) (BSD 3 License) d3-shape v2.1.0 (https://github.com/d3/d3-shape) + (BSD 3 License) cgroupspy 0.2.1 (https://github.com/cloudsigma/cgroupspy) ======================================================================== See licenses/LICENSES-ui.txt for packages used in `/airflow/www` diff --git a/airflow/_vendor/README.md b/airflow/_vendor/README.md new file mode 100644 index 0000000000000..e708f1e507160 --- /dev/null +++ b/airflow/_vendor/README.md @@ -0,0 +1,37 @@ +# Vendor package + +## What vendored packages are for + +The `airflow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves +because authors of the packages do not have time to modify them themselves. This is often temporary +and once the packages implement fixes that we need, and then we remove the packages from +the `_vendor` package. + +All Vendored libraries must follow these rules: + +1. Vendored libraries must be pure Python--no compiling (so that we do not have to release multi-platform airflow packages on PyPI). +2. Source code for the libary is included in this directory. +3. License must be included in this repo and in the [LICENSE](../../LICENSE) file and in the + [licenses](../../licenses) folder. +4. Requirements of the library become requirements of airflow core. +5. Version of the library should be included in the [vendor.md](vendor.md) file. +6. No modifications to the library may be made in the initial commit. +7. Apply the fixes necessary to use the vendored library as separate commits - each package separately, + so that they can be cherry-picked later if we upgrade the vendored package. Changes to airflow code to + use the vendored packages should be applied as separate commits/PRs. +8. The `_vendor` packages should be excluded from any refactorings, static checks and automated fixes. + +## Adding and upgrading a vendored package + +Way to vendor a library or update a version: + +1. Update ``vendor.txt`` with the library, version, and SHA256 (`pypi` provides hashes as of recently) +2. Remove all old files and directories of the old version. +3. Replace them with new files (only replace relevant python packages:move LICENSE ) + * move licence files to [licenses](../../licenses) folder + * remove README and any other supporting files (they can be found in PyPI) + * make sure to add requirements from setup.py to airflow's setup.py with appropriate comment stating + why the requirements are added and when they should be removed +4. If you replace previous version, re-apply historical fixes from the "package" folder by + cherry-picking them. + diff --git a/airflow/_vendor/cgroupspy/__init__.py b/airflow/_vendor/cgroupspy/__init__.py new file mode 100644 index 0000000000000..1c55e2e68fd1b --- /dev/null +++ b/airflow/_vendor/cgroupspy/__init__.py @@ -0,0 +1,27 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CloudSigma AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +__version__ = "0.2.1" diff --git a/airflow/_vendor/cgroupspy/contenttypes.py b/airflow/_vendor/cgroupspy/contenttypes.py new file mode 100644 index 0000000000000..f18b556745719 --- /dev/null +++ b/airflow/_vendor/cgroupspy/contenttypes.py @@ -0,0 +1,155 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CLOUDSIGMA AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" + + +class BaseContentType(object): + + def __str__(self): + raise NotImplementedError("Please implement this method in subclass") + + def __repr__(self): + return "<{self.__class__.__name__}: {self}>".format(self=self) + + @classmethod + def from_string(cls, value): + raise NotImplementedError("This method should return an instance of the content type") + + +class DeviceAccess(BaseContentType): + TYPE_ALL = "all" + TYPE_CHAR = "c" + TYPE_BLOCK = "b" + + ACCESS_UNSPEC = 0 + ACCESS_READ = 1 + ACCESS_WRITE = 2 + ACCESS_MKNOD = 4 + + def __init__(self, dev_type=None, major=None, minor=None, access=None): + self.dev_type = dev_type or self.TYPE_ALL + + # the default behaviour of device access cgroups if unspecified is as follows + if major is not None: + self.major = int(major) + else: + self.major = "*" + + if minor is not None: + self.minor = int(minor) + else: + self.minor = "*" + + if isinstance(access, str): + value = 0 + if 'r' in access: + value |= self.ACCESS_READ + if 'w' in access: + value |= self.ACCESS_WRITE + if 'm' in access: + value |= self.ACCESS_MKNOD + self.access = value + else: + self.access = access or (self.ACCESS_READ | self.ACCESS_WRITE | self.ACCESS_MKNOD) + + def _check_access_bit(self, offset): + mask = 1 << offset + return self.access & mask + + @property + def can_read(self): + return self._check_access_bit(0) == self.ACCESS_READ + + @property + def can_write(self): + return self._check_access_bit(1) == self.ACCESS_WRITE + + @property + def can_mknod(self): + return self._check_access_bit(2) == self.ACCESS_MKNOD + + @property + def access_string(self): + accstr = "" + if self.can_read: + accstr += "r" + if self.can_write: + accstr += "w" + if self.can_mknod: + accstr += "m" + return accstr + + def __str__(self): + return "{self.dev_type} {self.major}:{self.minor} {self.access_string}".format(self=self) + + def __eq__(self, other): + return self.dev_type == other.dev_type and self.major == other.major \ + and self.minor == other.minor and self.access_string == other.access_string + + @classmethod + def from_string(cls, value): + dev_type, major_minor, access_string = value.split() + major, minor = major_minor.split(":") + major = int(major) if major != "*" else None + minor = int(minor) if minor != "*" else None + + access_mode = 0 + for idx, char in enumerate("rwm"): + if char in access_string: + access_mode |= (1 << idx) + return cls(dev_type, major, minor, access_mode) + + +class DeviceThrottle(BaseContentType): + + def __init__(self, limit, major=None, minor=None, ): + self.limit = limit + + if major is not None and major != '*': + self.major = int(major) + else: + self.major = '*' + + if minor is not None and minor != '*': + self.minor = int(minor) + else: + self.minor = '*' + + def __eq__(self, other): + return self.limit == other.limit and self.major == other.major and self.minor == other.minor + + def __str__(self): + return "{self.major}:{self.minor} {self.limit}".format(self=self) + + @classmethod + def from_string(cls, value): + try: + major_minor, limit = value.split() + major, minor = major_minor.split(":") + return cls(int(limit), major, minor) + except Exception: + raise RuntimeError("Value {} cannot be converted to a string that matches the pattern: " + "[device major]:[device minor] [throttle limit in bytes]".format(value)) diff --git a/airflow/_vendor/cgroupspy/controllers.py b/airflow/_vendor/cgroupspy/controllers.py new file mode 100644 index 0000000000000..6685c625bce10 --- /dev/null +++ b/airflow/_vendor/cgroupspy/controllers.py @@ -0,0 +1,324 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CloudSigma AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +import os +import errno +from cgroupspy.contenttypes import DeviceAccess, DeviceThrottle + +from .interfaces import BaseFileInterface, FlagFile, BitFieldFile, IntegerFile, SplitValueFile, DictOrFlagFile +from .interfaces import MultiLineIntegerFile, CommaDashSetFile, DictFile, IntegerListFile, TypedFile + + +class Controller(object): + + """ + Base controller. Provides access to general files, existing in all cgroups and means to get/set properties + """ + + tasks = MultiLineIntegerFile("tasks") + procs = MultiLineIntegerFile("cgroup.procs") + notify_on_release = FlagFile("notify_on_release") + clone_children = FlagFile("cgroup.clone_children") + + def __init__(self, node): + self.node = node + + def filepath(self, filename): + """The full path to a file""" + + return os.path.join(self.node.full_path, filename) + + def list_interfaces(self): + result = {} + + for data in [self.__class__.__dict__, Controller.__dict__]: + for k, interface in data.items(): + if not issubclass(interface.__class__, BaseFileInterface): + continue + + result[k] = interface + + return result + + def get_interface(self, key): + if key in self.__class__.__dict__: + interface = self.__class__.__dict__[key] + elif key in Controller.__dict__: + interface = Controller.__dict__[key] + else: + return None + + if not issubclass(interface.__class__, BaseFileInterface): + return None + + return interface + + def get_property(self, filename): + """Opens the file and reads the value""" + + with open(self.filepath(filename)) as f: + return f.read().strip() + + def get_content(self, key): + interface = self.get_interface(key) + + if interface is None or interface.writeonly: + return None + + try: + content = self.get_property(interface.filename) + except IOError as e: + if e.errno == errno.ENOENT: + # does not exist + return None + elif e.errno == errno.EACCES: + # cannot be read + return None + elif e.errno == errno.EINVAL: + # invalid argument + return None + raise + + if not content.strip(): + return '' + + return interface.sanitize_get(content) + + def set_property(self, filename, value): + """Opens the file and writes the value""" + + with open(self.filepath(filename), "w") as f: + return f.write(str(value)) + + +class CpuController(Controller): + + """ + Cpu cGroup controller. Provides access to + + cpu.cfs_period_us + cpu.cfs_quota_us + cpu.rt_period_us + cpu.rt_runtime_us + cpu.shares + cpu.stat + """ + cfs_period_us = IntegerFile("cpu.cfs_period_us") + cfs_quota_us = IntegerFile("cpu.cfs_quota_us") + rt_period_us = IntegerFile("cpu.rt_period_us") + rt_runtime_us = IntegerFile("cpu.rt_runtime_us") + shares = IntegerFile("cpu.shares") + stat = DictFile("cpu.stat", readonly=True) + + +class CpuAcctController(Controller): + + """ + cpuacct.stat + cpuacct.usage + cpuacct.usage_percpu + """ + acct_stat = DictFile("cpuacct.stat", readonly=True) + usage = IntegerFile("cpuacct.usage") + usage_percpu = IntegerListFile("cpuacct.usage_percpu", readonly=True) + + +class CpuSetController(Controller): + + """ + CpuSet cGroup controller. Provides access to + + cpuset.cpu_exclusive + cpuset.cpus + cpuset.mem_exclusive + cpuset.mem_hardwall + cpuset.memory_migrate + cpuset.memory_pressure + cpuset.memory_pressure_enabled + cpuset.memory_spread_page + cpuset.memory_spread_slab + cpuset.mems + cpuset.sched_load_balance + cpuset.sched_relax_domain_level + """ + + cpus = CommaDashSetFile("cpuset.cpus") + mems = CommaDashSetFile("cpuset.mems") + + cpu_exclusive = FlagFile("cpuset.cpu_exclusive") + mem_exclusive = FlagFile("cpuset.mem_exclusive") + mem_hardwall = FlagFile("cpuset.mem_hardwall") + memory_migrate = FlagFile("cpuset.memory_migrate") + memory_pressure = FlagFile("cpuset.memory_pressure") + memory_pressure_enabled = FlagFile("cpuset.memory_pressure_enabled") + memory_spread_page = FlagFile("cpuset.memory_spread_page") + memory_spread_slab = FlagFile("cpuset.memory_spread_slab") + sched_load_balance = FlagFile("cpuset.sched_load_balance") + + sched_relax_domain_level = IntegerFile("cpuset.sched_relax_domain_level") + + +class MemoryController(Controller): + + """ + Memory cGroup controller. Provides access to + + memory.failcnt + memory.force_empty + memory.limit_in_bytes + memory.max_usage_in_bytes + memory.memsw.failcnt + memory.memsw.limit_in_bytes + memory.memsw.max_usage_in_bytes + memory.memsw.usage_in_bytes + memory.move_charge_at_immigrate + memory.numa_stat + memory.oom_control + memory.pressure_level + memory.soft_limit_in_bytes + memory.stat + memory.swappiness + memory.usage_in_bytes + memory.use_hierarchy + """ + + failcnt = IntegerFile("memory.failcnt") + memsw_failcnt = IntegerFile("memory.memsw.failcnt") + + limit_in_bytes = IntegerFile("memory.limit_in_bytes") + soft_limit_in_bytes = IntegerFile("memory.soft_limit_in_bytes") + usage_in_bytes = IntegerFile("memory.usage_in_bytes") + max_usage_in_bytes = IntegerFile("memory.max_usage_in_bytes") + + memsw_limit_in_bytes = IntegerFile("memory.memsw.limit_in_bytes") + memsw_usage_in_bytes = IntegerFile("memory.memsw.usage_in_bytes") + memsw_max_usage_in_bytes = IntegerFile("memory.memsw.max_usage_in_bytes") + swappiness = IntegerFile("memory.swappiness") + + stat = DictFile("memory.stat", readonly=True) + + use_hierarchy = FlagFile("memory.use_hierarchy") + force_empty = FlagFile("memory.force_empty") + oom_control = DictOrFlagFile("memory.oom_control") + + move_charge_at_immigrate = BitFieldFile("memory.move_charge_at_immigrate") + + # Requires special file interface + # numa_stat = + + # Requires eventfd handling - https://www.kernel.org/doc/Documentation/cgroups/memory.txt + # pressure_level = + + +class DevicesController(Controller): + """ + devices.allow + devices.deny + devices.list + """ + + allow = TypedFile("devices.allow", DeviceAccess, writeonly=True) + deny = TypedFile("devices.deny", DeviceAccess, writeonly=True) + list = TypedFile("devices.list", DeviceAccess, readonly=True, many=True) + + +class BlkIOController(Controller): + """ + blkio.io_merged + blkio.io_merged_recursive + blkio.io_queued + blkio.io_queued_recursive + blkio.io_service_bytes + blkio.io_service_bytes_recursive + blkio.io_serviced + blkio.io_serviced_recursive + blkio.io_service_time + blkio.io_service_time_recursive + blkio.io_wait_time + blkio.io_wait_time_recursive + blkio.leaf_weight + blkio.leaf_weight_device + blkio.reset_stats + blkio.sectors + blkio.sectors_recursive + blkio.throttle.io_service_bytes + blkio.throttle.io_serviced + blkio.throttle.read_bps_device + blkio.throttle.read_iops_device + blkio.throttle.write_bps_device + blkio.throttle.write_iops_device + blkio.time + blkio.time_recursive + blkio.weight + blkio.weight_device + """ + + io_merged = SplitValueFile("blkio.io_merged", 1, int) + io_merged_recursive = SplitValueFile("blkio.io_merged_recursive", 1, int) + io_queued = SplitValueFile("blkio.io_queued", 1, int) + io_queued_recursive = SplitValueFile("blkio.io_queued_recursive", 1, int) + io_service_bytes = SplitValueFile("blkio.io_service_bytes", 1, int) + io_service_bytes_recursive = SplitValueFile("blkio.io_service_bytes_recursive", 1, int) + io_serviced = SplitValueFile("blkio.io_serviced", 1, int) + io_serviced_recursive = SplitValueFile("blkio.io_serviced_recursive", 1, int) + io_service_time = SplitValueFile("blkio.io_service_time", 1, int) + io_service_time_recursive = SplitValueFile("blkio.io_service_time_recursive", 1, int) + io_wait_time = SplitValueFile("blkio.io_wait_time", 1, int) + io_wait_time_recursive = SplitValueFile("blkio.io_wait_time_recursive", 1, int) + leaf_weight = IntegerFile("blkio.leaf_weight") + # TODO: Uncomment the following properties after researching how to interact with files + # leaf_weight_device = + reset_stats = IntegerFile("blkio.reset_stats") + # sectors = + # sectors_recursive = + # throttle_io_service_bytes = + # throttle_io_serviced = + throttle_read_bps_device = TypedFile("blkio.throttle.read_bps_device", contenttype=DeviceThrottle, many=True) + throttle_read_iops_device = TypedFile("blkio.throttle.read_iops_device", contenttype=DeviceThrottle, many=True) + throttle_write_bps_device = TypedFile("blkio.throttle.write_bps_device ", contenttype=DeviceThrottle, many=True) + throttle_write_iops_device = TypedFile("blkio.throttle.write_iops_device ", contenttype=DeviceThrottle, many=True) + # time = + # time_recursive = + weight = IntegerFile("blkio.weight") + # weight_device = + + +class NetClsController(Controller): + + """ + net_cls.classid + """ + class_id = IntegerFile("net_cls.classid") + + +class NetPrioController(Controller): + + """ + net_prio.prioidx + net_prio.ifpriomap + """ + prioidx = IntegerFile("net_prio.prioidx", readonly=True) + ifpriomap = DictFile("net_prio.ifpriomap") diff --git a/airflow/_vendor/cgroupspy/interfaces.py b/airflow/_vendor/cgroupspy/interfaces.py new file mode 100644 index 0000000000000..be3066c9190d4 --- /dev/null +++ b/airflow/_vendor/cgroupspy/interfaces.py @@ -0,0 +1,339 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CloudSigma AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +from collections import Iterable +from cgroupspy.contenttypes import BaseContentType + + +class BaseFileInterface(object): + + """ + Basic cgroups file interface, implemented as a python descriptor. Provides means to get and set cgroup properties. + """ + readonly = False + writeonly = False + + def __init__(self, filename, readonly=None, writeonly=None): + if readonly and writeonly: + raise RuntimeError("This interface cannot be both readonly and writeonly") + + try: + self.filename = filename.encode() + except AttributeError: + self.filename = filename + self.readonly = readonly or self.readonly + self.writeonly = writeonly or self.writeonly + + def __get__(self, instance, owner): + if self.writeonly: + raise RuntimeError("This interface is writeonly") + + value = instance.get_property(self.filename) + return self.sanitize_get(value) + + def __set__(self, instance, value): + if self.readonly: + raise RuntimeError("This interface is readonly") + + value = self.sanitize_set(value) + if value is not None: + return instance.set_property(self.filename, value) + + def sanitize_get(self, value): + return value + + def sanitize_set(self, value): + return value + + +class FlagFile(BaseFileInterface): + + """ + Converts True/False to 1/0 and vise versa. + """ + + def sanitize_get(self, value): + return bool(int(value)) + + def sanitize_set(self, value): + return int(bool(value)) + + +class BitFieldFile(BaseFileInterface): + + """ + Example: '2' becomes [False, True, False, False, False, False, False, False] + """ + + def sanitize_get(self, value): + v = int(value) + # Calculate the length of the value in bits by converting to hex + length = (len(hex(v)) - 2) * 4 + # Increase length to the next multiple of 8 + length += (7 - (length - 1) % 8) + return [bool((v >> i) & 1) for i in range(length)] + + def sanitize_set(self, value): + try: + value = value.encode() + except AttributeError: + pass + if isinstance(value, bytes) or not isinstance(value, Iterable): + return int(value) + return sum((int(bool(value[i])) << i) for i in range(len(value))) + + +class IntegerFile(BaseFileInterface): + + """ + Get/set single integer values. + """ + + def sanitize_get(self, value): + val = int(value) + if val == -1: + val = None + return val + + def sanitize_set(self, value): + if value is None: + value = -1 + return int(value) + + +class DictFile(BaseFileInterface): + def sanitize_get(self, value): + res = {} + for el in value.split("\n"): + key, val = el.split() + res[key] = int(val) + return res + + def sanitize_set(self, value): + if not isinstance(value, dict): + raise ValueError("Value {} must be a dict".format(value)) + + keys = sorted(value.keys()) + return "\n".join("{} {}".format(k, value[k]) for k in keys) + + +class ListFile(BaseFileInterface): + readonly = True + + def sanitize_get(self, value): + return value.split() + + +class IntegerListFile(ListFile): + + """ + ex: 253237230463342 317756630269369 247294096796305 289833051422078 + """ + + def sanitize_get(self, value): + value_list = super(IntegerListFile, self).sanitize_get(value) + return list(map(int, value_list)) + + def sanitize_set(self, value): + if value is None: + value = '-1' + + return " ".join([str(v) for v in value]) + + +class CommaDashSetFile(BaseFileInterface): + + """ + Builds a set from files containing the following data format 'cpuset.cpus: 1-3,6,11-15', + returning {1,2,3,5,11,12,13,14,15} + """ + + def sanitize_get(self, value): + elems = [] + for el_group in value.strip().split(','): + if "-" in el_group: + start, end = el_group.split("-") + for el in range(int(start), int(end) + 1): + elems.append(el) + else: + if el_group != '': + elems.append(int(el_group)) + return set(elems) + + def sanitize_set(self, value): + if len(value) == 0: + return ' ' + + try: + value = value.decode() + except AttributeError: + pass + + if isinstance(value, str): + value = value.strip() + if not value: + return ' ' + value = value.split(',') + + if isinstance(value, Iterable): + value = set(value) + else: + raise ValueError("Value {} must be a sequence of int".format(value)) + + for k in value: + if not isinstance(k, int): + raise ValueError("Value {} must be a sequence of int".format(value)) + + value = sorted(list(value)) + index = [i for i in range(len(value))] + for i in range(len(value)): + if index[i] != i: + continue + + j = i + + while j < len(value) - 1: + if value[j] + 1 == value[j + 1]: + index[j + 1] = i + j += 1 + else: + break + + parts = [] + for i in range(len(value)): + if i > 0 and index[i - 1] == index[i]: + continue + + j = i + + while j + 1 < len(value) and index[j + 1] == index[j]: + j += 1 + + if i == j: + parts.append(str(value[i])) + else: + parts.append('{}-{}'.format(value[i], value[j])) + + return ','.join(parts) + + +class MultiLineIntegerFile(BaseFileInterface): + + def sanitize_get(self, value): + int_list = [int(val) for val in value.strip().split("\n") if val] + return int_list + + def sanitize_set(self, value): + if value is None: + return '-1' + + return '\n'.join(str(x) for x in value) + + +class SplitValueFile(BaseFileInterface): + """ + Example: Getting int(10) for file with value 'Total 10'. Readonly. + """ + readonly = True + + def __init__(self, filename, position, restype=None, splitchar=" ", prefix="Total"): + super(SplitValueFile, self).__init__(filename) + self.position = position + self.restype = restype + self.splitchar = splitchar + self.prefix = prefix + + def sanitize_get(self, value): + res = value.strip().split(self.splitchar)[self.position] + if self.restype and not isinstance(res, self.restype): + return self.restype(res) + return res + + def sanitize_set(self, value): + return '{}{}{}'.format(self.prefix, self.splitchar, value) + + +class TypedFile(BaseFileInterface): + + def __init__(self, filename, contenttype, readonly=None, writeonly=None, many=False): + if not issubclass(contenttype, BaseContentType): + raise RuntimeError("Contenttype should be a class inheriting " + "from BaseContentType, not {}".format(contenttype)) + + self.contenttype = contenttype + self.many = many + super(TypedFile, self).__init__(filename, readonly=readonly, writeonly=writeonly) + + def sanitize_set(self, value): + if isinstance(value, self.contenttype): + return str(value) + + if self.many: + items = [] + for entry in value: + if isinstance(entry, str): + entry = entry.strip() + if not entry: + continue + items.append(str(self.contenttype.from_string(entry))) + else: + items.append(str(entry)) + return "\n".join(items) + else: + return str(self.contenttype.from_string(value)) + + def sanitize_get(self, value): + if self.many: + result = [] + for line in value.splitlines(): + line = line.strip() + if not line: + continue + result.append(self.contenttype.from_string(line)) + return result + + return self.contenttype.from_string(value) + + +class DictOrFlagFile(BaseFileInterface): + def __init__(self, filename, readonly=None, writeonly=None): + super(DictOrFlagFile, self).__init__(filename, readonly=readonly, writeonly=writeonly) + self.interfaces = { + 'dict': DictFile(filename), + 'flag': FlagFile(filename), + } + + def sanitize_get(self, value): + try: + return self.interfaces['dict'].sanitize_get(value) + except Exception: + return self.interfaces['flag'].sanitize_get(value) + + def sanitize_set(self, value): + try: + return self.interfaces['dict'].sanitize_set(value) + except Exception: + return self.interfaces['flag'].sanitize_set(value) diff --git a/airflow/_vendor/cgroupspy/nodes.py b/airflow/_vendor/cgroupspy/nodes.py new file mode 100644 index 0000000000000..8789aae99308b --- /dev/null +++ b/airflow/_vendor/cgroupspy/nodes.py @@ -0,0 +1,283 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CLOUDSIGMA AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +import logging + +import os + +from .controllers import CpuAcctController, CpuController, CpuSetController, MemoryController, DevicesController, \ + BlkIOController, NetClsController, NetPrioController +from .utils import walk_tree, walk_up_tree + + +LOG = logging.getLogger(__name__) + + +class Node(object): + + """ + Basic cgroup tree node. Provides means to link it to a parent and set a controller, depending on the cgroup the node + exists in. + """ + NODE_ROOT = b"root" + NODE_CONTROLLER_ROOT = b"controller_root" + NODE_SLICE = b"slice" + NODE_SCOPE = b"scope" + NODE_CGROUP = b"cgroup" + + CONTROLLERS = { + b"memory": MemoryController, + b"cpuset": CpuSetController, + b"cpu": CpuController, + b"cpuacct": CpuAcctController, + b"devices": DevicesController, + b"blkio": BlkIOController, + b"net_cls": NetClsController, + b"net_prio": NetPrioController, + } + + def __init__(self, name, parent=None): + if isinstance(name, str): + name = name.encode() + + self.name = name + self.verbose_name = name + + if parent and not isinstance(parent, Node): + raise ValueError('Parent should be another Node') + + self.parent = parent + self.children = [] + self.node_type = self._get_node_type() + self.controller_type = self._get_controller_type() + self.controller = self._get_controller() + + def __eq__(self, other): + if isinstance(other, self.__class__) and self.full_path == other.full_path: + return True + return False + + def __repr__(self): + return "<{} {}>".format(self.__class__.__name__, self.path.decode()) + + @property + def full_path(self): + """Absolute system path to the node""" + + if self.parent: + return os.path.join(self.parent.full_path, self.name) + return self.name + + @property + def path(self): + """Node's relative path from the root node""" + + if self.parent: + try: + parent_path = self.parent.path.encode() + except AttributeError: + parent_path = self.parent.path + return os.path.join(parent_path, self.name) + return b"/" + + def _get_node_type(self): + """Returns the current node's type""" + + if self.parent is None: + return self.NODE_ROOT + elif self.parent.node_type == self.NODE_ROOT: + return self.NODE_CONTROLLER_ROOT + elif b".slice" in self.name or b'.partition' in self.name: + return self.NODE_SLICE + elif b".scope" in self.name: + return self.NODE_SCOPE + else: + return self.NODE_CGROUP + + def _get_controller_type(self): + """Returns the current node's controller type""" + + if self.node_type == self.NODE_CONTROLLER_ROOT and self.name in self.CONTROLLERS: + return self.name + elif self.parent: + return self.parent.controller_type + else: + return None + + def _get_controller(self): + """Returns the current node's controller""" + + if self.controller_type: + return self.CONTROLLERS[self.controller_type](self) + return None + + def create_cgroup(self, name): + """ + Create a cgroup by name and attach it under this node. + """ + if isinstance(name, str): + name = name.encode() + + node = Node(name, parent=self) + if node in self.children: + raise RuntimeError('Node {} already exists under {}'.format(name, self.path)) + + fp = os.path.join(self.full_path, name) + os.mkdir(fp) + self.children.append(node) + return node + + def delete_cgroup(self, name): + """ + Delete a cgroup by name and detach it from this node. + Raises OSError if the cgroup is not empty. + """ + name = name.encode() + fp = os.path.join(self.full_path, name) + if os.path.exists(fp): + os.rmdir(fp) + node = Node(name, parent=self) + try: + self.children.remove(node) + except ValueError: + return + + def delete_empty_children(self): + """ + Walk through the children of this node and delete any that are empty. + """ + removed_children = [] + + for child in self.children: + child.delete_empty_children() + try: + if os.path.exists(child.full_path): + os.rmdir(child.full_path) + removed_children.append(child) + except OSError: + pass + + for child in removed_children: + self.children.remove(child) + + def walk(self): + """Walk through this node and its children - pre-order depth-first""" + return walk_tree(self) + + def walk_up(self): + """Walk through this node and its children - post-order depth-first""" + return walk_up_tree(self) + + +class NodeControlGroup(object): + + """ + A tree node that can group together same multiple nodes based on their position in the cgroup hierarchy + + For example - we have mounted all the cgroups in /sys/fs/cgroup/ and we have a scope in each of them under + /{cpuset,cpu,memory,cpuacct}/isolated.scope/. Then NodeControlGroup, can provide access to all cgroup properties + like + + isolated_scope.cpu + isolated_scope.memory + isolated_scope.cpuset + + Requires a basic Node tree to be generated. + """ + + def __init__(self, name, parent=None): + self.name = name + self.parent = parent + self.children_map = {} + self.controllers = {} + self.nodes = [] + + @property + def path(self): + if self.parent: + base_name, ext = os.path.splitext(self.name) + if ext not in [b'.slice', b'.scope', b'.partition']: + base_name = self.name + return os.path.join(self.parent.path, base_name) + return b"/" + + def add_node(self, node): + """ + A a Node object to the group. Only one node per cgroup is supported + """ + if self.controllers.get(node.controller_type, None): + raise RuntimeError("Cannot add node {} to the node group. A node for {} group is already assigned".format( + node, + node.controller_type + )) + self.nodes.append(node) + if node.controller: + self.controllers[node.controller_type] = node.controller + setattr(self, node.controller_type.decode(), node.controller) + + def __repr__(self): + return "<{} {}>".format(self.__class__.__name__, self.name.decode()) + + @property + def children(self): + return list(self.children_map.values()) + + @property + def group_tasks(self): + """All tasks in the hierarchy, affected by this group.""" + tasks = set() + for node in walk_tree(self): + for ctrl in node.controllers.values(): + tasks.update(ctrl.tasks) + return tasks + + @property + def tasks(self): + """Tasks in this exact group""" + tasks = set() + for ctrl in self.controllers.values(): + tasks.update(ctrl.tasks) + return tasks + + +class NodeVM(NodeControlGroup): + + """Abstraction of a QEMU virtual machine node.""" + + @property + def verbose_name(self): + try: + return self.nodes[0].verbose_name + except Exception: + return "AnonymousVM" + + @property + def emulator(self): + return self.children_map.get(b"emulator", None) + + @property + def vcpus(self): + return [node for name, node in self.children_map.items() if name.startswith(b"vcpu")] diff --git a/airflow/_vendor/cgroupspy/trees.py b/airflow/_vendor/cgroupspy/trees.py new file mode 100644 index 0000000000000..f773f22db5aa0 --- /dev/null +++ b/airflow/_vendor/cgroupspy/trees.py @@ -0,0 +1,246 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CLOUDSIGMA AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +import os + +from .nodes import Node, NodeControlGroup, NodeVM +from .utils import walk_tree, walk_up_tree, split_path_components + + +class BaseTree(object): + + """ A basic cgroup node tree. An exact representation of the filesystem tree, provided by cgroups. """ + + def __init__(self, root_path=b"/sys/fs/cgroup/", groups=None, sub_groups=None): + """ + Construct a basic cgroup node tree. An exact representation of the filesystem tree, provided by cgroups. + + :param root_path: str -> The path of the root folder containing the cgroups. By default it is /sys/fs/cgroup/ + :param groups: None | list -> Use only those controllers to collect information in this tree instance + :param sub_groups: None | list -> Use only those slices to retrieve information. If the slice does not exist, + then create it + """ + if isinstance(root_path, str): + root_path = root_path.encode() + + self.root_path = root_path + self._groups = groups or [] + self._sub_groups = sub_groups or [] + self.root = Node(root_path) + self._build_tree() + + @property + def groups(self): + return self._groups + + @property + def sub_groups(self): + return self._sub_groups + + def _build_tree(self): + """ + Build a full or a partial tree, depending on the groups/sub-groups specified. + """ + + groups = self._groups or self.get_children_paths(self.root_path) + for group in groups: + node = Node(name=group, parent=self.root) + self.root.children.append(node) + self._init_sub_groups(node) + + def _init_sub_groups(self, parent): + """ + Initialise sub-groups, and create any that do not already exist. + """ + if not self._sub_groups: + self._init_children(parent) + return + + sub_group_components = dict() + + for sub_group in self._sub_groups: + sub_group = sub_group.strip() + if not sub_group: + continue + + components = split_path_components(sub_group) + if not components: + continue + + sub_group_components[sub_group] = components + + self._sub_groups = list(sub_group_components.keys()) + if not self._sub_groups: + self._init_children(parent) + return + + for sub_group, components in sub_group_components.items(): + for component in components: + if isinstance(component, str): + component = component.encode() + + fp = os.path.join(parent.full_path, component) + if os.path.exists(fp): + node = Node(name=component, parent=parent) + parent.children.append(node) + else: + node = parent.create_cgroup(component) + parent = node + self._init_children(node) + + def _init_children(self, parent): + """ + Initialise each node's children - essentially build the tree. + """ + + for dir_name in self.get_children_paths(parent.full_path): + child = Node(name=dir_name, parent=parent) + parent.children.append(child) + self._init_children(child) + + def get_children_paths(self, parent_full_path): + for dir_name in os.listdir(parent_full_path): + if os.path.isdir(os.path.join(parent_full_path, dir_name)): + yield dir_name + + def walk(self, root=None): + """Walk through each each node - pre-order depth-first""" + + if root is None: + root = self.root + return walk_tree(root) + + def walk_up(self, root=None): + """Walk through each each node - post-order depth-first""" + + if root is None: + root = self.root + return walk_up_tree(root) + + +class Tree(BaseTree): + def get_node_by_path(self, path): + try: + path = path.encode() + except AttributeError: + pass + path = path.rstrip(b"/") + for node in self.walk(): + if node.path == path: + return node + + +class GroupedTree(object): + """ + A grouped tree - that has access to all cgroup partitions with the same name ex: + 'machine' partition in memory, cpuset, cpus, etc cgroups. + All these attributes are accessed via machine.cpus, machine.cpuset, etc. + + """ + + def __init__(self, root_path=b"/sys/fs/cgroup", groups=None, sub_groups=None): + + self.node_tree = BaseTree(root_path=root_path, groups=groups, sub_groups=sub_groups) + self.control_root = NodeControlGroup(name=b"cgroup") + for ctrl in self.node_tree.root.children: + self.control_root.add_node(ctrl) + + self._init_control_tree(self.control_root) + + def _init_control_tree(self, cgroup): + new_cgroups = [] + for node in cgroup.nodes: + + for child in node.children: + if child.name not in cgroup.children_map: + new_cgroup = self._create_node(child.verbose_name, parent=cgroup) + cgroup.children_map[child.name] = new_cgroup + new_cgroups.append(new_cgroup) + + cgroup.children_map[child.name].add_node(child) + + for new_group in new_cgroups: + self._init_control_tree(new_group) + + def _create_node(self, name, parent): + return NodeControlGroup(name, parent=parent) + + def walk(self, root=None): + if root is None: + root = self.control_root + return walk_tree(root) + + def walk_up(self, root=None): + if root is None: + root = self.control_root + return walk_up_tree(root) + + def get_node_by_name(self, pattern): + try: + pattern = pattern.encode() + except AttributeError: + pass + for node in self.walk(): + if pattern in node.name: + return node + + def get_node_by_path(self, path): + try: + path = path.encode() + except AttributeError: + pass + for node in self.walk(): + if path == node.path: + return node + + +class VMTree(GroupedTree): + + def __init__(self, *args, **kwargs): + self.vms = {} + super(VMTree, self).__init__(*args, **kwargs) + + def _create_node(self, name, parent): + if b"libvirt-qemu" in name or b"machine-qemu" in name or parent.name == b"qemu": + vm_node = NodeVM(name, parent=parent) + if isinstance(name, bytes): + key = name.decode() + else: + key = name + self.vms[key] = vm_node + return vm_node + return super(VMTree, self)._create_node(name, parent=parent) + + def get_vm_node(self, name): + keys = [ + name, + '%s.libvirt-qemu' % name, + "machine-qemu%s" % name + ] + + for key in keys: + if key in self.vms: + return self.vms[key] diff --git a/airflow/_vendor/cgroupspy/utils.py b/airflow/_vendor/cgroupspy/utils.py new file mode 100644 index 0000000000000..884f8071f1226 --- /dev/null +++ b/airflow/_vendor/cgroupspy/utils.py @@ -0,0 +1,69 @@ +""" +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the CloudSigma AG nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL CLOUDSIGMA AG BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +""" +import os + + +def walk_tree(root): + """Pre-order depth-first""" + yield root + + for child in root.children: + for el in walk_tree(child): + yield el + + +def walk_up_tree(root): + """Post-order depth-first""" + for child in root.children: + for el in walk_up_tree(child): + yield el + + yield root + + +def split_path_components(path): + if isinstance(path, bytes): + path = str(path.decode()) + + if path.endswith('/'): + path = path.rstrip('/') + + components = [] + while True: + path, component = os.path.split(path) + if component != "": + components.append(component) + else: + if path != "": + components.append(path) + break + components.reverse() + + if len(components) > 0 and components[0] == '/': + return components[1:] + + return components diff --git a/airflow/_vendor/vendor.md b/airflow/_vendor/vendor.md new file mode 100644 index 0000000000000..94d18bc452513 --- /dev/null +++ b/airflow/_vendor/vendor.md @@ -0,0 +1,3 @@ +| Package | Version | File | SHA256 | +|-----------|---------|------------------------|------------------------------------------------------------------| +| cgroupspy | 0.2.1 | cgroupspy-0.2.1.tar.gz | 2a9e578566b99ac05c452d23044ea3061c9f9445752360c2ce4e9f2439fa1577 | diff --git a/licenses/LICENSE-cgroupspy.txt b/licenses/LICENSE-cgroupspy.txt new file mode 100644 index 0000000000000..6070cf1c8ef6a --- /dev/null +++ b/licenses/LICENSE-cgroupspy.txt @@ -0,0 +1,27 @@ +Copyright (c) 2014, CloudSigma AG +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +* Neither the name of cgroupspy nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/setup.cfg b/setup.cfg index dbe49e2be68b7..cba642d68e0ff 100644 --- a/setup.cfg +++ b/setup.cfg @@ -30,6 +30,7 @@ license_files = # Start of licenses generated automatically licenses/LICENSE-bootstrap.txt licenses/LICENSE-bootstrap3-typeahead.txt + licenses/LICENSE-cgroupspy.txt licenses/LICENSE-d3-shape.txt licenses/LICENSE-d3-tip.txt licenses/LICENSE-d3js.txt