News

Updates on our open source project Yaook

In the last newsletter we already gave you a technical update on our focus project Yaook given. In the following, we would like to inform you about the innovations of the last three months and the current development status of our open source lifecycle management tool.

Features:

• Yaook/K8s: To deal with the changing URL of the Tigera Operator in different versions of Calico, a new approach has been taken. Instead of manually downloading and applying the Operator Manifest, we now use a Helm Chart for deployment. This change makes it easier for us to maintain the previous approach across past and future versions. Despite this change, we retain the flexibility to customise the deployment by passing in a custom values file. This allows us to customise and configure the deployment according to our specific needs while benefiting from the convenience and consistency of the Helm Chart.
• We have introduced support for offline installation of Python packages in the Yaook/K8s subproject. This addition allows users to install required Python packages without an active internet connection and ensures a smoother installation process. There is also an implementation for the Yaook/Baremetal project to allow execution without an internet connection.
• Yaook/Operator: We are pleased to announce support for shutting down the hypervisor in Nova Compute Eviction to avoid a lengthy workflow until the node is actually removed. This feature enhances the operator's capabilities and provides improved control over the eviction process. It also reduces the impact of a faulty hypervisor on the user and speeds up the process many times over.
  • This is a new function called "Eviction Manager" in a system called NovaDeployment. The purpose of this function is to monitor the status of hypervisors or compute services using the Nova API (a programme that helps manage virtual machines). When the Nova API indicates that a hypervisor has failed, the Eviction Manager performs the following actions:
    1. Verify that the hypervisor has actually failed by using existing checks such as "virsh list" or pinging the node. If it is determined that the hypervisor is running, the job fails to raise an alarm.
    2. Shut down the hypervisor with a tool called Ironic.
    3. Wait until all locks, especially those related to volumes, are released. The duration of this wait can be configured. The duration should be based on the last communication received from Nova (so-called "Nova heartbeat").
    4. Remove the hypervisor with the Nova API, which means removing it from the system.
  • To prevent erroneous removal of healthy hypervisors due to network problems or failures in the control plane, a threshold should be set. This threshold ensures that the eviction process (termination of pods) is stopped if too many hypervisors are reported as failed at the same time to avoid unnecessary evictions.
  • Ultimately, the existing logic for the eviction or migration process can remain unchanged, as Kubernetes already prevents two eviction jobs with the same name from running at the same time.
• We are also pleased to announce the latest addition to our cloud infrastructure: the Ironic 2023.1 release, codenamed "Antelope".
• Yaook/Baremetal: Additional flexibility through Netbox integration: We have integrated Netbox, a popular tool for IP address management and Data Centre Infrastructure Management (DCIM), to improve the flexibility and expandability of the project.

Fixes:

• Hotfix: CVE-2023-1668 - This hotfix resolves a critical vulnerability in Open vSwitch: Remote Traffic Denial of Service through manipulated packets with IP protocol 0. The details of the vulnerability are as follows: https://mail.openvswitch.org/pipermail/ovs-discuss/2023-April/052344.html

Several versions of Open vSwitch are vulnerable to manipulated IP packets with ip proto set to 0, leading to a potential denial of service. To trigger the vulnerability, an attacker would need to send a tampered IP packet with a protocol field of 0 and flow rules with "set" actions on other fields in the IP protocol header. The resulting flows would omit required actions and not mask the IP protocol field, resulting in a large bucket in which all IP packets are captured.

All versions of Open vSwitch, at least from 1.5.0, are affected.

• Yaook/Operator: "crash looping" of the OVSDB relay: We have fixed a problem with a crash loop of the OVSDB Relay components to ensure a more stable and reliable operation. This is achieved by deleting the database: "The ovsdb-tool create command does not work if a database file already exists."
  • OVSDB Relay is a tool that helps different parts of a computer network to communicate with each other. It acts as a translator or mediator between the Open vSwitch Database (OVSDB) and other network devices.
  • Open vSwitch Database (OVSDB) is a system that stores information about network configuration and status, e.g. which devices are connected and how they should communicate.
  • OVSDB Relay monitors changes in the OVSDB and forwards this information to other network devices that need to be informed of these changes. For example, when a new network device is added to or removed from the OVSDB, the OVSDB Relay notifies other devices so they can update their configurations accordingly.
  • Simply put, the OVSDB Relay helps different parts of a network stay in sync by translating and relaying information between the OVSDB and other devices and have a common understanding of network configuration and status.
• Yaook/Operator: Stability issues with OVN fixed: We have addressed several stability issues with OVN to further improve the reliability and performance of the system.
  • OVN stands for Open Virtual Network. It is a virtual network technology that helps manage and connect virtual machines (VMs) and containers in a computer network. OVN is based on Open vSwitch, a software switch that connects and routes network traffic between virtual machines.
  • Simply put, OVN provides a way to create and manage virtual networks for virtual machines and containers. It helps control how these virtual entities communicate with each other and with the physical network. OVN simplifies the configuration and management of virtual networks, making it easier to set up and maintain network connectivity in a virtualised environment.
• Yaook/BaremetalFixed issues with file permissions: During the Docker build, copied files lose their original permissions and receive insecure permissions such as UMASK 666 or 777. These insecure permissions persist during deployment and create security risks.
• Yaook/Baremetal: Added missing dependency of pip package "six": The Metal Controller crashes because the updated Keystoneauth package added a new dependency: https://opendev.org/openstack/keystoneauth/commit/ca28df84808787342303666e1b286dbc5ec88c61
• Yaook/Baremetal: Permissions for the pip configuration have been corrected.
• Yaook/Baremetal: Correctly revoke authentication token for Vault: The code was causing exceptions that broke access to Vault for the Metal Controller since we updated hvac to 0.11.2.
• Various other fixes and bug fixes

Updates:

• Added support for Kubernetes (K8s) 1.25 and all upcoming patch versions. With this update, Yaook now supports the latest version of Kubernetes and ensures compatibility with future patches.
• Support for Kubernetes versions 1.19 to 1.23 discontinued. In order to focus on the latest versions of Kubernetes and reduce maintenance costs, we have discontinued support for older versions. We recommend that users upgrade to the latest supported version.
• OpenStack version removed from ovn-bgp-agent: To streamline and simplify the project, we have removed the OpenStack version dependency from the ovn-bgp-agent component.
  • "ovn-bgp-agent" is a component of the Open Virtual Network (OVN) system that manages the functionality of the Border Gateway Protocol (BGP). BGP is a protocol used in computer networks to exchange routing information between different autonomous systems (ASes).
• Documentation improvements: We have significantly improved the project documentation, making it more comprehensive, accessible and user-friendly.
• Increased security: We have set more restrictive permissions for copied files in the metal controller image to improve the overall security situation of the project.

Outlook:

Looking to the future, the Yaook project aims to address several key challenges and implement exciting new features. These include:

• Upgrade path for newer OpenStack/OVN releases: We are actively working on developing a seamless upgrade path for the latest OpenStack and OVN releases to ensure that users can effortlessly stay up to date with the latest developments.
• Stability of OVN: We are committed to further improving the stability of the OVN component, focusing on eliminating potential problems and improving overall performance.
• Node lifecycle operator for updates: To enable smoother updates and maintenance, we are looking into developing a node lifecycle operator. This operator would automate the process of reinstalling nodes during updates, optimising the process for administrators.
• Release management: We are currently working to establish a robust routine, workflow and tools for release management of the project. This includes implementing automated release notes and documentation updates. As Yaook/K8s is relatively new in this respect, we will use it as a "playground" to experiment and find a suitable workflow that can later be applied to other project components.
• Further development of the Yaook/K8s subproject to improve integration with other Kubernetes features and tools.
• Extension of Yaook/Operator functionality to enable automation of routine tasks and improvement of system performance.
• Continue work on the Yaook/Baremetal project to extend support for different hardware and infrastructure platforms.
• Improve the documentation and usability of the entire Yaook project to facilitate implementation and use for developers and administrators.