In the rapidly evolving landscape of BPF as kernel extensions, where the industry is deploying an increasing count of simultaneously running BPF programs, the need for accounting BPF- induced overhead on latency-sensitive kernel functions is becoming critical. We also find that eBPF's termination guarantee is insufficient to protect systems from BPF programs running extraordinarily long due to compute-heavy operations and runtime factors such as contention. Operators lack a crucial mechanism to identify and avoid installing long-running BPF programs while also requiring a mechanism to abort such BPF programs when found to be adding high latency overhead on performance-critical kernel functions. In this work, we propose a runtime estimator and a dynamic termination mechanism to solve these two issues, respectively. We use a hybrid of static and dynamic analysis to provide a runtime range that we demonstrate to encompass the actual runtime of the BPF program. For safe BPF termination, we propose a short-circuiting approach to skip all costly operations and quickly reach completion. We evaluate the proposed solutions to find the obtained performance estimate as too broad, but when paired with the dynamic termination, can be used by a BPF Orchestrator to impose policies on the overhead due to BPF programs in a call path. The proposed dynamic termination solution has zero overhead on BPF programs for no-termination cases while having a verification overhead proportional to the number of helper calls in a BPF program. In the future, we aim to make BPF execution atomic to guarantee that kernel objects modified within a BPF program are always left in a consistent state in the event of program termination. / Master of Science / The Linux kernel OS has a relatively recent feature called eBPF which allows adding new code into a running system without needing a system reboot. Due to the flexibility offered by eBPF, the technology is attracting widespread adoption for diverse use cases such as system health monitoring, security, accelerating programs, etc. In this work, we identify that eBPF programs have a non-negligible performance impact on a system which, in the extreme case, can cause Denial-of-Service attacks on the host machine despite going through all security checks enforced by eBPF. We propose a two-part solution: the eBPF runtime estimator and the Fast-Path termination mechanism. The runtime estimator aims to prevent the instal- lation of eBPF programs that can cause a large performance impact, while the Fast-Path termination will act as a safety net for cases when the installed program unexpectedly runs longer. The overall solution will enable better management of eBPF programs concerning their performance impact and enforce strict bounds on the added latency. Potential future work includes factoring in the impacts other than performance in our solution such as inter- BPF interaction and designing easy-to-use knobs which an operator can easily tune to relax or constrain the side-effects of the eBPF programs installed in the system.

Clustered container infrastructures are increasingly popular for deploying applications. The networking in these clusters is provided by specialized container networking solutions that often lead to complex network configurations on the nodes hosting the containers. Thereby, they can have a significant impact on the performance of the applications hosted in the cluster. While the throughput that can be achieved by the container networking solutions is regularly studied, the latency and subsequently jitter introduced by them is often underreported. This thesis investigates the latency and jitter introduced by the packet processing in the Linux kernel using different container networking solutions. This requires very detailed data about the processing of packets, which existing tracing tools for Linux fail to provide. Therefore, a custom tracing application is developed using eBPF that focuses on the flow of packets through the kernel. The application is evaluated and then used to compare the latency and jitter behavior of commonly used container networking solutions. The results show that the choice of transport protocols for real-time applications has a significant impact on the latency introduced by the kernel irrespective of the container networking. Also, some container networking solutions fall short of providing their proclaimed benefits in their default configurations. This highlights the need for performance evaluation in environments representative of the production setting and the need for tuning the configuration of container networking solutions and system resources to match the requirements of real-time use cases. The data also show that there is a need for more lightweight tracing technologies for packet processing. / Klustrade containerinfrastrukturer blir alltmer populära för att distribuera applikationer. Nätverket i dessa kluster tillhandahålls av specialiserade nätverkslösningar för containrar, vilket ofta leder till komplexa nätverkskonfigurationer på de noder som är värdar för containrarna. De kan därför ha en betydande inverkan på prestandan hos de applikationer som finns i klustret. Även om den bandbredd som kan uppnås med olika nätverkslösningar för containrar regelbundet studeras, är den latens och det jitter som de medför ofta underrapporterade. I den här avhandlingen undersöks den latens och jitter som introduceras av pakethanteringen i Linux-kärnan med hjälp av olika nätverkslösningar för containrar. Detta kräver mycket detaljerade uppgifter om pakethanteringen, som befintliga spårningsverktyg för Linux inte kan tillhandahålla. Därför utvecklas ett anpassat spårningsprogram med eBPF som fokuserar på flödet av paket genom kärnan. Programmet utvärderas och används sedan för att jämföra latens- och jitterbeteendet hos vanliga containernätverkslösningar. Resultaten visar att valet av transportprotokoll för realtidsapplikationer har en betydande inverkan på den latens som införs av kärnan, oberoende av containernätverken. Vissa lösningar för containernätverk klarar inte heller av att ge de fördelar de utlovat i sina standardkonfigurationer. Detta understryker behovet av att utvärdera prestanda i miljöer som är representativa för produktionsmiljöer och behovet av att justera konfigurationen av containernätverkslösningar och systemresurser så att de motsvarar kraven för realtidsbaserade användningsfall. Resultaten visar också att det finns ett behov av mindre resurskrävande spårningstekniker för pakethantering.

Počítačové systémy, ktoré musia poskytovať svoje služby s vysokou dostupnosťou vyžadujú isté bezpečnostné opatrenia na to, aby ostali dostupné aj pod paketovými sieťovými útokmi. Nevyžiadané pakety musia byť zahodené čo najskôr a čo najrýchlejšie. Táto práca analyzuje eXpress Data Path (XDP) ako techniku skorého zahodenia paketov a extended Berkeley Packet Filter (eBPF) ako mechanizmus rýchlej analýzy obsahu packetov. Poskytuje sa pohľad na dnešnú prax v oblasti firewallov v systémoch s linuxovým jadrom a navrhne sa systém rýchlej filtrácie paketov založený na eBPF a XDP. Do detailov popisujeme naimplementované filtračné riešenie. Nakoniec sa vyzdvihujú výhody XDP oproti ostatným súčasným technikám filtrácie paketov na sérii výkonnostných testov.

Extended Berkeley Packet Filter (eBPF) is an instruction set and an execution environment inside the Linux kernel. eBPF improves flexibility for data processing and is realized via a virtual machine featuring both a Just-In-Time (JIT) compiler and an interpreter running in the kernel. It executes custom eBPF programs supplied by the user, effectively moving kernel functionality into user space. eBPF has received widespread adoption by companies such as Facebook, Netflix, and academia for a wide range of application domains. eBPF can be used to program the eXpress DataPath (XDP), a kernel network layer that processes packets closer to the NetworkInterface Card (NIC) for fast packet processing. In this thesis, eBPF with XDP, and Iptables, are considered as a Network function(NF), implemented in a Virtual Machine (VM) for packet filtering. The traffic source(source VM) and traffic sink (destination VM) are present in the same subnet. The aim of this thesis is, to understand and investigate the implementation of NFs inVMs and to analyze performance metrics. In VirtualBox, VMs are created to implement the NFs. The results are obtained for the measurements that are essential for the performance evaluation of the NFs, and presented in graphs.

The Internet plays an important role in modern society, and its network performance impacts billions of users every day. For many network applications, network latency has a large impact on the quality of experience for the end user. Due to a lack of extensive network latency monitoring, the observability of network latency in real networks is often limited. This poses a problem for understanding network latency on the Internet today, and for assessing the impact various solutions that aim to reduce network latency have once they are deployed in the wild. This thesis addresses shortcomings with current solutions for monitoring network latency, in particular the performance of passive monitoring solutions on general-purpose commodity hardware, aiming to enable more ubiquitous latency monitoring and ultimately provide a comprehensive view of real-world network latency. We utilize the recently emerging eBPF technology to implement passive network latency monitoring inside the Linux kernel. Through experiments on a testbed, we show that our solution can monitor packets at over an order of magnitude higher rates than comparable previous solutions, allowing it to successfully monitor the latency for multi-gigabit traffic on general-purpose commodity hardware. Additionally, we demonstrate the feasibility of continuously monitoring network latency by deploying our solution inside an Internet Service Provider and monitoring the network latency for all customer traffic. Through an extensive analysis of the collected latency data, we show large differences in how network latency is distributed across different parts of the network. / The Internet plays a vital role in modern society, and its performance affects billions of users daily. Network latency often has a significant impact on the end users' experience. However, due to limited monitoring of network latency, the observability of latency in real networks is often poor. This hinders our understanding of latency on the Internet today and makes it challenging to assess how the deployment of new networking technologies impacts latency. This thesis uses the emerging eBPF technology to improve the performance of passive network latency monitoring, aiming to enable latency monitoring on more network devices to create a more comprehensive view of latency on the Internet. By conducting controlled experiments on a testbed, we find that our solution is over an order of magnitude faster than previous solutions, making it possible to monitor multi-gigabit traffic on general-purpose commodity hardware. Furthermore, we demonstrate the feasibility of continuously monitoring latency by deploying our solution inside the network of an Internet Service Provider to monitor all their traffic. Our analysis of the latency data reveals large differences in how latency is distributed across different parts of the network.

The world of computer networks is constantly evolving towards more efficient algorithms and more effective ways of using hardware resources. One of these evolutions is the eXpress Data Path (XDP). XDP is an interrupt based data path in the Linux kernel. XDP uses JIT-compiled programs in a virtual machine in a device driver context. Through XDP, fast packet processing is possible while still keeping the functionality of the Linux kernel intact. Therefore, this thesis aims to illuminate possible use cases for XDP in 5G simulators, as this real-world application of XDP is of interest. Specifically, use-cases where there is a need for fast packet processing. This thesis evaluates the use-cases using a performance evaluation of XDP and a literary review of 5G simulators, XDP, and technologies relating to XDP. This evaluation indicates that XDP is a candidate for packet processing in 5G simulators, specifically when compared to what performance is possible currently. This thesis argues from the performance evaluation and the literary review that XDP can be used for small programs, preferably data ingestion, in 5G simulators.

<p>Microservices have emerged as a popular pattern for developing large-scale applications in cloud environments for their flexibility, scalability, and agility benefits. However, microservices make management more complex due to their scale, multiple languages, and distributed nature. Orchestration and automation tools like Kubernetes help deploy microservices running simultaneously, but it can be difficult for an operator to understand their behaviors, interdependencies, and interactions. In such a complex and dynamic environment, performance problems (e.g., slow application responses and high resource usage)  require significant human effort spent on diagnosis and recovery. Moreover, manual diagnosis of cloud microservices tends to be tedious, time-consuming, and impractical. Effective and automated performance analysis and anomaly detection require an observable system, which means an application's internal state can be inferred by observing and tracking metrics, traces, and logs. Traditional APM uses libraries and SDKs to improve application monitoring and tracing but has additional overheads of rewriting, recompiling, and redeploying the applications' code base. Therefore, there is a critical need for a standardized automated microservices observability solution that does not require rewriting or redeploying the application to keep up with the agility of microservices.</p> <p><br></p> <p>This thesis studies observability for microservices and implements an automated Extended Berkeley Packet Filter (eBPF) based observability solution. eBPF is a Linux feature that allows us to write extensions to the Linux kernel for security and observability use cases. eBPF does not require modifying the application layer and instrumenting the individual microservices. Instead, it instruments the kernel-level API calls, which are common across all hosts in the cluster. eBPF programs provide observability information from the lowest-level system calls and can export data without additional performance overhead. The Prometheus time-series database is leveraged to store all the captured metrics and traces for analysis. With the help of our tool, a DevOps engineer can easily identify abnormal behavior of microservices and enforce appropriate countermeasures. Using Chaos Mesh, we inject anomalies at the network and host layer, which we can identify with root cause identification using the proposed solution. The Chameleon cloud testbed is used to deploy our solution and test its capabilities and limitations.</p>

Cílem této magisterské práce je návrh a implementace síťové sondy pro sledování toků na 10GbE rozhraní. Text se zabývá přehledem GNU/Linux nástrojů využívaných ve vysokorychlostních sítích a principů jejich fungování. Dále pak je uveden návrh a implementace sondy využívající mechanismu zero-copy pro sledování provozu na 10GbE rozhraní. Aplikace využívá Expresní datové cesty (XDP) a jeho AF_XDP soketu pro zachycení provozu na rozhraní. Jako testovací platforma byla vybrána platforma NETX používaná na FIT VUT.

Tato práce představuje optimalizační techniky zaměřené na proces sběru výkonnostních dat v rámci výkonnostní analýzy a profilování programů v nástroji Perun.   Rozšíření architektury a implementace těchto nových optimalizačních technik v nástroji Perun (a převážně pak v jeho modulu Tracer) zlepšuje jeho škálovatelnost a umožňuje tak provádět výkonnostní analýzu i nad rozsáhlými projekty.   Zaměřujeme se především na zvýšení přesnosti sběru dat, redukci množství instrumentovaných bodů programu, omezení časové režie procesu sběru dat a výkonnostního profilování, snížení objemu sbíraných dat a velikosti výsledného výkonnostního profilu.   Optimalizace je dosažena pomocí aplikace statistických metod, množství technik statické a dynamické analýzy (případně jejich kombinací) a využitím pokročilých možností a schopností nástrojů SystemTap a eBPF.   Na základě vyhodnocení provedeného na dvou vybraných projektech a množství experimentů můžeme konstatovat, že se nám úspěšně podařilo dosáhnout značné optimalizace u téměř všech sledovaných metrik a kritérií.

Evolving technology in wireless telecommunication, such as 5G, provides opportunities to utilize wireless communication more in an industrial setting where reliability and predictability are of great concern. More capable Industrial Internet of Things devices (IIoT) are, indeed, a catalyst for Industry 4.0. Still, before the IIoT devices can be deemed capable enough, a method to evaluate the IIoT systems unobtrusively—so that the evaluation does not affect the performance of the systems—must be established. This thesis aims to answer how the performance of a distributed control system can be unobtrusively evaluated, and also determine what the state-of-the-art is in latency measurements in distributed control systems. To answer the question, a novel diagnostics method for time-critical control systems in cloud-fog automation is proposed and extensively evaluated on real-life testbeds that use 5G, WiFi 6, and Ethernet in an edge-computing topology with real control systems. The feasibility of the proposed method was verified by experiments conducted with a diagnostics framework prototype developed in this thesis. In the proposed diagnostics framework, the controller application is monitored by a computing probe based on an extended Berkeley Packet Filter program. Network communication between the controller and control target is evaluated with a multi-channel Ethernet probe and custom-made software that computes several metrics related to the performance of the distributed system. The data from the unobtrusive probes are sent to a time-series database that is used for further analysis and real-time visualization in a graphical interface created with Grafana. The proposed diagnostics method together with the developed prototype can be used as a research infrastructure for future evaluations of distributed control systems.