Today, data scientists, web developers, and application developers build complex data processing pipelines by combining diferent tools and programming languages. To this end, most data processing systems support user-defined functions (UDFs) in common languages like Java, Python, or JavaScript. These UDFs enable users to express arbitrary business logic in their preferred programming language, leverage 3rd-party libraries, and increase the modularity and testability of their data processing pipelines. Although UDFs provide a large degree of freedom, their flexibility comes with a high-performance cost compared to traditional relational queries. As a result, most experts recommend avoiding UDFs whenever possible.
we propose in this talk BabelfishLib, which provides our Babelfish Engine [ 1] as an extensible component for the eficient and secure execution of UDFs. In an environment where virtually every data management system requires UDF support, BabelfishLib can centralize these eforts and provide a unified UDF runtime that can be used across diferent systems. In particular, BabelfishLib targets three major design goals. First, it provides eficient execution strategies for UDFs in diferent programming languages. Second, it ensures that the execution of untrusted UDF code is isolated from the data processing system, guaranteeing system security. Third, it analyzes UDFs and provides information for further query optimizations. As a result, BabelfishLib mitigates the performance overhead of UDFs in state-of-the-art systems while it ensures security and isolation at the same time. Currently, we leverage BabelfishLib to accelerate UDFs in our data processing platform NebulaStream [2, 3].
We believe that BabelfishLib can be a first step towards a unified accelerator for UDFs, which can be integrated across diferent data processing systems. Furthermore, it provides a playground for further research focusing on specific aspects of the acceleration of UDF. Finally, through this presentation, we intend to spark a discussion across the community to consolidate requirements for eficient UDF execution and combine diferent eforts in the same direction. execution of polyglot queries, Proc. VLDB Endow. 3489496.3489501. doi:10.14778/3489496.3489501. [2] S.
LGOBE rOcid nEvelop-O Joint Workshops at 49th International Conference on Very Large Data Bases (VLDBW’23) — Second International Workshop on Composable
D. P. A. Nu
ronpub.com/ojiot/OJIOT_2020v6i1n07_Zeuch.html.