Remote Backend Compiler (RBC)
Extending heavysql Using Python
Using Python, you can interact with databases in multiple ways. Libraries like SQLAlchemy provide a translation mechanism that converts Python to SQL; this is an example of an ORM (Object-Relational Mapping). With SQLAlchemy and similar approaches, user interactions with the database are simplified—and optimized—as a set of high-level functions provided by the ORM. Unfortunately, to run tasks not supported by the ORM, you need to write SQL code.
You can define your own SQL functions in HeavyDB, but to realize the full power of HeavyDB, you have to re-compile the engine to add your functions. To write GPU-compatible functions to execute on GPUs, HeavyDB supports User Defined Functions (UDFs) and User Defined Table Functions (UDTFs). A UDF operates on elements of tables; a UDTF operates on an entire table itself.
The Remote Backend Compiler (RBC) package provides a Python interface to define UDFs and UDTFs easily. Any UDF or UDTF written in Python can be registered at run time on the HeavyDB server and subsequently used in any SQL query by any client.
User define function schematic. Decorate a Python function to be able to call it with SQL.
Functions are not persisted on the database and need to be registered if the server is restarted.
Internally, the RBC converts the Python function to an intermediate representation (IR), which is then sent to the server. The IR is compiled on a CPU or a GPU, depending on specified hardware resources .
Ibis is an ORM that supports defining UDFs in C++ for some type of databases. However, it doesn’t provide a Python interface.
Export as PDF
Copy link