Data Handling & Engineering
15 talks from the 2025 edition.
-
Analyze data easily with duckdb - and the implications on data architectures ▶ Recording 📝 Transcript
Matthias Niehoff
duckdb is increasingly becoming a universal tool for accessing and analyzing data. In this talk I will show with slides and live demo what duckdb is capable of and will dive deeper in how it will influence modern...
-
Bridging the gap: unlocking SAP data for data lakes with Python and PySpark via SAP Datasphere
Rostislaw Krassow
SAP's data often remains locked away, hindering the creation of a complete data picture. This talk presents a hands-on proof of concept leveraging SAP Datasphere, Python and PySpark to bridge an Azure-based, data...
-
Build a personalized Commute agent in Python with Hopsworks, LangGraph and LLM Function Calling ▶ Recording 📝 Transcript
Javier de la Rúa Martínez
The invention of the clock and the organization of time in zones have helped synchronize human activities across the globe. While timekeepers are better at planning and sticking to the plan, time optimists somehow...
-
Cache me if you can: Boosted application performance with Redis and client-side caching ▶ Recording 📝 Transcript
David Maier
Did you know Redis can notify your app about server-side data changes? This feature enables client-side tracking and caching in redis-py, helping to reduce network round-trips and optimize performance. In this talk,...
-
Challenges and Lessons Learned While Building a Real-Time Lakehouse using Apache Iceberg and Kafka
Jonas Böer, Elena Ouro Paz
How do you build a large-scale data lakehouse architecture that makes data available for business analytics in real time, while being more cost-effective, more flexible and faster than the previous proprietary...
-
Dataframely — A declarative, 🐻❄️-native data frame validation library ▶ Recording 📝 Transcript
Daniel Elsner, Oliver Borchert
Understanding the structure and content of data frames is crucial when working with tabular data — a core requirement for the robust pipelines we build at QuantCo. Libraries such as `pandera` or `patito` already...
-
Distributed file-systems made easy with Python's fsspec ▶ Recording 📝 Transcript
Einat Orr, Barak Amar
The cloud native revolution has impacted all aspects of engineering, and data engineering is not exempt. One of the ongoing challenges in the data engineering world remains the local and distributed cloud native...
-
From Rules to Reality: Python's Role in Shaping Roundnet ▶ Recording 📝 Transcript
Larissa Haas
Roundnet is a dynamic and fast-growing sport that combines quick reaction, athleticism, and strong community. However, like many emerging sports, it faces challenges in balancing competition, optimizing rules, and...
-
From Tensors to Clouds — A Practical Guide to Zarr V3 and Zarr-Python 3 ▶ Recording 📝 Transcript
Sanket Verma
A key feature of the Python data ecosystem is the reliance on simple but efficient primitives that follow well-defined interfaces to make tools work seamlessly together (Cf. http://data-apis.org/). NumPy provides an...
-
Introducing the Synthetic Data SDK - Privacy Preserving Synthetic Data for AI/ML
Michael Platzer
AI-generated synthetic data is gaining traction as a privacy-safe solution for data access and sharing. This data is created from original datasets, maintaining privacy without compromising utility. In this Session,...
-
Open Table Formats in the Wild: From Parquet to Delta Lake and Back ▶ Recording 📝 Transcript
Franz Wöllert
Open table formats have revolutionized analytical, columnar storage on cloud object stores with critical features like ACID compliance and enhanced metadata management, once exclusive to proprietary cloud data...
-
Power up your Polars code with Polars extention ▶ Recording 📝 Transcript
Cheuk Ting Ho
While Polars is written in Rust and has the advantages of speed and multi-threaded functionalities., everything will slow down if a Python function needs to be applied to the DataFrame. To avoid that, a Polar...
-
PyData Stack: Pure Python open source data platforms
Eric Thanenthiran
Modern open source Python data packages offer the opportunity to build and deploy pure Python, production-ready data platforms. Engineers can and do play a big role in helping companies become data-driven by...
-
Scraping LEGO for Fun: A Hacky Dive into Dynamic Data Extraction ▶ Recording 📝 Transcript
Peter Lodri
Unlock the full potential of modern web scraping by combining Python, Scrapy, and Playwright to extract data from dynamic, JavaScript-heavy sites—exemplified by LEGO product pages. This talk introduces Model Context...
-
Where have all the post offices gone? Discovering neighborhood facilities with Python and OSM ▶ Recording 📝 Transcript
Katie Richardson
When it comes to open geographic data, OpenStreetMap is an awesome resource. Getting started and figuring out how to make the most out of the data available can be challenging. Using a personal example: frustration...