Data Handling & Engineering
9 talks from the 2024 edition.
-
Data valuation for machine learning ▶ Recording 📝 Transcript
Miguel de Benito Delgado, Kristof Schröder
Data valuation techniques compute the contribution of training points to the final performance of machine learning models. They are part of so-called data-centric ML, with immediate applications in data engineering...
-
Exploring Zarr: From Fundamentals to Version 3.0 and Beyond ▶ Recording 📝 Transcript
Sanket Verma
A key feature of the Python data ecosystem is the reliance on simple but efficient primitives that follow well-defined interfaces to make tools work seamlessly together (Cf. http://data-apis.org/). NumPy provides an...
-
Going beyond Parquet's default settings – be surprised what you can get ▶ Recording 📝 Transcript
Uwe L. Korn
Apache Parquet has become the de facto format for storing tabular (DataFrame) data on disk. This is done through universal compression and efficient knowledge of the stored data structure. As part of this talk, we...
-
Lose your fear of equations! ▶ Recording 📝 Transcript
Darina Goldin
The skill of quickly judging what a formula does and how changing a parameter will affect the result is crucial when dealing with real-life data science - but it's a skill not easily acquired if you don't come from a...
-
Next Stop: Insights! How Streamlit and Snowflake Power Up Data Stories ▶ Recording 📝 Transcript
Marie-Kristin Wirsching
Data stories transform complex data insights into clear, actionable and context rich narratives to drive business value. The presentation of data stories to different audiences in a visually compelling manner while...
-
Pandas + Dask DataFrame 2.0 - Comparison to Spark, DuckDB and Polars ▶ Recording 📝 Transcript
Florian Jetter, Patrick Hoefler
Dask is a library for distributed computing with Python that integrates tightly with pandas. Historically, Dask was the easiest choice to use (it’s just pandas) but struggled to achieve robust performance (there were...
-
Polars and Time Series: what it can do, and how to overcome any limitation ▶ Recording 📝 Transcript
Marco Gorelli
Time series analysis is ubiquitous in applied data science because of the value it delivers. In order to do effective time series analysis, you need to know your tools well. Polars has excellent built-in time series...
-
The pragmatic Pythonic data engineer ▶ Recording 📝 Transcript
Robson Junior
Learn to make practical decisions in data engineering with Python's vast ecosystem. Avoid blindly following market guidelines and consider the reality of your situation for better performance and architecture.
-
The Struggles We Skipped: Data Engineering for the TikTok Generation ▶ Recording 📝 Transcript
Anuun, Hiba Jamal
In a world increasingly embracing Python, plug-and-play solutions and AI-generated code, our generation growing up with these advancements may not fully grasp the challenges faced by our predecessors. Meanwhile, data...