Data Handling
9 talks from the 2023 edition.
-
Apache StreamPipes for Pythonistas: IIoT data handling made easy! ▶ Recording 📝 Transcript
Tim Bossenmaier, Sven Oehler
The industrial environment offers a lot of interesting use cases for data enthusiasts. There are myriads of interesting challenges that can be solved by data scientists. However, collecting industrial data in general...
-
Common issues with Time Series data and how to solve them ▶ Recording 📝 Transcript
Vadim Nelidov
Time-series data is all around us: from logistics to digital marketing, from pricing to stock markets. It’s hard to imagine a modern business that has no time series data to forecast. However, mastering such...
-
Most of you don't need Spark. Large-scale data management on a budget with Python ▶ Recording 📝 Transcript
Guillem Borrell
The Python data ecosystem has matured during the last decade and there are less and less reasons to rely only large batch process executed in a Spark cluster, but with every large ecosystem, putting together the key...
-
Neo4j graph databases for climate policy ▶ Recording 📝 Transcript
Marcus Tedesco
In this talk we walkthrough our experience using Neo4j and Python to model climate policy as a graph database. We discuss how we did it, some of the challenges we faced, and what we learnt along the way!
-
Polars - make the switch to lightning-fast dataframes ▶ Recording 📝 Transcript
Thomas Bierhance
In this talk, we will report on our experiences switching from Pandas to Polars in a real-world ML project. Polars is a new high-performance dataframe library for Python based on Apache Arrow and written in Rust. We...
-
Postmodern Architecture: The Python Powered Modern Data Stack ▶ Recording 📝 Transcript
John Sandall
The Modern Data Stack has brought a lot of new buzzwords into the data engineering lexicon: "data mesh", "data observability", "reverse ETL", "data lineage", "analytics engineering". In this light-hearted talk we...
-
Pragmatic ways of using Rust in your data project ▶ Recording 📝 Transcript
Christopher Prohm
Writing efficient data pipelines in Python can be tricky. The standard recommendation is to use vectorized functions implemented in Numpy, Pandas, or the like. However, what to do, when the processing task does not...
-
Raised by Pandas, striving for more: An opinionated introduction to Polars ▶ Recording 📝 Transcript
Nico Kreiling
Pandas is the de-facto standard for data manipulation in python, which I personally love for its flexible syntax and interoperability. But Pandas has well-known drawbacks such as memory in-efficiency, inconsistent...
-
WALD: A Modern & Sustainable Analytics Stack ▶ Recording 📝 Transcript
Florian Wilhelm
The name **WALD**-stack stems from the four technologies it is composed of, i.e. a cloud-computing **W**arehouse like Snowflake or Google BigQuery, the open-source data integration engine **A**irbyte, the open-source...