Pandas IO Tools: Reading and Writing DataFrames as Files and Databases

Reading a CSV file in Pandas can be as easy as dfr = pd.read_csv('filename.csv') and work each time, but still could return unexpected results. You may have to deal with ambiguous timestamps, broken timezone handling, obscure NaN notation, non-standard numbers representation, language-specific formats, or heterogeneous value types. The worst case is when it handles these problems somehow automatically and returns wrong data without warning.

On a series of examples we'll try to import real-world cases, discuss the problems and find a stable way to handle them. After CSV we'll have a look at most other formats supported by Pandas IO tools, such as pickle, JSON, Msgpack, HTML, Excel, HDF5, Parquet and a PostgreSQL database.

The participants will need Python 3.x and a recent Pandas installation. Jupyter Notebook may be useful but not necessary. A repository with all code examples and test data will be published before the conference.