Preventing Data Flat-lining

05:00 PM - 05:30 PM on August 16, 2014, Room 705

Anna Smith

Audience level:

novice

Category:

Data Analysis

Description

In this talk I will provide tips on working with glorious world of batch/ETL systems and explore open source projects to help maintain these data pipelines. Also cats.

Abstract

At Rent the Runway we spend much of our time moving and transforming our data with Python. We have a complex system that combines multiple data sources into one analyst platform. Through the myriad scripts and coders touching the data, it can be easy to lose a few bits along the way. While we aren’t traders, it is not fun to send caveats to reports day after day.

Come learn a few ways you can build a robust data pipeline and add monitoring to it all while using Python. We will go through the pros and cons of different modules and approaches so you can pick the right solution for you.