Preventing Data Flat-lining05:00 PM - 05:30 PM on August 16, 2014, Room 705
- Audience level:
- Data Analysis
In this talk I will provide tips on working with glorious world of batch/ETL systems and explore open source projects to help maintain these data pipelines. Also cats.
At Rent the Runway we spend much of our time moving and transforming our data with Python. We have a complex system that combines multiple data sources into one analyst platform. Through the myriad scripts and coders touching the data, it can be easy to lose a few bits along the way. While we aren’t traders, it is not fun to send caveats to reports day after day.
Come learn a few ways you can build a robust data pipeline and add monitoring to it all while using Python. We will go through the pros and cons of different modules and approaches so you can pick the right solution for you.