stage(squidington)

Using Airbyte on Day 1 of our startup

Calendar Icon - Evently Webflow Template
07 Dec
 
Clock Icon - Evently Webflow Template
10:30 am
 - 
10:40 am PST

About the Session

When we wanted to start Anecdote we planned to hire X data engineers to build all of the pipelines we need, but then we discovered Airbyte which allowed us to ship our first MVP in half of the time we initially planned.

At Anecdote we’re on a mission to Harness the power of qualitative feedback to help organizations be more customer-centric, build the right things, and retain their customers.

We analyze various unstructured data such as support tickets, app reviews, surveys, calls, etc. for our clients and map this feedback to their business KPIs like retention, churn and overall customer satisfaction.

In this presentation we will walk you through our feeling from when we discovered Airbyte, how we started testing it, and how we made the decision to rely on it to be the backbone of our data engineering efforts to power the daily millions (and soon billions) data points.

1. What kind of experience do we have with Airbyte?
We have extensive experience with Airbyte for today and we want to share it with all of You. We use Airbyte, runned in EC2, as a part of our daily data pipelines. We wrote four new data sources for it: Google Play, AppStore RSS, AppStore AMP and Twitter and are going to add support for Live Engage. Also we use predefined sources for Zendesk Support, Freshdesk, MixPanel and destination for S3.Airbyte saved our time and money and brought solidity to our pipelines.

2. What does Anecdote's infrastructure look like?
We will show our infrastructure before and after Airbyte integration to it and talk about our future plans for building a data warehouse over S3 and total automatization of data pipelines with Apache Airflow. In each of these tasks Airbyte occupies a very important place.

3. How does Airbyte work in our infrastructure? How did we add new data sources to it?
We will talk about data sources, which we wrote for Airbyte by ourselves. We will discuss some common problems that we should take into account, when writing connectors for data sources with limited by time tickets access. Using AppStore as an example, we will show an approach for using different algorithms and connectors for the same data source.

4. What are our plans for the future? What can improve our experience with Airbyte?
We hope that Anecdote will grow from a little startup to a large company. When we started Anecdote 6 months ago, there were only three members in our team, but today we have nearly fifteen people in it. We are working on scalability of our solutions so we are waiting for the production release of Airbyte On Kubernetes and ability to add custom connectors to Airbyte Cloud. Also it would be great to add the possibility to choose workspace to Airbyte frontend.

Add to Calendar
Want to Join the movement?

Register today for move(data)!