Content

This is a collection of blogs and talks at conferences.

Have questions? Chat with us on Github or Slack:

Case Studies

• Lyft - How LyftLearn Democratizes Distributed Compute through Kubernetes Spark and Fugue

• Lyft - Distributed Machine Learning at Lyft

• Clobotics - Large Scale Image Processing with Spark through Fugue

Blogs

2022

Fugue Core

• Introducing Fugue — Reducing PySpark Developer Friction

• Why Pandas-like Interfaces are Sub-optimal for Distributed Computing

Fugue SQL

• Why SQL-Like Interfaces are Sub-optimal for Distributed Computing

Integrations

• PyCaret

  • Scaling PyCaret with Spark (or Dask) through Fugue (Towards Data Science)

• DuckDB

  • Fugue and DuckDB: Fast SQL Code in Python (Towards Data Science by Khuyen Tran)

• WhyLogs

  • Large Scale Data Profiling with whylogs and Fugue on Spark, Ray or Dask

• Nixtla

  • Distributed Forecast of 1M Time Series in Under 15 Minutes with Spark, Nixtla, and Fugue

2021

Fugue Core

• Fugue - Reducing Spark Developer Friction (James Le Blog)

• Creating Pandas and Spark Compatible Functions with Fugue (Towards Data Science)

Data Validation

• Using Pandera on Spark for Data Validation through Fugue (Towards Data Science)

FugueSQL

• Interoperable Python and SQL in Jupyter Notebooks (Towards Data Science)

• Data Analysis with FugueSQL on Coiled Dask Clusters (Coiled Blog)

• Introducing FugueSQL — SQL for Pandas, Spark, and Dask DataFrames (By Khuyen Tran on Towards Data Science)

Conferences and Meetups

2022

Fugue

• Comparing the Different Ways to Scale Python and Pandas Code (PyCon US)

• Comparing the Different Ways to Scale Python and Pandas Code (SciPy)

• Testing Big Data Applications (Spark, Dask, and Ray) (PyData NYC)

Machine Learning

• Distributed Machine Learning at Lyft

Tune

• Distributed Hybrid Parameter Tuning (Databricks Summit)

• An Introduction to Distributed Hybrid Hyperparameter Optimization (SciPy)

FugueSQL

• FugueSQL - The Enhanced SQL Interface for Pandas and Spark DataFrames (Databricks Summit)

2021

Data Validation

• Large Scale Data Validation with Spark and Dask (PyCon US)

• Fully Utilizing Spark for Data Validation (Spark AI Summit)

• Large Scale Data Validation with Fugue (PyData Global)

FugueSQL

• Dask SQL Query Engines (Dask Summit)

• FugueSQL: Extending SQL Interface for End-to-End Data Pipelines (Dremio Subsurface)

• FugueSQL - The Enhanced SQL Interface for Pandas, Spark, and Dask DataFrames (PyData Global)

Machine Learning

• Superworkflow of Graph Neural Networks with K8S and Fugue (Spark AI Summit)

• Scaling Machine Learning Workflows to Big Data with Fugue (KubeCon)

• Distributed ML to Learn Causal Effect Using Fugue and Spark (AI Camp)

Tune

• Intuitive and Scalable Hyperparameter Tuning with Apache Spark + Fugue (Spark AI Summit)

• Fugue Tune (PyData Global)

Testing Spark

• Simplifying Testing of Spark Applications (PyData Global)

2020

• Unifying Spark and Non-Spark Ecosystems for Big Data Analytics (Spark AI Summit)