David J. Phillips @davj / 2:30 PM PST • March 2, 2023

Today, DAGWorks launched on Y Combinator’s "Launch YC

Launch YC: DAGWorks: Avoid Machine Learning Pipeline Horror Stories 😱

DAGWorks is building a more human capital efficient way for Data Science teams to build and maintain Machine Learning Pipelines, from creation to maintenance.

Founded by Stefan Krawczyk & Elijah ben Izzy, the pair built the self-service ML Platform for 100+ data scientists at Stitch Fix. They learned firsthand the challenges of building and maintaining ML pipelines across diverse modeling disciplines, as maintaining these pipelines becomes increasingly difficult as a team and codebase grows. The team's background includes building tools, quants, data & ML systems at Nextdoor, LinkedIn, and Two Sigma.


DAGWorks' solution is an open source SaaS platform for data science teams to build and maintain model pipelines. It integrates with as much of your existing MLOps and data infrastructure to the extent that you want.

Debugging Insights, Image Credits: DAGWorks

Hamilton, DAGWorks' open source project that helps teams follow software engineering best practices, enables users to logically express actions, preventing vendor and infrastructure lock-in.

Lineage: see upstream & downstream dependencies, Image Credits: DAGWorks

Using the DAGWorks Platform & Client, your Hamilton code runs on your existing infrastructure, providing out-of-the-box connectors to common systems, and post-execution insights.

Catalog of Functions, Data Observability, & Links w/ Materialized Artifacts, Image Credits: DAGWorks

The DAGWorks Platform serves as your ML Pipeline control plane. Building on a solid foundation that clearly decouples concerns, traditional issues with ML Pipelines can be avoided, enabling Data Science teams to work faster, more independently, and more effectively. The result? Greater human capital efficiency for ML projects.

Save time on managing ML ETLs and focus more on developing features and models. Eliminate the hassle of dealing with convoluted code, unclear data dependencies, and isolated knowledge silos.


Want updates? Follow them on Twitter: @DagWorks, @hamilton_os

Sign up for early access at www.dagworks.io (closed beta)

Star their open source project Hamilton on github and get started via tryhamilton.dev.

If you know of any larger organizations experiencing difficulties with their ML pipelines, particularly those with compliance or auditing needs, contact the DAGWorks team!

March 2, 2023
← Back to all posts  

Join Our Newsletter and Get the Latest
Posts to Your Inbox

No spam ever. Read our Privacy Policy
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.