#apache-spark

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

lakeFS - Data version control for your data lake | Git for data