#spark

Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

An open protocol for secure data sharing

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.

A whitespace formatter for different query languages

Local Microsoft Fabric emulator in one Go binary. Control plane, OneLake, T-SQL warehouse and Spark, verified against unmodified Microsoft tools.

Nessie: Transactional Catalog for Data Lakes with Git-like semantics

SQL Parsers for BigData, built with antlr4.

Python SDK to interact with the Hopsworks API

多平台模型接入，可扩展，多种输出格式，提供大语言模型聊天服务的插件 | A bot plugin for LLM chat with multi-model integration, extensibility, and various output formats

Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

Workbench: An easy to use Python API for creating and deploying AWS SageMaker Models

Expressive types for Spark.

Tools that make it easier to use FHIR and clinical terminology within data analytics, built on Apache Spark.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.