#big-data

Apache DataFusion SQL Query Engine

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Apache DataFusion Ballista Distributed Query Engine

Extensible SQL Lexer and Parser for Rust

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

The Open Source Feature Store for AI/ML

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

The official home of the Presto distributed SQL query engine for big data

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Apache Beam is a unified programming model for Batch and Streaming data processing.

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

Apache Spark - A unified analytics engine for large-scale data processing

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

Cloud-native OSS search engine for observability

The AI search platform

Apache TsFile

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)

Apache Paimon Rust The rust implementation of Apache Paimon.

The most widely used Python to C compiler

🚄 FASTJSON2 is a Java JSON library with excellent performance.

Apache Flink