apache/spark   4.0.0

Apache Spark - A unified analytics engine for large-scale data processing

com-lihaoyi/mill   1.0.0

Mill is a better build tool for Java, Scala and Kotlin: 3-6x faster than Maven or Gradle, less fiddling with plugins, and more easily explorable in your IDE

apache/flink   1.20.2

Apache Flink

combust/mleap   0.23.3

MLeap: Deploy ML Pipelines to Production

bigdatagenomics/adam   0.23.0

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

lucacanali/sparkmeasure   0.25

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of Spark metrics, making it a practical choice for both developers and data engineers.

apache/sedona   1.7.2

A cluster computing framework for processing large-scale geospatial data

scalapy/scalapy   0.5.2

Use the world of Python from the comfort of Scala!

aws/sagemaker-spark   spark_2.4.0-1.4.2.dev0

A Spark library for Amazon SageMaker.

g-research/spark-extension   1.0.0

A library that provides useful extensions to Apache Spark and PySpark.

catboost/catboost   1.2.8

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

locationtech-labs/geopyspark   0.3.0

GeoTrellis for PySpark

shadaj/scalapy-tensorflow   0.1.0

Static facades for using TensorFlow in ScalaPy

isarn/isarn-sketches-spark   0.6.0-sp3.2

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

scalapy/python-native-libs   0.2.4

Helpers for setting up an embedded Python interpreter

fqaiser94/mse   0.2.4

Make Structs Easy (MSE)

jelmerk/hnswlib-spark   1.1.3

Spark library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

ozancicek/artan   0.5.1

Online latent state estimation with Spark

timvw/adobe-analytics-datafeed-datasource   0.1.0

Apache Spark data source for Adobe Analytics Data Feed

diana-hep/spark-root   0.1.16

Apache Spark Data Source for ROOT File Format