apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

apache/flink

Apache Flink

combust/mleap

MLeap: Deploy ML Pipelines to Production

bigdatagenomics/adam

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

lucacanali/sparkmeasure

This is the development repository for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task and stage metrics data.

scalapy/scalapy

Use the world of Python from the comfort of Scala!

apache/sedona

A cluster computing framework for processing large-scale geospatial data

g-research/spark-extension

A library that provides useful extensions to Apache Spark and PySpark.

locationtech-labs/geopyspark

GeoTrellis for PySpark

shadaj/scalapy-tensorflow

Static facades for using TensorFlow in ScalaPy

isarn/isarn-sketches-spark

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

fqaiser94/mse

Make Structs Easy (MSE)

scalapy/python-native-libs

Helpers for setting up an embedded Python interpreter

diana-hep/spark-root

Apache Spark Data Source for ROOT File Format

ozancicek/artan

Online latent state estimation with Spark

ctongfei/kingscross

Portal from the magical world of Scala to dangerous external interpreters outside JVM

liquidsvm/liquidsvm

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.

cdapio/cdap

An open source framework for building data analytic applications.

mrdimosthenis/synapses

A group of neural-network libraries for functional and mainstream languages

eclipse/deeplearning4j

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.