cerndb/sparkplugins   0.3

Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.

hydrospheredata/spark-ml-serving   0.3.3

Spark ML Lib serving library

coxautomotivedatasolutions/spark-distcp   0.2

A re-implementation of Hadoop DistCP in Apache Spark

hablapps/sparkoptics   0.1.1

Optics for Spark DataFrames

locationtech-labs/geopyspark   0.3.0

GeoTrellis for PySpark

benfradet/struct-type-encoder   0.6.0

Deriving Spark DataFrame schemas from case classes

tharwaninitin/etlflow   1.7.3

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases, SFTP servers, On-Prem Systems and more.

absaoss/hyperdrive   4.7.0

Extensible streaming ingestion pipeline on top of Apache Spark

univalence/zio-spark   0.12.0

A functional wrapper around Spark to make it works with ZIO

g-research/spark-dgraph-connector   0.2.0

A connector for Apache Spark and PySpark to Dgraph databases.

xskipper-io/xskipper   1.6.0

An Extensible Data Skipping Framework

zuinnote/spark-hadoopoffice-ds   1.7.0

A Spark datasource for the HadoopOffice library

heartsavior/spark-sql-kafka-offset-committer   0.2.0

Kafka offset committer for structured streaming query

absaoss/spark-hats   0.3.0

Nested array transformation helper extensions for Apache Spark

tupol/spark-utils   0.6.2

Basic framework utilities to quickly start writing production ready Apache Spark applications

agile-lab-dev/darwin   1.2.2

Avro Schema Evolution made easy

music-of-the-ainur/almaren-framework   2.4.5-2.4.5

The Almaren Framework provides a simplified consistent minimalistic layer over Apache Spark. While still allowing you to take advantage of native Apache Spark features. You can still combine it with standard Spark code.

sansa-stack/archived-sansa-query   0.7.1

SANSA Query Layer

intenthq/pucket   1.7.3

Bucketing and partitioning system for Parquet

sansa-stack/archived-sansa-inference   0.7.1

A general Inference API based on two of the most popular Big Data processing engines: Apache Spark and Apache Flink