chermenin/spark-states   0.2

Custom state store providers for Apache Spark

galliaproject/gallia-core   0.6.1

A schema-aware Scala library for data transformation

helgeho/archivespark   3.0

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

kakao/cuesheet   0.10.0

A framework for writing Spark 2.x applications in a pretty way

sansa-stack/sansa-stack   0.9.5

Big Data RDF Processing and Analytics Stack built on Apache Spark and Apache Jena http://sansa-stack.github.io/SANSA-Stack/

dwestheide/kontextfrei   0.7.0

Writing application logic for Spark jobs that can be unit-tested without a SparkContext

simplexspatial/osm4scala   1.0

Scala and Spark library focused on reading OpenStreetMap Pbf files.

hablapps/doric   0.0.8

Type safety for spark columns

coxautomotivedatasolutions/waimak   2.9.7

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

swoop-inc/spark-records   3.0.1

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

jelmerk/hnswlib   1.2.1

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

mskimorg/spark-annoy   0.1.5

Building Annoy Index on Apache Spark

samelamin/spark-bigquery   0.2.7

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

googleclouddataproc/spark-bigquery-connector   0.42.4

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

mrpowers/spark-stringmetric   0.5.0

Spark functions to run popular phonetic and string matching algorithms

yaooqinn/itachi   0.3.0

A library that brings useful functions from various modern database management systems to Apache Spark

databrickslabs/automl-toolkit   0.7.2

Toolkit for Apache Spark ML for Feature clean-up, feature Importance calculation suite, Information Gain selection, Distributed SMOTE, Model selection and training, Hyper parameter optimization and selection, Model interprability.

pishen/sbt-lighter   1.2.0

SBT plugin for Apache Spark on AWS EMR

cerndb/sparkplugins   0.4

Code and examples of how to write and deploy Apache Spark Plugins. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.

potix2/spark-google-spreadsheets   0.6.3

Google Spreadsheets datasource for SparkSQL and DataFrames