Search *

7 results

swoop-inc/spark-alchemy 1.2.1

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

Scala versions: 2.12

data-engineering data-science scala spark

182 7
setl-framework/setl 1.0.0-SNAPSHOT

A simple Spark-powered ETL framework that just works 🍺

Scala versions: 2.12 2.11

data-engineering etl-pipeline etl pipeline data-science spark scala machine-learning setl framework big-data modularization data-transformation dataset data-analysis

177 7
treeverse/lakefs 0.14.1

lakeFS - Data version control for your data lake | Git for data

Scala versions: 2.12

apache-sparksql data-version-control data-engineering datalake aws-s3 data-quality go lakefs data-lake git-for-data azure-storage object-storage hadoop-filesystem golang apache-spark data-versioning datalakes google-cloud-storage azure-blob-storage

4349 89
galliaproject/gallia-core 0.6.1

A schema-aware Scala library for data transformation

Scala versions: 3.x 2.13 2.12

data-manipulation data-engineering nesting json etl data-science spark scala data-transformation feature-engineering

84 1
coxautomotivedatasolutions/waimak 2.9.7

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Scala versions: 2.13 2.12

data-engineering hadoop scala spark

75 9
coxautomotivedatasolutions/spark-distcp 0.2.5

A re-implementation of Hadoop DistCP in Apache Spark

Scala versions: 2.13 2.12 2.11

apache-spark data-engineering distcp hadoop spark

42 3
harrystech/hyppo-worker 0.7.5

The hyppo data ingestion system worker components

Scala versions: 2.11

data-engineering

0 1