Awesome Scala

Data Sources and Connectors

delta-io/delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Scala versions: 2.11 2.12 2.13

big-data delta-lake spark acid analytics

6855 1567 293
datastax/spark-cassandra-connector

DataStax Connector for Apache Spark to Apache Cassandra

Scala versions: 2.10 2.11 2.12 2.13

cassandra scala spark

1929 914 11 95
akka/alpakka-kafka

Alpakka Kafka connector - Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Scala versions: 2.10 2.11 2.12 2.13 3.x

akka reactive-streams reactive java kafka akka-streams scala kafka-connector kafka-client

1418 388 102
akka/alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.

Scala versions: 2.11 2.12 2.13 3.x

hacktoberfest integration messaging akka reactive-streams reactive akka-streams-connectors akka-stream

1261 649 109 307
redislabs/spark-redis

A connector for Spark that allows reading and writing to/from Redis cluster

Scala versions: 2.11 2.12

dataframe java redis spark

932 361 18
databricks/spark-csv

CSV Data Source for Apache Spark 1.x

Scala versions: 2.10 2.11

1048 446 32
databricks/spark-redshift

Redshift data source for Apache Spark

Scala versions: 2.10 2.11

600 350 26
databricks/spark-xml

XML data source for Spark SQL and DataFrames

Scala versions: 2.10 2.11 2.12 2.13

483 222 32
databricks/spark-avro

Avro Data Source for Apache Spark

Scala versions: 2.10 2.11

540 310 30
microsoft/sql-spark-connector

Apache Spark Connector for SQL Server and Azure SQL

Scala versions: 2.12

261 109 12
memsql/singlestore-spark-connector

A connector for SingleStore and Spark

Scala versions: 2.10 2.11 2.12

158 54 12
delta-io/connectors

This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake.

Scala versions: 2.11 2.12 2.13

apache-flink apache-hive java delta-lake scala prestodb trino

382 159 29
couchbase/couchbase-spark-connector

The Official Couchbase Spark Connector

Scala versions: 2.10 2.11 2.12

119 43 15
scullxbones/akka-persistence-mongo

Implementation of akka-persistence storage plugins for mongodb

Scala versions: 2.10 2.11 2.12 2.13

read-journals akka-persistence akka scala mongodb

103 55 28
indix/schemer

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Scala versions: 2.11

schema-registry json parquet tsv avro spark graphql-api schema-inference

112 6 4
tuplejump/kafka-connect-cassandra

Kafka Connect Cassandra Connector. This project includes source/sink connectors for Cassandra to/from Kafka.

Scala versions: 2.10 2.11

78 40 3
springml/spark-sftp

Spark connector for SFTP

Scala versions: 2.10 2.11

98 95 0 6
googleclouddataproc/spark-bigquery-connector

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Scala versions: 2.11 2.12 2.13

spark google-cloud-dataproc google-cloud google-bigquery bigquery bigquery-storage-api

348 189 63
monix/monix-connect

A set of connectors for Monix. 🔛

Scala versions: 2.12 2.13 3.x

workflow dynamodb elasticsearch reactive-streams sqs scala monix redis mongodb google-cloud-storage hdfs aws s3 connectors parquet

60 17 13
zuinnote/hadoopoffice

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

Scala versions: 2.11 2.12

bigdata poi office hadoop spark hadoopoffice hadoop-ecosystem flink excel analyze-office-documents hive

64 31 1