🛑 DEPRECATED 🛑
The code under this repository has been moved under
connectors module in https://github.com/delta-io/delta repository. No changes are allowed to this repository. Any issues or changes, please propose them on the https://github.com/delta-io/delta repository. More details on deprecation are found at #556.
This is the repository for Delta Lake Connectors. It includes
- Delta Standalone: a native library for reading and writing Delta Lake metadata.
- Connectors to popular big-data engines (e.g., Apache Hive, Presto, Apache Flink) and to common reporting tools like Microsoft Power BI.
Please refer to the main Delta Lake repository if you want to learn more about the Delta Lake project.
Delta Standalone, formerly known as the Delta Standalone Reader (DSR), is a JVM library to read and write Delta tables. Unlike https://github.com/delta-io/delta, this project doesn't use Spark to read or write tables and it has only a few transitive dependencies. It can be used by any application that cannot use a Spark cluster.
- To compile the project, run
- To test the project, run
- To publish the JAR, run
See Delta Standalone for detailed documentation.
Use the Flink/Delta Connector to read and write Delta tables from Apache Flink applications. The connector includes a sink for writing to Delta tables from Apache Flink, and a source for reading Delta tables using Apache Flink (still in progress.) See the dedicated README.md for more details.
sql-delta-import allows for importing data from a JDBC source into a Delta table.
Power BI connector
The connector for Microsoft Power BI is basically just a custom Power Query function that allows you to read a Delta table from any file-based data source supported by Microsoft Power BI. Details can be found in the dedicated README.md.
We welcome contributions to Delta Lake Connectors repository. We use GitHub Pull Requests for accepting changes.
There are two mediums of communication within the Delta Lake community.
Local Development & Testing
- Before local debugging of
standalonetests in IntelliJ, run all
standalonetests using SBT. This helps IntelliJ recognize the golden tables as class resources.