onema / vff   0.6.1

Apache License 2.0 GitHub

Virtual File Framework inspired by FlySystem

Scala versions: 2.13 2.12 2.11

Virtual File Framework (VFF)

CodeBuild Badge Codacy Badge Codacy Badge LICENSE

A File System abstraction for scala inspired by FlySystem.

Currently two adapters are provided:

  • Local (included with VFF)
  • S3 (as of version 0.6.0 must be installed separately "io.onema" % "vff-s3-adapter_2.12" % "VERSION")

Getting started


for the latest version please visit

libraryDependencies += "io.onema" % "vff_2.12" % "LATEST_VERSION"

Create a file system object

import io.onema.vff.FileSystem
import io.onema.vff.adapter.{Local, AwsS3Adapter}

// Local file system
val fs = new FileSystem(new Local)

// S3 file system
val s3fs = new FileSystem(AwsS3Adapter("bucket-name"))

Reading a file

All read operations return an Option:


val data: Option[InputStream] ="path/to/file.txt")
val str: String = data.mkString

Other convenience methods are provided to read the data as a string or as an iterator of strings


val strOption: Option[String] = fs.readAsString("path/to/file.txt")


val iterOption: Option[Iterator[String]] = fs.readAsIterator("path/to/file.txt")

Creating a file

VFF uses byte iterators to write to the destination


import io.onema.vff.extensions.StreamExtension._

// If you have an input stream you can convert it using the stream extensions"path/to/image0.png").foreach((image: InputStream) => {
  val result = fs.write("path/to/image1.png", image.toBytes)
  val copyResult = fs.copy("path/to/image1.png","path/to/image1.png")

// A convenience method is included to write strings directly
fs.write("path/to/file2.txt", "Some text")

Extension Methods


Using the InputStream methods and extensions provided in the you can quickly convert the data returned by the read method into other types

import io.onema.extensions.StreamExtension._

val data ="path/to/file.txt")
val str: String = data.mkString
val byteIter: Iterator[Byte] = data.toBytes
val byteArray: Array[Byte] = data.toByteArray
val lines: Iterator[String] = data.getLines

Async API

As of version 0.6.0 an Async file system is provided, the method names remain the same but all calls async and return a scala Future Future[ReturnType]. Implementations for each of the adapters are provided.


import java.util.UUID
import scala.concurrent.Await
import scala.concurrent.duration._
import io.onema.vff.FileSystemAsync

    // Arrange
    val fs = FileSystemAsync()
    val uuid = UUID.randomUUID().toString
    val dir = s"/tmp/$uuid"
    // create a single file, this will not be part of the test
    Await.result(fs.write(s"$dir/tmp", "0"), 500.millis)

    // Act
    val results: Seq[Boolean] = (0 to 100000).map { i =>
      fs.write(s"$dir/$i", i.toString)
    }.map(Await.result(_, 5000.millis))

    // Assert
    results.foreach(x => {
      x should be (true)