A SBT plugin to generate typescript definitions from thrift files

This plugin leverages Twitter's Scrooge plugin to resolve thrift dependencies.

The Guardian already uses Scrooge to generate scala classes from thrift files, and we also heavily relies on Scrooge to model the dependencies between projects (thrift files are published to maven).

We're now using typescript more and more, so the intent of this plugin is to allow us to hit the ground running and keep our project structure.

This plugin depends on the Scrooge SBT plugin. The dependency resolution and jar unpacking is left to the scrooge plugin, but the typescript generation is handled by this plugin (via scrooge). We've essentially implemented a new backend for the scrooge compiler, and wrapped it into an sbt plugin for convenience.

• Thrift services. By lack of time, it's a matter of implementing it.
• Thrift annotations. Similarly by lack of time.
• Thrift's typedefs will work as expected, however they really should match Typescript's type aliasing feature. I was unable to declare type aliases as in the backend of the scrooge compiler that information is missing.
• Keyword protection isn't implemented as I haven't found any case where it was necessary yet.

• Generate typescript source from thrift files
  • Type mapping
  • Thrift namespaces are mapped to npm packages
• Generate a package.json
• Generate a tsconfig.json
• Publish to NPM

Namespaces are mapped to npm packages. There are limitations on what characters a thrift namespace can contains, so the namespace string gets transformed to generate the package name. The transformations are the following:

• _at_ is replaced by @
• _ is replaced by - to be closer the the NPM naming style
• . are replaced by /

For instance the namespace _at_guardian.school.common will point to the folder common of the package @guardian/school.

In order to be backward compatible with the thrift CLI, the scrooge namespaces are prefixed by the # symbol, making them look like a comment to the CLI, but they are indeed processed by scrooge.

#@namespace typescript _at_guardian.school.common

Enums are directly mapped to typescript, here's an example of generated enum:

enum Type {
  ROBOT
  HUMAN = 4
  DOG
  CAT
}

export enum Type {
    ROBOT = 0,
    HUMAN = 4,
    DOG = 5,
    CAT = 6,
}

structs are mapped to interfaces.

struct Student {
  1: required Denomination denomination
  2: required i32 age
  3: optional set<i32> grades = [0, 4]
  5: optional Type type
}

export interface Student {
    denomination: Denomination
    age: number
    grades: number[] //non optional as there's a default value
    type?: Type
}

Unions are handled with typescript's union types of a set of anonymous interfaces. An extra attribute named kind is added to these interfaces in order to discriminate with a switch statement in the client code.

union Denomination {
  1: string fullName
  2: string nickName
  3: i32 barcode
}

export type Denomination =
    {
        kind: "fullName",
        fullName: string
    } |
    {
        kind: "nickName",
        nickName: string
    } |
    {
        kind: "barcode",
        barcode: number
    } ;

Not supported yet

You need the following installed and configured:

• tsc
• npm

Assuming your project is already set up with scrooge to generate scala files

In project/plugins.sbt

addSbtPlugin("com.twitter" % "scrooge-sbt-plugin" % "20.4.1")

addSbtPlugin("com.gu" % "sbt-scrooge-typescript" % "<latest_version>")

In build.sbt

enablePlugins(ScroogeTypescriptGen)
scroogeLanguages in Compile := Seq("typescript"),
scroogeTypescriptNpmPackageName := "@guardian/mymodule",

// although it might appear odd at first, these two lines are use to tell the scrooge compiler what NPM package we are generating
// "scroogeDefaultJavaNamespace" is actually badly named as it is the "defaultNamespace" option passed to the compiler
Compile / scroogeDefaultJavaNamespace := scroogeTypescriptNpmPackageName.value,
Test / scroogeDefaultJavaNamespace := scroogeTypescriptNpmPackageName.value,
scroogeTypescriptPackageLicense := "Apache-2.0",
scroogeTypescriptPackageMapping := Map(
  "scala-library-name" -> "@org/npm-module-name"
)

Then you're ready to generate and compile your models

compile

You can control more finely what part of the generation is triggered with the following tasks:

• scroogeTypescriptGenPackageJson: Generates the package.json
• scroogeTypescriptGenNPMPackage: Generates the typescript, package.json and copy the README.md (if any)
• scroogeTypescriptCompile: Runs npm install and tsc to check the generated files are compiling
• scroogeTypescriptNPMPublish: Runs npm publish --access public to publish the package on NPM

Here are some settings you may want to use:

• scroogeTypescriptNpmPackageName: The name of the package in package.json
• scroogeTypescriptPackageLicense: The license of the package on NPM
• scroogeTypescriptPackageMapping: The mapping between scala libraries and their NPM equivalent
• scroogeTypescriptDevDependencies: The dev dependencies to put in the package.json
• scroogeTypescriptDependencies: The dependencies to put in the package.json
• scroogeTypescriptDryRun: To be able to run everything without publishing to NPM
• scroogeTypescriptPublishTag: An optional tag for your NPM release, primarily for beta.

With each struct or union in the thrift definitions, two types are generated.

For instance, for the struct School, you'll have a generated interface called School, and a generated SchoolSerde class that will be able to serialise and deserialise objects of type School. Hence the suffix "Serde" for SERialization-DEserialization.

const protocol: TProtocol = ...;
const school = SchoolSerde.read(protocol);

const protocol: TProtocol = ...;
SchoolSerde.write(protocol, school);

We have an end-to-end tests that takes a scala model, serialises it from scala, deserialises it from typescript, serialises it from typescript and finally deserialises it from scala. This ensures the value sent and reveived are the same.

Any contribution should ensure the tests are running, in the sbt shell:

test

Releases are done using gha-scala-library-release-workflow release process. The details are here: https://github.com/guardian/gha-scala-library-release-workflow/blob/main/docs/making-a-release.md