I gotta have macros!

Anyone who's worked with Rust (opens new window) knows that a huge chunk of why it's so excellent is the macro system. When using a statically typed language, there are simply so many programs that will have mountains of boilerplate if they can't automatically generate code.

Why is this the case? Well, when you've committed to using a typesafe language (which you should), there are many small tasks (such as mapping one type to another, or validating a type, or constructing a type with defaults) where the code simply mirrors the structure of the types, and can easily be generated by a rote algorithm. With this capability, the potential tedium of writing typesafe code can be largely removed, and the language is a joy. But without it, the language can become a chore. Typescript is one such language, and it's insanely incomplete without real macros.

Since WebAssembly can't directly interact with the dom (yet) (opens new window), it's an unfortunate reality for some of us that we can't use Rust in all our projects. So when we need to write browser code that performs well, typescript is the only available option that's acceptably typesafe (I'm a bit of a typesafety fundamentalist 😅). But typescript doesn't have macros (opens new window), and it doesn't seem the core team is terribly interested in the idea. So I decided to do it myself.

You may be saying: but typescript does have macros! (opens new window) However, it's very important to notice that the transformation process allowed by the typescript compiler doesn't typecheck the transformed output. This makes it unsafe and not very useful.

Mapped (opens new window) and Conditional (opens new window) types can do some of the work that traditional macros would usually do, but they aren't nearly complete. They can only transform the program on a type level, which basically just guards the dynamic nature of javascript with (leaky) type safety. True meta-programming, in contrast, can produce actual code.

Enter macro-ts

macro-ts is my attempt to solve the above problems, at least in the short term.

Here's the hypothetical example given in the project README (opens new window), which demonstrates all four types of macros: import, block, decorator, and function.

You could write code like this:

// an import macro
import ProductDetails from sql!!('./productDetails.sql')

// a block macro
db!!;{
  port = 5432; host = 'db'
  user = DB_ENV_USER
  password = DB_ENV_PASS
}

// a decorator macro
@get!!('/product/{id}')
async function productDetails(id: number) {
  const queryResult = await db.query(ProductDetails.compile(id))
  // a function macro
  const products = required!!(queryResult)
  return { products }
}

and have it transformed into something like the following, and then fully typechecked:

// import macros can inspect the contents of real files
// and produce "virtual" typescript files
import ProductDetails from './productDetails.sql.ts'

// block macros can inspect a list of statements
// and expand to any other list of statements
import driver from 'some-database-driver-library'
const dbUser = process.env.DB_ENV_USER
if (!dbUser) throw new Error(`DB_ENV_USER isn't set`)
const dbPassword = process.env.DB_ENV_PASS
if (!dbPassword) throw new Error(`DB_ENV_PASS isn't set`)
const db = new driver.Pool({
  port: 5432, host: 'db',
  user: dbUser, password: dbPassword,
})

// a decorator macro can inspect the statement it's attached to
// choose to replace that statement
// and provide additional statements to place around it
import * as v from 'some-validation-library'
const productDetailsParamsValidator = v.object({ id: v.number })
async function productDetails(params: { [key: string]: unknown }) {
  const paramsResult = productDetailsParamsValidator.validate(params)
  if (paramsResult === null)
    throw new v.ValidationError(params)
  const { id } = paramsResult

  const queryResult = await db.query(ProductDetails.compile(id))
  // function macros can inspect the args it's passed
  // return any expression to replace the function call
  // and provide additional statements to place around it
  if (queryResult === null)
    throw new Error()
  const products = queryResult.value
  return { products }
}
app.get('/product/{id}', productDetails)

As you can see, a lot of tedium can be removed from your code, while still retaining all the delights of typesafety.

Some reasons I'm excited about this project:

• Makes typesafe "loaders" extremely intuitive. I don't terribly like webpack, and it's especially irritating that typesafety is such an afterthought when processing files in other languages. macro-ts could allow things like arbitrary domain specific languages or file preprocessing, all in a typesafe way. The implications of this flexibility are vast.
• Simply removing verbosity. I very often find myself wanting constructor or validation functions for type aliases, or wanting to trim down some pattern of steps that are all necessary for typesafety. macro-ts enables nice clean reusable solutions for these situations.
• We don't have to compromise so often between performance and readability. Macros can be written that wrap the most efficient version of a pattern in an abstraction that looks like a less efficient one. For performance critical hotspots that don't lend themselves to a webassembly shim, this could be a life-saver.
• Finally having a convenient way to interact with typescript code. tsc has always been missing both some kind of check command that typechecks the code without actually compiling it, and a run command. ts-node did a great job filling that latter gap (and the run command in macro-ts uses basically the same internal mechanisms as ts-node 😇), but it's nice to have one tool that unifies all that functionality. Also, I really don't like json config files. json is such a cluttered and inflexible language to use for human consumption, so the config for macro-ts is in toml.

Why ident!!? It's so ugly!

The syntax of the macros has to fit a few critera, which is made more difficult by the fact that macro-ts is an unsanctioned hack on top of typescript. The syntax must be:

• Accepted by the typescript parser. We can't change the parser, so we have to hijack some existing syntax.
• Visually obvious and explicit. A macro usage should be obviously different than the surrounding syntax.
• Unambiguous. A macro usage shouldn't conflict with some useful syntax that people are already using.

Typescript 2.0 added the non-null assertion operator (opens new window), which one can argue shouldn't be used since it cancels the benefits of the strict null-checking mode (opens new window) and can introduce unsafety. However there are times when it's reasonable, so we shouldn't just entirely hijack the operator.

However using the operator twice never really makes sense, and it's still technically valid according to typescript. So of all the options I was considering, !! seemed the most reasonable.

It also doesn't hurt that !! mimics the Rust macro syntax (opens new window).

Could this be used for web app bundling?

Yeah I think so! Import macros certainly offer that possibility, and as I was designing how they work I kept potential applications like web app bundling in mind.

Right now I don't think it would be terribly easy to do, since I was just focused on getting this typescript-first application out the door. But in the near future I plan to generalize some of the compilation functions so macro-ts could be the foundation of arbitrary specialized compilers, like a typesafe web application bundler.

Caveat Utilitor

This project is extremely rough around the edges. I wanted to get this project in front of people for feedback sooner rather than later, so I haven't really put in the elbow grease yet.

• The macro transformation process completely ruins sourcemaps and type error location values. This is a tricky question that I'm not sure how to completely solve. However, personally I prefer having macros with misaligned type error messages than not having macros at all.
• Many incredibly obvious performance improvements haven't been made. Right now the cli does a lot of avoidable duplicated work, doesn't cache all accesses to the filesystem, and isn't thinking about using efficient algorithms even a little bit.
• The cli doesn't parse command line arguments very intelligently.
• The cli outputs are ugly and inconsistent. I've improved the logging quite a bit, but still, I'm no designer.
• The dev mode concept I've implemented so far is too strict and inflexible, and probably not really useful.
• Internally the code is not as organized as I would prefer.

Feedback and pull requests are welcome! This project exists mainly to make my life easier, so whether a pull request gets merged will depend more on if I like it than if it's a reasonable idea. I'm sure we can still get along 🤗

Enjoy!

I look forward to hearing from you!

Once again here's the repo: macro-ts (opens new window).