Data Transformation Language

What is DTL?

DTL is a language for extracting, manipulating and transforming data. It works anywhere you can use Javascript, on the server, in your node.js code, or even in the browser (you can try it out below). It also includes CLI tools for transforming data in bulk (dtl) and for exploring the syntax (dtlr).

DTL allows you to describe how to transform one set of data into another. DTL can be used within your backend or frontend code to transform your data between formats, validate input data, or perform complex calculations. It can also be used to reliably manipulate huge amounts of data both in batch processing and in realtime scenarios.

In short, DTL is useful whenever you need to generate a new piece of data using data you already have.

DTL is great for handling input data from forms or API calls, it's fantastic for converting between data formats, and is tailor-made for transforming your data to and from formats used by the APIs you use.

Also, if you are into that whole ETL thing, DTL is amazing.

The dtl command line tool is like having all the *nix text tools (like grep, awk, sed, cut etc.) for your complex data structures, and works with JSON, yaml, and csv data.

Finally, the dtlr REPL tool has built-in help and is great for trying things out on the command line.

TL;DR

dtlr REPL demo

Try it out!

You can try DTL below. Select an example to auto-fill with sample data and related DTL transform, then click the button to apply the DTL transform. Your result will be displayed in the Result box. (Pro tip: Try editing the transform or input data to change the result.)

Load an example:

Input data
DTL Transform
Result data

DTL Docs

How do I get DTL?

Installing DTL is easy...

$ npm install -g dtl-js

To use DTL in your site or browser-based app, simply use the dtl-transform-browser.min.js from the npm package, or include it from the jsdelivr.net CDN:

<script src="https://cdn.jsdelivr.net/npm/dtl-js/src/js/dist/dtl-transform-browser.min.js"> </script> <script> let result = DTL.apply_transform(input_data, transform); </script>

Usage

To use DTL to experiment, you can load the dtlr REPL environment by issuing the following command:

$ dtlr

Once the REPL environment is loaded, you can enter the command .help to get general help, or .help followed by the name of a helper function to get help on that function, for example:.help grep will show you the documentation for the grep function.

How do I use it in my code?

To use DTL in your code, simply require the DTL module in your code:

const DTL = require("dtl-js"); let transform = { "out": "(: $. :)" }; let input = { /* your input data */ }; let result = DTL.apply_transform(input, transform);

You can browse the DTL npm page for a longer example.

How do I use the dtl CLI tool

The dtl command line tool allows you to apply complex (or simple) DTL transforms to data on the command line without writing any code. It understands JSON, JSONLines, YAML and CSV data for both input and output. Issue the command below for help:

$ dtl -h

To convert data from yaml to JSON:

$ dtl -o data.json data.yaml

To convert data from JSON to yaml:

$ dtl -o new.yaml data.json

Take JSON input on stdin, and output YAML on stdout

$ dtl -I json -O yaml

To extract names from a list of people

$ dtl -e 'map($people "(: $item.name :)")' astros.json

More DTL docs

DTL has extensive documentation on its syntax and helpers. You can find information on the expression syntax here and information about the helper functions within DTL in the helper reference. You can also try out dtl and get help in the dtlr command line tool.

Talk to us!

If you would like to chat about DTL and related topics, you can find us on Discord here.

Who made this thing?!?

That'd be mainly this guy, Jay Kuri with a little help from his friends