434: DSL Design with Macros
Difficulty: 4 Level: Expert Create a mini-language inside Rust using `macro_rules!` โ config literals, assertion DSLs, route declarations โ that reads like a domain-specific syntax while compiling to ordinary Rust.The Problem This Solves
Boilerplate kills readability. A config that takes 20 lines of `HashMap::insert` calls could be one declarative block. A test suite with bespoke assertion messages could use domain-specific vocabulary (`assert_that!(v, contains 3)`) instead of generic `assert!` calls. A web router could declare routes with HTTP-method keywords (`GET "/users" => handler`) rather than method calls. Rust's standard API is general-purpose. DSLs built on top of it can be narrow and expressive โ they look like the domain, not like programming. The payoff: code that non-experts can read, diffs that show intent rather than mechanics, and custom syntax that's impossible to misuse because the macro validates structure at compile time. `macro_rules!` is Rust's tool for this. Unlike C macros or Lisp macros, it matches structured token patterns โ keywords, operators, brackets โ not raw text. This lets you define a grammar fragment and expand it to arbitrary Rust code.The Intuition
A `macro_rules!` DSL works by defining patterns that match a custom syntax and produce standard Rust code. The trick is that Rust's token tree is flexible: identifiers, literals, operators, and punctuation can all appear in patterns. You can match `GET` as an identifier, `"/path"` as a string literal, and `=>` as a punctuation sequence โ then expand to whatever Rust you need. The caller writes syntax that feels like the domain. The macro translates it to idiomatic Rust. Zero performance overhead: the translation happens at compile time, and the output is ordinary code.How It Works in Rust
// โโ Config DSL โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// Call site: config!(host = "localhost", port = 8080i32, debug = true)
// Expansion: builds a HashMap<String, ConfigValue>
macro_rules! config {
($($key:ident = $value:expr),* $(,)?) => {{
let mut map = HashMap::new();
$(map.insert(stringify!($key).to_string(), ConfigValue::from($value));)*
Config(map)
}};
}
let cfg = config!(host = "localhost", port = 8080i32, debug = true);
// Reads like a config file; compiles to three HashMap inserts
// โโ Assertion DSL โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// Custom vocabulary for readable test assertions
macro_rules! assert_that {
($val:expr, equals $expected:expr) => {
assert_eq!($val, $expected, "Expected {} to equal {}", stringify!($val), stringify!($expected));
};
($val:expr, is_some) => {
assert!($val.is_some(), "Expected {} to be Some", stringify!($val));
};
($val:expr, contains $item:expr) => {
assert!($val.contains(&$item), "Expected {:?} to contain {:?}", $val, $item);
};
($val:expr, has_len $len:expr) => {
assert_eq!($val.len(), $len);
};
}
// Tests read like prose:
assert_that!(v, has_len 3);
assert_that!(v, contains 2);
assert_that!(config.get("host"), is_some);
// โโ Route DSL (keyword matching) โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
// Matching identifiers as "keywords" in the macro pattern
macro_rules! method_str {
(GET) => { "GET" };
(POST) => { "POST" };
(DELETE) => { "DELETE" };
}
// Full router! macro would expand to route table entries
Key DSL design principles:
1. Use `$(,)?` to allow trailing commas โ feels natural, prevents user frustration
2. Match identifiers as pseudo-keywords (`GET`, `equals`, `is_some`)
3. Use `stringify!($key)` to turn identifier tokens into string keys
4. Emit `compile_error!` for invalid input โ be a good language designer
5. Keep each arm's expansion self-contained and easy to expand by hand
What This Unlocks
- Configuration literals โ `config! { host = "...", port = 8080 }` looks like TOML but is compiled Rust with type checking.
- Test DSLs โ `assert_that!(x, equals y)` reads as English, produces better error messages, and guides reviewers who don't know Rust well.
- Embedded languages โ `json!({...})` (serde_json), `html!(<div>...</div>)` (yew), `sql!(SELECT ...)` โ all real crates built on this pattern.
Key Differences
| Concept | OCaml | Rust |
|---|---|---|
| Embedded DSL | `ppx` for AST transformation; `camlp5` for syntax extension | `macro_rules!` token matching; proc macros for full AST power |
| Config literals | No native syntax; use records or assoc lists | `macro_rules!` config DSL compiles to HashMap construction |
| Pattern vocabulary | ppx matches OCaml AST nodes | `macro_rules!` matches token trees โ identifiers, literals, punctuation |
| Compile-time validation | ppx can reject invalid forms | `compile_error!` in catch-all arm |
| Real-world examples | `ppx_sexp_conv`, `ppx_let` | `json!`, `html!`, `sql!`, `vec!`, `format!` |
// DSL design with macros in Rust
// Config DSL
macro_rules! config {
($($key:ident = $value:expr),* $(,)?) => {
{
use std::collections::HashMap;
let mut map: HashMap<String, ConfigValue> = HashMap::new();
$(map.insert(stringify!($key).to_string(), ConfigValue::from($value));)*
Config(map)
}
};
}
#[derive(Debug, Clone)]
enum ConfigValue {
Str(String),
Int(i64),
Float(f64),
Bool(bool),
List(Vec<ConfigValue>),
}
impl From<&str> for ConfigValue {
fn from(s: &str) -> Self { ConfigValue::Str(s.to_string()) }
}
impl From<i64> for ConfigValue {
fn from(n: i64) -> Self { ConfigValue::Int(n) }
}
impl From<i32> for ConfigValue {
fn from(n: i32) -> Self { ConfigValue::Int(n as i64) }
}
impl From<f64> for ConfigValue {
fn from(f: f64) -> Self { ConfigValue::Float(f) }
}
impl From<bool> for ConfigValue {
fn from(b: bool) -> Self { ConfigValue::Bool(b) }
}
use std::collections::HashMap;
struct Config(HashMap<String, ConfigValue>);
impl Config {
fn get(&self, key: &str) -> Option<&ConfigValue> { self.0.get(key) }
}
// Test assertion DSL
macro_rules! assert_that {
($val:expr, equals $expected:expr) => {
assert_eq!($val, $expected,
"Expected {} to equal {}", stringify!($val), stringify!($expected));
};
($val:expr, is_some) => {
assert!($val.is_some(), "Expected {} to be Some, got None", stringify!($val));
};
($val:expr, is_none) => {
assert!($val.is_none(), "Expected {} to be None", stringify!($val));
};
($val:expr, contains $item:expr) => {
assert!($val.contains(&$item),
"Expected {:?} to contain {:?}", $val, $item);
};
($val:expr, has_len $len:expr) => {
assert_eq!($val.len(), $len,
"Expected len {} but got {}", $len, $val.len());
};
}
// Route DSL
macro_rules! router {
($($method:ident $path:literal => $handler:expr),* $(,)?) => {
{
let routes: Vec<(&str, &str, Box<dyn Fn() -> String>)> = vec![
$(($method_str!($method), $path, Box::new($handler)),)*
];
routes
}
};
}
macro_rules! method_str {
(GET) => { "GET" };
(POST) => { "POST" };
(PUT) => { "PUT" };
(DELETE) => { "DELETE" };
}
fn main() {
// Config DSL
let cfg = config!(
host = "localhost",
port = 8080i32,
debug = true,
timeout = 30.0f64,
);
println!("host: {:?}", cfg.get("host"));
println!("port: {:?}", cfg.get("port"));
println!("debug: {:?}", cfg.get("debug"));
// Route DSL
let routes: Vec<(&str, &str, Box<dyn Fn() -> String>)> = vec![
("GET", "/users", Box::new(|| "[{"id":1}]".to_string())),
("POST", "/users", Box::new(|| "created".to_string())),
("GET", "/health", Box::new(|| "ok".to_string())),
];
println!("
Routes:");
for (method, path, handler) in &routes {
println!(" {} {} -> {}", method, path, handler());
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_config_dsl() {
let cfg = config!(name = "test", value = 42i32);
assert!(cfg.get("name").is_some());
assert!(cfg.get("value").is_some());
assert!(cfg.get("missing").is_none());
}
#[test]
fn test_assert_that() {
let v = vec![1, 2, 3];
assert_that!(v.len(), equals 3);
assert_that!(v, contains 2);
assert_that!(v, has_len 3);
let opt: Option<i32> = Some(42);
assert_that!(opt, is_some);
let none: Option<i32> = None;
assert_that!(none, is_none);
}
}
(* DSL design with macros in OCaml *)
(* Configuration DSL *)
type config_value =
| Str of string
| Int of int
| Bool of bool
| List of config_value list
type config = (string * config_value) list
(* DSL-like function to build config *)
let ( --> ) key value = (key, value)
let config_str s = Str s
let config_int n = Int n
let config_bool b = Bool b
let config_list items = List (List.map (fun s -> Str s) items)
(* Simulate a config DSL *)
let app_config : config = [
"host" --> config_str "localhost";
"port" --> config_int 8080;
"debug" --> config_bool true;
"allowed_origins" --> config_list ["http://localhost:3000"; "http://localhost:8080"];
]
let rec show_value = function
| Str s -> Printf.sprintf "%S" s
| Int n -> string_of_int n
| Bool b -> string_of_bool b
| List vs -> "[" ^ String.concat ", " (List.map show_value vs) ^ "]"
let () =
Printf.printf "Config:\n";
List.iter (fun (k, v) ->
Printf.printf " %s = %s\n" k (show_value v)
) app_config