427: syn + quote for Proc Macro AST

Difficulty: 4 Level: Expert The two crates that make proc macros practical: `syn` parses Rust token streams into a usable AST, `quote` generates Rust code from templates.

The Problem This Solves

A proc macro receives a raw `TokenStream` — a flat sequence of tokens. You can't usefully inspect or generate code working at that level. You need to know: is this a struct or an enum? What are the field names and types? Does this field have a `#[serde(rename)]` attribute? `syn` parses a `TokenStream` into a full Rust AST with named structs: `DeriveInput`, `ItemFn`, `Expr`, `Type`, and hundreds more. You navigate the AST with Rust pattern matching — natural, safe, with compile-time exhaustiveness. `quote!` is the inverse: write Rust code as a template with `#variable` interpolation and `#(repetition)*` patterns, and get a `TokenStream` back. It's the cleanliest way to generate code — you write what the output should look like, not how to construct it token by token. Together they're the foundation of essentially every non-trivial proc macro in the ecosystem.

The Intuition

`syn` turns tokens → AST (read code), `quote!` turns template → tokens (write code) — together they're the read/write API for proc macros.

How It Works in Rust

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, Data, Fields};

#[proc_macro_derive(Getters)]
pub fn derive_getters(input: TokenStream) -> TokenStream {
 // syn: parse raw tokens into structured AST
 let input = parse_macro_input!(input as DeriveInput);

 let struct_name = &input.ident;           // Ident: the type name
 let generics = &input.generics;           // Generic parameters

 // Navigate the AST
 let fields = match &input.data {
     Data::Struct(s) => match &s.fields {
         Fields::Named(f) => &f.named,
         _ => panic!("Only named fields supported"),
     },
     _ => panic!("Only structs supported"),
 };

 // quote: generate code with template interpolation
 let getters = fields.iter().map(|f| {
     let name = f.ident.as_ref().unwrap();  // field name as Ident
     let ty = &f.ty;                         // field type
     quote! {
         pub fn #name(&self) -> &#ty {       // #name interpolated
             &self.#name
         }
     }
 });

 // #(#getters)* — expand the iterator of token streams
 quote! {
     impl #struct_name {
         #(#getters)*
     }
 }.into()
}

// Usage:
#[derive(Getters)]
struct User { name: String, age: u32 }

let u = User { name: "Alice".into(), age: 30 };
println!("{}", u.name());  // getter generated by macro

Key `syn` types:

`DeriveInput` — top-level parse target for derive macros
`ItemFn` — for attribute macros on functions
`Ident` — a name/identifier
`Type` — a type expression
`Expr` — an expression

Key `quote!` patterns:

`#name` — interpolate a variable
`#(#items)*` — repeat over an iterator
`#(#items),*` — comma-separated repetition

What This Unlocks

Struct field iteration: Read all fields, their types, their attributes — generate per-field code.
Span-preserving errors: `syn::Error::new(field.span(), "message")` points errors at the right source location.
Full AST access: Generics, lifetimes, where clauses, visibility — all accessible and usable in generation.

Key Differences

Concept	OCaml	Rust
AST parsing	`Ppxlib.Ast` (built-in OCaml AST)	`syn` crate — parses `TokenStream`
Code generation	`Ppxlib.Ast_builder`	`quote!` macro — template-based
Field access	`type_declaration.ptype_manifest`	`DeriveInput` → `Data::Struct` → `Fields::Named`
Identifier manipulation	`Longident.Lident name`	`syn::Ident`, `quote::format_ident!`
Error with location	`Location.raise_errorf ~loc`	`syn::Error::new(span, msg)`

// syn + quote for proc macro AST — concept and demonstration

// ===========================================================
// REAL syn + quote USAGE (in a proc-macro crate):
//
// use proc_macro::TokenStream;
// use quote::quote;
// use syn::{parse_macro_input, DeriveInput, Data, Fields};
//
// #[proc_macro_derive(Getters)]
// pub fn derive_getters(input: TokenStream) -> TokenStream {
//     let input = parse_macro_input!(input as DeriveInput);
//     let struct_name = &input.ident;
//
//     // Extract named fields
//     let fields = match &input.data {
//         Data::Struct(s) => match &s.fields {
//             Fields::Named(f) => &f.named,
//             _ => panic!("Only named fields supported"),
//         },
//         _ => panic!("Only structs supported"),
//     };
//
//     // Generate getter for each field
//     let getters = fields.iter().map(|f| {
//         let name = f.ident.as_ref().unwrap();
//         let ty = &f.ty;
//         quote! {
//             pub fn #name(&self) -> &#ty {
//                 &self.#name
//             }
//         }
//     });
//
//     // Wrap in impl block
//     let expanded = quote! {
//         impl #struct_name {
//             #(#getters)*
//         }
//     };
//
//     TokenStream::from(expanded)
// }
//
// Then usage:
// #[derive(Getters)]
// struct User { name: String, age: u32 }
// let u = User { name: "Alice".to_string(), age: 30 };
// println!("{}", u.name()); // generated getter
// ===========================================================

// Simulation: macro_rules! generates getters
macro_rules! derive_getters {
    (
        struct $name:ident {
            $($field:ident : $ty:ty),* $(,)?
        }
    ) => {
        #[derive(Debug)]
        struct $name { $($field: $ty,)* }

        impl $name {
            $(
                pub fn $field(&self) -> &$ty { &self.$field }
            )*
        }
    };
}

// Simulation: generate Display impl from fields
macro_rules! derive_display {
    ($name:ident { $($field:ident),* $(,)? }) => {
        impl std::fmt::Display for $name {
            fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
                write!(f, concat!(stringify!($name), " {{ "
                    $(, stringify!($field), ": {:?}, ")*
                    , "}}"),
                    $(&self.$field,)*
                )
            }
        }
    };
}

derive_getters!(
    struct User {
        name: String,
        age: u32,
        email: String,
    }
);

derive_getters!(
    struct Point {
        x: f64,
        y: f64,
    }
);

fn main() {
    let user = User {
        name: "Alice".to_string(),
        age: 30,
        email: "alice@example.com".to_string(),
    };

    // Generated getters
    println!("name: {}", user.name());
    println!("age: {}", user.age());
    println!("email: {}", user.email());

    let point = Point { x: 3.14, y: 2.72 };
    println!("x: {}, y: {}", point.x(), point.y());

    println!("
syn parses Rust into typed AST:");
    println!("  DeriveInput.ident = struct name");
    println!("  DeriveInput.data = struct/enum/union body");
    println!("  quote! interpolates with #var, #(#vec)*");
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_generated_getters() {
        let u = User { name: "Bob".to_string(), age: 25, email: "b@b.com".to_string() };
        assert_eq!(u.name(), "Bob");
        assert_eq!(*u.age(), 25);
    }
}

(* syn + quote concepts in OCaml via ppxlib *)

(* Showing what ppxlib-equivalent code looks like *)
(* ppxlib.Ast_pattern + ppxlib.Ast_builder *)

(* Simulate what syn::DeriveInput parses *)
type ast_field = {
  name: string;
  typ: string;
}

type ast_struct = {
  struct_name: string;
  fields: ast_field list;
}

(* Simulate what quote! generates *)
let generate_impl ast =
  let field_impls = List.map (fun f ->
    Printf.sprintf "  fn %s(&self) -> &%s { &self.%s }"
      f.name f.typ f.name
  ) ast.fields in
  Printf.sprintf "impl %s {\n%s\n}"
    ast.struct_name
    (String.concat "\n" field_impls)

let () =
  let ast = {
    struct_name = "User";
    fields = [
      {name = "name"; typ = "String"};
      {name = "age"; typ = "u32"};
      {name = "email"; typ = "String"};
    ]
  } in
  let code = generate_impl ast in
  Printf.printf "Generated code:\n%s\n" code