RustQ internal structure

Copy Markdown View Source

This is a maintainer map, not an additional public API contract. Public stability is defined in compatibility.md.

Ownership

AreaResponsibility
RustQ, Template, Splice, Generated, ConfigTemplate/rendering entrypoints, generation plans, and checked output
MetaRusty-Elixir macro frontend and compilation orchestration
Meta.ASTPublic compiled-item access and function/type construction
Meta.Type, Spec, Meta.Typing, Meta.InferenceType representation, spec conversion, expression synthesis, and coercion decisions
Meta.Lower and Meta.Lower.*Translation to Rust AST; evaluation order and expected-type propagation
BindingCallable signatures and lookup indexes, populated from structural sources
Syn, Syn.Index, CargoExternal Rust parsing, package discovery, and metadata indexing
Rust.AST and its buildersRust syntax model and structural composition
Rust, Rust.AST.RenderRendering boundary, with native syntax construction where appropriate
RustlerReusable Rustler bridge-generation APIs
NativePublic zero-handwritten-Rust NIF authoring facade
Native.OptionsNative option vocabulary, defaults, and validation
Native.ABIBoundary codec/resource preparation and internal result-call routing
Native.ManifestCargo manifest policy as data, serialized by the TOML library
Native.BuildCargo invocation, artifact installation, and loader generation
CodegenRustQ's own native AST-decoder generator; not a second consumer compiler
ReachArchitecture and authoring checks
SourceFingerprintShared content identity for source-backed caches

Native.Ref, Native.Descriptor, and Native.EnumDescriptor are existing public metadata APIs. Their placement predates the NIF authoring implementation; keep their names stable rather than moving them for visual symmetry. Native.Nif is the private loader for RustQ's own native support, not the code that builds a consumer's generated crate.

Boundaries to preserve

  • The macro frontend supplies resolved aliases and callable metadata to lowering. Lowering must not discover packages or run Cargo while processing expressions.
  • Expected types must cross branch and closure boundaries explicitly. The enclosing function's return type is not the mapper's return type.
  • ABI adaptation belongs at the exported boundary. Internal native calls retain their native signatures; do not globally replace result constructors.
  • Source and package caches must observe content changes. Derived callable caches must not bypass their source's freshness checks.
  • AST construction is separate from serialization. Use RustQ for Rust and a TOML encoder for manifests, not interpolated function bodies or configuration syntax.
  • Resource ownership, synchronization, scheduling, and numeric overflow policy remain explicit native decisions.

Bootstrap and dogfooding

The checked-in native/rustq_nif/src/generated_ast.rs is a bootstrap input. RustQ first compiles that input, then uses its public AST and Rusty-Elixir APIs under lib/rustq/codegen/ to regenerate it. Consumers do not need to regenerate RustQ's native support to compile the package.

After changing self-hosted generation, verify:

  1. Clean Elixir compilation against checked-in native support.
  2. mix rustq.gen.
  3. Rebuild the native crate and recompile Elixir in a fresh Mix invocation.
  4. mix rustq.gen --check and compare the generated file with step 2.
  5. mix ci, including packaged consumers.

A clean Elixir build with existing Cargo caches is useful, but is not equivalent to an isolated no-cache bootstrap. Keep that distinction in validation reports.

The native syn parsing, traversal, and token conversion layer is intentional. Move repetitive boundary logic to Rusty-Elixir where that simplifies its source; do not port clear domain Rust merely to increase generated-code counts.

Facts about Rust std

Which std methods mutate their receiver, return an iterator or an option, or belong to Option, Result, or std collections lives in the checked-in lib/rustq/meta/std_methods.ex. The rustq.std Mix task derives it from the source rustup installs with rustup component add rust-src, reading it through RustQ.Syn.parse(source, unstable: true). The task and its derivation live in dev/, outside the package.

The file records the Rust release it came from, and CI pins that toolchain in .github/workflows/ci.yml; mix rustq.std --check fails under any other one. To move to a newer Rust:

  1. rustup component add rust-src for the new stable toolchain.
  2. Run the rustq.std Mix task and review the diff of std_methods.ex.
  3. Set rust-toolchain in .github/workflows/ci.yml to the same release.
  4. mix ci.

When std adopts unstable syntax syn cannot parse, the rustq.std task names the file; teach native/rustq_nif/src/unstable_syntax.rs to rewrite it, with a test.

Further extraction criteria

Meta.Lower, Meta.Type, and Syn are substantial modules. Size alone is not a reason to divide them. Extract a cohesive operation only when its inputs and outputs can be named and tested independently. Prefer branch/closure lowering, type conversion, or source-query responsibilities over generic Utils modules.

Keep focused tests parallel to their owning modules under test/rustq/. Cross-package runtime behavior belongs in integration/ fixtures driven through test/support/; source grep tests are not architecture enforcement.