CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design

Computer-aided design (CAD) is the digital construction of 2D and 3D objects, and is central to a wide range of engineering and manufacturing applications like automobile and aviation. Despite its importance, CAD modeling remains largely a time-intensive, manual task. Recent works have attempted to automate this process with small transformer-based models and handcrafted CAD sequence representations. However, there has been little effort to leverage the potential of large language models (LLMs) for sequential CAD design.

Contributions

To address these limitations, we introduce CADmium, a novel approach that reformulates CAD generation as a purely text-to-text task. Our contributions are as follows:

We developed a novel annotation pipeline using GPT-4.1 that processes multi-view images of 3D objects and their construction sequences to generate annotations combining natural language fluency with geometric precision. We applied this pipeline to 176,017 CAD sequences to release a new dataset with improved, expert-quality annotations.
We fine-tuned a state-of-the-art, instruction-tuned code LLM, specifically Qwen-2.5-Coder, on our dataset to generate JSON-formatted CAD sequences from natural language prompts. This leverages the pre-trained model's capabilities without specialized embedding layers. We evaluated our approach against leading models using established metrics and validated it on the Fusion360 Reconstruction dataset and the expert-annotated CADPrompt benchmark.
We introduced novel metrics (sphericity, mean curvature, Euler characteristic, and watertightness) to better evaluate the structural and topological characteristics of generated objects. Our work highlights the human-like fluency of CADmium's annotations, and our fine-tuned LLM achieves competitive performance, with improvements as model size increases.

The CADmium Pipeline

CADmium reformulates CAD generation as a purely text-to-text task. First, GPT-4.1 generates natural-sounding yet geometrically precise descriptions of 176,017 objects using their construction sequences in minimal JSON, and up to 10 multi-view images rendered with Blender. Then, the Qwen2.5-Coder LLM is fine-tuned with LoRA to translate these descriptions back into CAD sequences.

CADmium vs. Text2CAD annotations

Comparative analysis of CADmium and Text2CAD expert-level annotations. (a) Vocabulary growth as a function of token count demonstrates that Text2CAD has a limited vocabulary compared to CADmium. (b) Human-likeness, clarity, visual faithfulness given image renders, and completeness against the minimal JSON as measured by Gemma-3 12B indicates that CADmium descriptions tend to produce more natural-sounding albeit challenging descriptions. (c-e) Distribution of word counts, unique words, and digit lengths within numerical expressions per annotation shows that CADmium descriptions are more concise and diverse.

Performance on CADPrompt

Performance comparison on the human-annotated CADPrompt dataset. Metrics are calculated on the set of valid meshes generated by each model individually.

Annotation Prompt

The user message contains the JSON description and the 10 multi-view renders of the 3D models, and instruct GPT-4.1 to generate a natural language description.

Generation Prompt