Director, Multi-Modal AI: Agentic World Building
Company: Unity Software
Location: Mountain View
Posted on: March 18, 2026
|
|
|
Job Description:
Generative AI has spent the last few years mastering pixels and
prose. Unity is here to make it move. We are seeking a visionary
Director, Multi-Modal AI to lead the charge in transforming "dream
pixels"—static AI hallucinations—into Dream Games. This is a
zero-to-one leadership opportunity to build Unity’s first-party
intelligence layer. You wont just be calling APIs; you will be
building the brain that understands 3D space, masters game
mechanics, and empowers every creator to manifest their vision into
a living, breathing Unity environment. You will lead the applied
research organization responsible for Agentic 3D World Building.
Your goal is to move beyond simple asset generation and solve the
industrys "holy grail": Spatial Reasoning. By leveraging
cutting-edge World Models and Multi-Modal LLMs, you will enable a
future where natural language doesnt just describe a scene—it
authors it, simulates it, and plays it. What youll be doing •
Architect the "World Brain": Lead the research and implementation
of Agentic workflows. You will use World Models to ensure AI
understands the "why" and "how" of a 3D scene—from physics-based
constraints to complex scene hierarchies. • Build the Multi-Modal
Arm: Own the fine-tuning and distillation of state-of-the-art
MLLMs. You will specialize these models to generate high-fidelity
game mechanics, shaders, and logic that respond to multi-modal
prompts (text, image, and spatial data). • Master the
Teacher-Student Pipeline: Since we arent just "pre-training" in a
vacuum, you will drive the strategy for distilling massive
foundation models into high-performance, edge-ready "students"
optimized for the Unity Editor. • Bridge Science and Play: Solve
the grounding problem. You will ensure that AI-generated content
isnt just a visual trick, but is engine-ready, mathematically sound
(transforms, quaternions, nav-meshes), and physically plausible. •
Scale the Data Flywheel: Turn Unity’s massive ecosystem into an
intelligence moat. Build secure, rights-managed pipelines and
"teacher-led" synthetic data generation to train models on the
nuances of game development that the rest of the world hasnt even
seen yet. What were looking for • The "Spatial DNA": You dont just
know AI; you know 3D. You understand that a game is a living system
of transforms, raycasts, and rigidbodies. • Proven AI Leadership:
10 years in ML/AI, with 5 years leading high-performance applied
research teams. You have a track record of shipping proprietary
models that solve real-world problems. • Fine-Tuning & Distillation
Experts: You are a master of instruction tuning, PEFT, and model
compression. You know how to take a "generalist" model and turn it
into a "Unity Specialist." • Agentic Visionaries: You have
experience with agentic frameworks and autonomous systems that can
reason through multi-step tasks in complex environments. •
Optimization Obsessed: You understand the constraints of a
real-time engine. You are comfortable with quantization (INT8/FP8)
and speculative decoding to ensure the AI feels like a
collaborator, not a bottleneck. Additional information •
International relocation support is not available for this position
Benefits At Unity, we want our team members to thrive. We offer a
wide range of benefits designed to support well-being and work-life
balance. Please note: Benefits eligibility, specific offerings, and
coverage vary based on the country and employment status. While
specific benefits vary, here are some of the ways we strive to take
care of our eligible team members globally: Comprehensive health,
life, and disability insurance | Commute subsidy | Employee stock
ownership | Competitive retirement/pension plans | Generous
vacation and personal days | Support for new parents through leave
and family-care programs | Office food snacks | Mental Health and
Wellbeing programs and support | Employee Resource Groups | Global
Employee Assistance Program | Training and development programs |
Volunteering and donation matching program
Keywords: Unity Software, Newark , Director, Multi-Modal AI: Agentic World Building, IT / Software / Systems , Mountain View, California
