AI Researcher (PHD) - Efficient & Interpretable LLMs

Job description

About

We’re building Graybox — a flexible platform for training, analyzing, and debugging neural networks in real time. Our next challenge: designing ultra-efficient large language models (LLMs) that are interpretable, adaptable, and under 256M parameters .

Job Description

We’re looking for a PhD student to help us extend WeightsLab to support LLM development , while also shipping a slim, high-quality LLM . You’ll contribute to both platform capabilities (e.g., transformer-level introspection, attention analysis) and model design.

This opportunity combines practical engineering with mechanistic interpretability : the goal is to understand the internals of small LLMs, intervene during training, and produce a model that’s not only efficient but also transparent and editable .

What You’ll Do

· 
Extend Graybox to support transformer-based models and LLM-specific insights

· 
Train and iteratively refine a sub-256M parameter LLM

· 
Apply mechanistic interpretability tools and concepts to guide architecture choice

· 
Enable interactive model operations: neuron pruning, attention head control, layer freezing

· 
Benchmark performance vs. transparency trade-offs

· 
Deliver a documented, reusable model with accompanying evaluation and tooling

Preferred Experience

You Might Be a Good Fit If You…

· 
PhD student in NLP, Machine Learning, or a related area

· 
Experience with LLMs (e.g. GPT-2, Mistral, Phi, etc.) and PyTorch

· 
Familiarity with model introspection , transformer internals, and training workflows

· 
Working knowledge of mechanistic interpretability methods

· 
Strong experimentation skills and interest in tooling

Bonus (Not Required)

· 
Experience with TransformerLens , Circuits, or interpretability frameworks (e.g. Captum)

· 
Contributions to open-source LLM tooling or lightweight architectures

· 
Interest in hybrid symbolic–neural approaches or efficient deployment

Additional Information

·  Contract Type: Internship (Between 3 and 6 months)
·  Location: Paris
·  Education Level: PhD and more
·  Possible full remote