At theInstitute of Applied Sciences and Simulation for Low-Carbon Energies(ISAS) of the CEA, we focus on research and innovation inanalytical sciences. As data analysis plays a pivotal role, we are interested in methodological advancements instatistics,mathematicsandcomputer science, for instance, via the development of state-of-the-art AI models, adapted to our needs.

Neural networkssometimes need to compromise betweenspeed and precision. Training of large architectures might last months and generate huge costs for academia and industry. As a consequence, it is sometimes crucial to cut or optimise the duration of training as much as possible for the task at hand. However, this might come atnegative impacton robustness, interpretability or, even, precision. For instance, neural networks are often considered asblack boxes, needing huge amounts of data (whose detailed properties and impact are often unknown or unexplored), and training can sometimes rely on low precision floating point numbers (potentially, even boolean variables in modern language models) to spare every possible bit of memory. For certain use cases,precision and interpretabilityare, nevertheless, fundamental components which cannot be sacrificed : applications to medical diagnosis or nuclear energy are just two straightforward examples.

• The internship targets the exploration of thestate-of-the-artand the development ofoptimisation techniquesfor neural networks. The objective is to find possible ways to increase precision and interpretability of deep learning algorithms. In particular, we shall focus on the following tasks:
• critical reviewthe state-of-the-art in neural network optimisation to better understand the critical aspects playing a role in neural network precision;
• analysis ofsecond orderneural network optimisers for reliable and interpretable machine learning in physics;
• generalisation of some proposed techniques to enhance precision in neural network predictions.

We shall first test techniques on simplified models (e.g. reproduction of mathematical functions). We shall then consider physicalreal-world scenarios, such as applications to fluid dynamics innuclear energy: the intern will apply different optimisation techniques to the deep learning computation of the initial conditions of aheat diffusionprocess in a solid-liquid interface. The goal will be to achieve an increased precision, granting access to faster computations by traditional solvers. Such techniques will also be useful for other applications beyond the scope of the current project, such as solving geometrical problems or approximating quantum states with neural networks.

The internship will be a collaboration between the DES (Direction of Energies) and the DRF (Direction of Fundamental Research) of CEA. The intern will be hosted by theLaboratory of Artificial Intelligence and Data Science(LIAD) at the DES, in collaboration with the Institute of Theoretical Physics (IPHT).

We look for a passionate student at the end of their studies (e.g. the French M2 level), with a good understanding ofmachine learningandcoding techniques. Good knowledge of anydeep learning framework(PyTorch, JAX, Tensorflow) in Python is mandatory, as well as abiding to good object- oriented coding practices. A basic understanding of physics (statistical mechanics) is appreciated and considered a plus, though not necessary.