Study and exploitation of activation characteristics in deep neural networks H/F (Systèmes d'information)

Job description

Domaine : Systèmes d'information

Contrat : Stage

Description du poste :

Artificial neural networks are currently used in many areas requiring the processing of sensor data: Visions, sound, etc.
Depending on different constraints, information processing can be carried out on the cloud (SIRI, AWS, TPU, etc.) or on an embedded basis (Jetson platform from Nvidia, Movidius, PNeuro/DNeuro from CEA LIST). In the second case, many material constraints must be taken into account when sizing the algorithm. To improve porting to embedded platforms, numerous research studies have led to various techniques for reducing the memory and computational footprint of an artificial neural network: Reduction of the number of parameters, numerical quantification, etc.
These different methods used to compress artificial neural networks lead to a loss of application accuracy. In order to optimize the footprint of an artificial neural network without degrading application performance, CEA LIST offers a 6-month internship to study the different characteristics of neural activations and their statistical distributions. This internship work should lead to an advanced method for optimizing these characteristics within the artificial neural network in order to drastically reduce the computational complexity of neural networks.
To achieve this, the candidate will use Python data analysis tools (such as Numpy, Scipy, Pandas, etc.) coupled with traditional Deep Learning tools (such as N2D2, Tensorflow, PyTorch, etc.) to analyze and optimize deep neural networks. This internship also offers the opportunity to contribute to N2D2, the LIST Deep Learning open source software (C++ skills required).
As the internship takes place in a scientific research environment, the subject of the internship may evolve according to the candidate's different results. Analysis work will make it possible to adjust the direction of the work to be carried out:
Analysis of the distribution of activation values in the different layers of a DNN according to the data
Analysis of the sparsity of representations (characteristics of intermediate layers) in convolutional neural networks
Analysis of the temporal redundancies of a video stream processed by neural network.
Optimization of computational operators based on data distribution and sparsity
The candidate must have a strong analytical mind and be able to make proposals in order to influence the direction of his/her research work.
This internship is a good opportunity to confront the world of research, and to develop skills by working at the frontier between the software and hardware domains of deep neural networks.
Based on the profile and interest of the student, the project can have an emphasis on machine learning, software or hardware. The candidate will have the opportunity to become a contributing member of the team that develops the French open source Deep Learning Framework N2D2.
 





Our embedded AI CEA/LIST research team, based in Palaiseau, is looking for a talented student for an end of studies Master project (projet fin d'études) at the CEA/LIST lab.
- Basic knowledge of Deep Learning/Machine learning/Statistics, Python, C++ (optional but required for contribution to N2D2)

Ville : Palaiseau