Pioneering neuroscience data visualization expert Cyrille Favreau has made significant contributions to the production of high-quality visualizations of brains in silico at the École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. His expertise lies in the field of real-time renderings of large neuroscience datasets using ray-tracing techniques (a high-fidelity technique to generate digital images in 3D graphics) and high-performance programming languages, such as C++, CUDA, and ISPC. Favreau created the large-scale scientific visualization platform Blue Brain Brayns, which can perform ray-traced renderings of neurons and highlight synapses in neural circuits.
Currently, Favreau is the visualization expert for the Blue Brain Project (BBP), a special research initiative at the EPFL (Markram, 2006). Directed by Professor Henry Markram, the BBP aims to produce a detailed reconstruction of the brain in silico (experimentation performed via computer simulations), in the form of an entire neuron-to-neuron connectome model that will allow for large-scale simulations. Recent breakthroughs include the generation and simulation of point-neuron models of the whole brain, which acts as a type of artificial neuron model that does not consider the spatial structure of the neuron, and a groundbreaking computational model of the thalamic microcircuit in the mouse brain, offering new insights into the role this region plays in brain function and dysfunction (Rodarie et al., 2022; Roussel et al., 2022). During his time working for Blue Brain, Favreau developed Blue Brain BioExplorer, an open-source application that allows for visualization and scientific exploration, allowing the reconstruction, description, and exploration of the structures and functions of brain regions down to molecular systems. Favreau's work aims to enhance the understanding of brain functions by generating highly detailed and comprehensible visualizations through software engineering and large data sets.
LO: What caused you to start creating visualization software for biological systems?
CF: Back in 2012, I was already working as a software engineer in the banking industry. I was looking for solutions to speed up our systems and came across Graphics Processing Unit (GPU) programming, a new technology developed by the Nvidia Corporation that would allow programmers to use graphics cards for any type of computation, be it in finance or life sciences. I wanted to find an application of these computational methods that would allow me to learn and understand that new technology firsthand. When I was a student, I was already interested in computer graphics, but there were not many job opportunities at the time. Still, during all those years, I kept that desire of returning to build computer-generated images in the back of my mind. So, I started to spend my evenings reactivating some of my very old code and experimenting with it on new hardware. The technique I am using is called ray tracing, and it is extremely intensive in terms of computation because the idea is to simulate light transport. As one can imagine, the world we see through our eyes is visible because it reflects light coming from many kinds of sources: the sun, light bulbs, etc. When I was a student, it would take the whole night to compute a small, simple image. But with the new hardware, the computing time was reduced to a millisecond. That was an amazing feeling. Suddenly, everything was possible. When looking for suitable 3D models to apply the new technology to, science was the answer: a wide range of openly accessible datasets describing fascinating structures. I wrote my first graphics engine, Sol-R for Speed of Light Ray-Tracer, and applied it to molecular visualization. This is how I started creating visualizations for biological systems.
LO: Could you explain your creation of the Blue Brain BioExplorer, in simplified terms?
CF: The creation of the Blue Brain BioExplorer resulted from a two-step process. At first, I created Blue Brain Brayns, the underlying rendering platform that can perform interactive high-quality and high-fidelity rendering of large datasets on ultra-high-resolution displays. This was the first milestone, a generic platform to render and explore any type of scientific dataset.
In early 2020, I moved to the brain simulation team to leverage the power of the rendering platform by adding scientific use cases to the system. This gave birth to Blue Brain BioExplorer, which was initially used to produce an educational movie on the hypothesized relationship between blood glucose levels and the severity of COVID-19 (Logette et al., 2021), with a strong focus on molecular systems and viruses. The BioExplorer uses advanced rendering techniques such as shadows and more generally, so-called global illumination to produce photo-realistic images, just like a camera would do in the real world. The BioExplorer has since been extended to explore other biological systems, such as brain microcircuits, neurons, astrocytes, blood vessels, white matter, brain atlases, enzyme reactions, brain connectivity, and more.
LO: What steps do you take when designing meticulous brain simulations with datasets?
CF: The first step is the reconstruction of realistic biological structures, such as neurons, astrocytes, and blood vessels, from raw skeletons. What we get from the lab is a list of points, linked together to form a skeleton made of sections and segments. A section is the part of the dendrite or axon between each bifurcation. A segment is a sub-part of a section.
From that information, it is possible to recreate a realistic shape by applying the signed distance field (SDF) technique. This technique consists of using simple geometric shapes such as spheres, cones, and cylinders and combining them with different modifiers to produce a realistic, biological shape. This technique is also well-suited in terms of memory footprint optimization. When hundreds of thousands of neurons have to be rendered at once, the computer memory occupied by each neuron must be as small as possible, and using those basic shapes is a very efficient way to do so.
Once the cells are reconstructed in 3D, we generate what we call a “simulation report” to visualize brain activity. A simulation report can be seen as a film, where each frame contains a value indicating the voltage of each segment. A color map is used to label each neuron segment with a color that corresponds to the value of the voltage: typically blue for low, red for medium, and yellow for high voltages. Considering that a neural circuit can be made of billions of segments, simulation reports can represent many gigabytes of data, depending on the number of frames used for the simulation. For the validation process, the code is double-checked by several developers and single cells are randomly selected and tested with other tools specialized in single-cell simulation to make sure that the results are identical.
LO: How do you manage the process of trial and error?
CF: Apart from intuitive observations, films are the crash test because they show everything. Are synapses really located at a place where neurons are close to each other? Do neurons intersect with each other, or do they intersect with glial cells or the vasculature? Is the location or scale of the model correct when compared to the official brain atlas? Validating models is a back-and-forth process, and errors are reported to scientific teams, who can then confirm the problems reported by the visualization tools.
LO: As a last question, would you consider visualization an art form?
CF: This is a tricky question because this depends on how we define art. From my point of view, art has to touch the heart of people for emotions to arise. From what I have observed in many of the presentations I gave to people with different backgrounds, colors have played an important role in the appreciation of the image. So maybe I could say that the choice of colors, the way they are disseminated within the image, the camera angle, and the lighting of the model all make it a work of art. In the end, making scientific images is very close to photography, except that the model and the camera are virtual. They are not randomly organized, they follow structural patterns, some form of logic. However, I should also state that the structures themselves are incredibly beautiful. What I always say is, despite their complexity, biological structures are still beautiful to watch.
About the Authors
Lara Ota (’23), Buse Toksöz (’26), and Kei Hayashi (’26), are currently students at Institut Le Rosey in Switzerland and are part of Le Rosey Neuroscience Society.
References
Additional links
The Blue Brain Project on EPFL’s website: https://www.epfl.ch/research/domains/bluebrain/
Blue Brain Mouse whole-neocortex connectome model: https://portal.bluebrain.epfl.ch/resources/models/mouse-whole-neocortex/
Blue Brain BioExplorer: https://github.com/BlueBrain/BioExplorer
Blue Brain Brayns: https://github.com/BlueBrain/Brayns
Sol-R: https://github.com/favreau/Sol-R
IMV (Interactive Molecular Visualization): https://cyrillefavreau.wixsite.com/molecularvisual
Massive Open Online Courses by EPFL on simulation neuroscience: https://www.edx.org/course/simulation-neuroscience#!
Blog by Cyrille Favreau: http://cudaopencl.blogspot.com
Acknowledgments:
I would like to thank Mr. Favreau for giving us his time to conduct this interview. I also thank Buse and Kei for their proactiveness in the production of this article. Finally, I am incredibly grateful to Mme Durlot for her generosity in introducing us to incredible Swiss researchers.
Photo credit: Mouse neocortex rendered at 3% density. Neurons are colored by morphological types (Blue Brain Project)
Currently, Favreau is the visualization expert for the Blue Brain Project (BBP), a special research initiative at the EPFL (Markram, 2006). Directed by Professor Henry Markram, the BBP aims to produce a detailed reconstruction of the brain in silico (experimentation performed via computer simulations), in the form of an entire neuron-to-neuron connectome model that will allow for large-scale simulations. Recent breakthroughs include the generation and simulation of point-neuron models of the whole brain, which acts as a type of artificial neuron model that does not consider the spatial structure of the neuron, and a groundbreaking computational model of the thalamic microcircuit in the mouse brain, offering new insights into the role this region plays in brain function and dysfunction (Rodarie et al., 2022; Roussel et al., 2022). During his time working for Blue Brain, Favreau developed Blue Brain BioExplorer, an open-source application that allows for visualization and scientific exploration, allowing the reconstruction, description, and exploration of the structures and functions of brain regions down to molecular systems. Favreau's work aims to enhance the understanding of brain functions by generating highly detailed and comprehensible visualizations through software engineering and large data sets.
LO: What caused you to start creating visualization software for biological systems?
CF: Back in 2012, I was already working as a software engineer in the banking industry. I was looking for solutions to speed up our systems and came across Graphics Processing Unit (GPU) programming, a new technology developed by the Nvidia Corporation that would allow programmers to use graphics cards for any type of computation, be it in finance or life sciences. I wanted to find an application of these computational methods that would allow me to learn and understand that new technology firsthand. When I was a student, I was already interested in computer graphics, but there were not many job opportunities at the time. Still, during all those years, I kept that desire of returning to build computer-generated images in the back of my mind. So, I started to spend my evenings reactivating some of my very old code and experimenting with it on new hardware. The technique I am using is called ray tracing, and it is extremely intensive in terms of computation because the idea is to simulate light transport. As one can imagine, the world we see through our eyes is visible because it reflects light coming from many kinds of sources: the sun, light bulbs, etc. When I was a student, it would take the whole night to compute a small, simple image. But with the new hardware, the computing time was reduced to a millisecond. That was an amazing feeling. Suddenly, everything was possible. When looking for suitable 3D models to apply the new technology to, science was the answer: a wide range of openly accessible datasets describing fascinating structures. I wrote my first graphics engine, Sol-R for Speed of Light Ray-Tracer, and applied it to molecular visualization. This is how I started creating visualizations for biological systems.
LO: Could you explain your creation of the Blue Brain BioExplorer, in simplified terms?
CF: The creation of the Blue Brain BioExplorer resulted from a two-step process. At first, I created Blue Brain Brayns, the underlying rendering platform that can perform interactive high-quality and high-fidelity rendering of large datasets on ultra-high-resolution displays. This was the first milestone, a generic platform to render and explore any type of scientific dataset.
In early 2020, I moved to the brain simulation team to leverage the power of the rendering platform by adding scientific use cases to the system. This gave birth to Blue Brain BioExplorer, which was initially used to produce an educational movie on the hypothesized relationship between blood glucose levels and the severity of COVID-19 (Logette et al., 2021), with a strong focus on molecular systems and viruses. The BioExplorer uses advanced rendering techniques such as shadows and more generally, so-called global illumination to produce photo-realistic images, just like a camera would do in the real world. The BioExplorer has since been extended to explore other biological systems, such as brain microcircuits, neurons, astrocytes, blood vessels, white matter, brain atlases, enzyme reactions, brain connectivity, and more.
LO: What steps do you take when designing meticulous brain simulations with datasets?
CF: The first step is the reconstruction of realistic biological structures, such as neurons, astrocytes, and blood vessels, from raw skeletons. What we get from the lab is a list of points, linked together to form a skeleton made of sections and segments. A section is the part of the dendrite or axon between each bifurcation. A segment is a sub-part of a section.
From that information, it is possible to recreate a realistic shape by applying the signed distance field (SDF) technique. This technique consists of using simple geometric shapes such as spheres, cones, and cylinders and combining them with different modifiers to produce a realistic, biological shape. This technique is also well-suited in terms of memory footprint optimization. When hundreds of thousands of neurons have to be rendered at once, the computer memory occupied by each neuron must be as small as possible, and using those basic shapes is a very efficient way to do so.
Once the cells are reconstructed in 3D, we generate what we call a “simulation report” to visualize brain activity. A simulation report can be seen as a film, where each frame contains a value indicating the voltage of each segment. A color map is used to label each neuron segment with a color that corresponds to the value of the voltage: typically blue for low, red for medium, and yellow for high voltages. Considering that a neural circuit can be made of billions of segments, simulation reports can represent many gigabytes of data, depending on the number of frames used for the simulation. For the validation process, the code is double-checked by several developers and single cells are randomly selected and tested with other tools specialized in single-cell simulation to make sure that the results are identical.
LO: How do you manage the process of trial and error?
CF: Apart from intuitive observations, films are the crash test because they show everything. Are synapses really located at a place where neurons are close to each other? Do neurons intersect with each other, or do they intersect with glial cells or the vasculature? Is the location or scale of the model correct when compared to the official brain atlas? Validating models is a back-and-forth process, and errors are reported to scientific teams, who can then confirm the problems reported by the visualization tools.
LO: As a last question, would you consider visualization an art form?
CF: This is a tricky question because this depends on how we define art. From my point of view, art has to touch the heart of people for emotions to arise. From what I have observed in many of the presentations I gave to people with different backgrounds, colors have played an important role in the appreciation of the image. So maybe I could say that the choice of colors, the way they are disseminated within the image, the camera angle, and the lighting of the model all make it a work of art. In the end, making scientific images is very close to photography, except that the model and the camera are virtual. They are not randomly organized, they follow structural patterns, some form of logic. However, I should also state that the structures themselves are incredibly beautiful. What I always say is, despite their complexity, biological structures are still beautiful to watch.
About the Authors
Lara Ota (’23), Buse Toksöz (’26), and Kei Hayashi (’26), are currently students at Institut Le Rosey in Switzerland and are part of Le Rosey Neuroscience Society.
References
- Logette, E., Lorin, C., Favreau, C., Oshurko, E., Coggan, J. S., Casalegno, F., Sy, M. F., Monney, C., Bertschy, M., Delattre, E., Fonta, P.-A., Krepl, J., Schmidt, S., Keller, D., Kerrien, S., Scantamburlo, E., Kaufmann, A.-K., & Markram, H. (2021). A machine-generated view of the role of blood glucose levels in the severity of COVID-19. Frontiers in Public Health, 9, 1068.
- Markram, H. (2006). The Blue Brain Project. Nature Reviews Neuroscience, 7, 153–160.
- Roussel Y, Verasztó C, Rodarie D, Damart T, Reimann M, Ramaswamy S, et al. (2023) Mapping of morpho-electric features to molecular identity of cortical inhibitory neurons. PLoS Comput Biol 19(1): e1010058.
- Rodarie D, Verasztó C, Roussel Y, Reimann M, Keller D, Ramaswamy S, et al. (2022) A method to estimate the cellular composition of the mouse brain from heterogeneous datasets. PLoS Comput Biol 18(12): e1010739.
Additional links
The Blue Brain Project on EPFL’s website: https://www.epfl.ch/research/domains/bluebrain/
Blue Brain Mouse whole-neocortex connectome model: https://portal.bluebrain.epfl.ch/resources/models/mouse-whole-neocortex/
Blue Brain BioExplorer: https://github.com/BlueBrain/BioExplorer
Blue Brain Brayns: https://github.com/BlueBrain/Brayns
Sol-R: https://github.com/favreau/Sol-R
IMV (Interactive Molecular Visualization): https://cyrillefavreau.wixsite.com/molecularvisual
Massive Open Online Courses by EPFL on simulation neuroscience: https://www.edx.org/course/simulation-neuroscience#!
Blog by Cyrille Favreau: http://cudaopencl.blogspot.com
Acknowledgments:
I would like to thank Mr. Favreau for giving us his time to conduct this interview. I also thank Buse and Kei for their proactiveness in the production of this article. Finally, I am incredibly grateful to Mme Durlot for her generosity in introducing us to incredible Swiss researchers.
Photo credit: Mouse neocortex rendered at 3% density. Neurons are colored by morphological types (Blue Brain Project)
