By combining videos from dozens of cameras, a unique 3D view of macroscopic experiments with microscopic details was obtained. The first photo a pair of brave graduate students took using their improvised microscope turned out better than expected. They managed to locate Waldo, although part of it had a hole and part of it turned on its side.
The next day, the team solved the software problems and successfully used the famous children's puzzle book as a proof-of-concept device. By integrating 24 smartphone cameras into a single platform and stitching together their photos, the team developed a single camera capable of capturing gigapixel images in an area roughly the size of a paper.
After six years, multiple design revisions, and a startup company, the researchers made an unexpected discovery. They were able to see the height of the object, perfecting the method of combining multiple cameras with sub-pixel resolution at once.
Roarke Horstmeyer, an assistant professor of biomedical engineering at Duke University, likens it to human vision. “When multiple perspectives are combined, objects are seen from various angles, which is what your two eyes do. This gives you a sense of height. It completely amazed our colleagues working on zebrafish when they used it for the first time. They immediately noticed new behaviors related to slope and depth that they had never noticed before.
In a paper published in the journal Nature Photonics, Horstmeyer and colleagues demonstrate the capabilities of a new high-speed, 3D, gigapixel microscope they call the Multi-Camera Array Microscopy (MCAM) (MCAM). The device provides new opportunities for researchers around the world by capturing 3D movies of the behavior of large numbers of free-swimming zebrafish or the grooming activities of fruit flies in near-cellular detail over a wide field of view. The latest iteration of the MCAM uses 54 lenses that are faster and more precise than the prototype that discovered Waldo.
Innovative software, Dr. Based on recent work completed in close collaboration with Eva Naumann's team at Duke, it enables the microscope to take 3D measurements, provide more detail at lower scales, and produce smoother films.
However, MCAM's highly parallelized design presents its own data processing challenges, as just a few minutes of recording can generate more than a terabyte of data. "We've developed new algorithms that can efficiently process these extraordinarily large video datasets," said Kevin C. Zhou, a postdoctoral fellow in Horstmeyer's group and lead author of the study. “Our methods integrate physics with machine learning to combine video streams from all cameras and retrieve 3D behavioral information across location and time.
Everyone can try our code as it is open source on Github.
Matthew McCarroll, a researcher at the University of California at San Francisco, studies how zebrafish behave when exposed to a neuroactive drug. By examining the behavioral changes caused by various types of drugs, researchers can find new treatments going forward or gain a better understanding of existing treatments.
Using this camera, McCarroll and his team were able to capture interesting movements they had never seen before. Thanks to MCAM's 3D capabilities and all-encompassing vision, they were able to capture changes in fish's tilt, whether they were heading above or below their tank, and how they were tracking their prey.
“We've been making our own rigs with single lenses and cameras for a long time, and these have worked well for our purposes, but this is on a whole other level,” said McCarroll, a freelance researcher pursuing a professional researcher series degree in pharmaceutical chemistry. We're just biologists experimenting with optics. It's incredible what a legitimate physicist can suggest to improve our experiments.
Zebrafish are used in research by Duke University cell biology professor Michel Bagnat. Rather than focusing on drug-induced behavioral changes, the researchers look at the biological processes animals go through as they grow from eggs to full-blown adults.
In previous research, developing fish had to be mounted and anesthetized to remain still while laser measurements were taken. However, prolonged unconsciousness could potentially alter the developmental course, which could skew the results of the experiment. Thanks to the new MCAM, the researchers have shown that they can achieve all these measurements without knocking or clamping fish.
According to Jennifer Bagwell, a research scientist and lab director in the Bagnat lab, the 3D and fluorescent imaging capabilities of this microscope could change the way many developmental biologists conduct their experiments. Especially if it turns out that drugging fish changes their development, a topic we are currently investigating.
Horstmeyer hopes this approach will enable more extensive automated parallel research as well as tracking entire populations of small animals, such as zebrafish, in trials. For example, the microscope can observe a 384-well plate filled with various organoids to assess potential drug interactions. It can record the cellular responses of each experiment and automatically flag results of interest.
According to Horstmeyer, who states that the modern laboratory is becoming more automated every day, large well plates are now filled and preserved without human touch. The need for new technology that can help automate the tracking and capture of findings stems from the sheer volume of data.
Horstmeyer and his colleague, Mark Harfouche, who was responsible for obtaining the first photo of Waldo, founded a startup firm called Ramona Optics to market the technology. One of its original licensees, MIRA Imaging uses this technology to “fingerprint” fine art, collectibles and luxury goods to protect them from counterfeiting and fraud.
source: scitechdaily
📩 21/03/2023 10:04