A Novel Multi-Camera Array Microscope: Expanding the View of Optical Microscopy
ORIP participates in two federal Small Business grant programs—the Small Business Innovation Research (SBIR) and the Small Business Technology Transfer (STTR)—that provide seed funding to small businesses to develop technologies that have potential for commercialization. The SBIR program is designed to encourage domestic small businesses to engage in biomedical research and development (R&D), and the STTR program is designed to facilitate cooperative R&D between small businesses and U.S. research institutions. ORIP has aligned its small business programs to develop new technologies that could benefit the research resources that the office supports, the communities that the resources support, and the research community at large. To date, this effort is enabling the support of preclinical technologies that are at several different stages of development, including diagnostic tools, drug therapies, instruments, and devices for animal facilities health monitoring and surveillance. Other ORIP-supported technologies have successfully become commercially available products in the national and international marketplace.
ORIP supports the development and commercialization of technologies through a structured set of competitive SBIR and STTR grant applications that include three phases: Phase I (proof of concept/protype development), Phase II (full research and development of the technology with a clear focus on bringing a commercial product into the market), and if needed, Phase IIB (available only for SBIR, intended to provide follow-on funding to Phase II awardees for those Phase II projects that require extraordinary time and effort in the research and development phase).
The research team behind Ramona Optics, Inc. successfully navigated all those phases, securing ORIP’s funding to develop a new microscope concept into a prototype and, finally, into a new scientific instrument. Along the way, the company overcame a variety of technical challenges, including designing new optical imaging hardware and writing algorithms for rapid data uptake and processing and for sample visualization. Indeed, having successfully developed an initial multi-camera array microscope prototype-1 (MCAM-1) during Phase I of the project, the company refined its design during Phase II into a cost-effective system that is capable of recording videos of freely moving organisms at high resolution across hundreds of square centimeters. During this phase, the team also created a suite of software that takes full advantage of the information-rich recordings from the microscope’s multi-camera array. Finally, in the current Phase IIB, Ramona Optics is in the process of perfecting the software to automatically track and measure key morphological and fluorescence features across all organisms (e.g., zebrafish) contained within a multi-well plate; this software will complete currently tedious and time-consuming tasks and will allow scientists for the first time to simultaneously monitor the movement and fluorescent activity of hundreds of freely moving organisms.
Because of challenges with large lens design, standard optical microscopes typically cannot capture image snapshots of more than 50 megapixels (i.e., 50 million resolved spatial locations, determined by the lateral resolution and magnification of the microscope). This limitation makes it impossible to simultaneously image at cellular resolution over a multicentimeter viewing area or field of view (FOV). For screening and monitoring in vivo model organisms, such as the zebrafish (Danio rerio), this resolution and FOV trade-off presents a critical bottleneck: each organism must be constrained or paralyzed to image at high resolution, freely moving organisms can be viewed only at low resolution, and no setups can monitor multiple swimming zebrafish at cellular resolution in parallel. These challenges are most acutely felt during high-throughput investigations when researchers aim to examine simultaneously many organisms per experiment.
With the support of the ORIP SBIR awards received from 2017 to 2023, described above, Ramona Optics Inc., was able to develop the Multi-Camera Array Microscope (MCAM™) imaging technology (Figure 1) that overcomes these challenges and limitations of current microscopes. The MCAM now allows researchers to easily observe multiple small animal model organisms, including zebrafish, at high resolution and high speed. This is particularly useful in high-throughput experiments that are common within toxicology and pharmacology, for example, where many organisms are studied at once.
Principal investigators Dr. Mark Harfouche and Dr. Roarke Horstmeyer (Figure 2) explained that at its heart, this technology packs up to 100 synchronized cameras in a small footprint. This design allows biology researchers to acquire high-resolution images and videos of their samples in a rapid, high-throughput manner, which provides new data about the biology of the samples of interest. Researchers analyze the captured videos to extract information on the behavior of the organisms and cell dynamics.
Drs. Harfouche and Horstmeyer started Optical Wavefront Laboratories LLC in 2017, shortly after graduating from the California Institute of Technology with doctoral degrees in electrical engineering. They had started to think about how to improve the optics in microscopy to improve the speed and scale of high-resolution image acquisition. Dr. Harfouche recalled that one of the first conversations he had with Dr. Horstmeyer, who has a passion for cameras, was about how to break the 100-megapixel resolution barrier. Drs. Harfouche and Horstmeyer noted that ORIP funding has been invaluable and that the initial Phase I and Phase II awards were important for developing prototypes and advancing their technology to finally transforming the prototypes into a commercial product.
Dr. Horstmeyer stated, “Without either of those grants, we would have nothing. ORIP SBIR funding really was instrumental, and we were so excited to receive it.” The first SBIR Phase I grant allowed the team (Figure 3) to complete a proof-of-concept demonstration.1 Dr. Horstmeyer noted that the ORIP Phase I award was the first funding the company received from NIH and the first grant he and Dr. Harfouche applied for as independent researchers.
The SBIR Phase II grant allowed Drs. Harfouche and Horstmeyer to take a “leap of technical faith” and attempt something no one had done before: directly record gigapixel images and video at micrometer-scale resolution over a FOV of several hundred square centimeters in parallel. This resulted in a new ability to simultaneously capture video of hundreds of freely moving organisms at cellular resolution within entire multi-well plates (Figure 4). Unlike standard optical microscopes, which can image only one well at a time, the MCAM’s synchronous observation of 96 or more wells unlocks new high-throughput experiments. Videos showcasing MCAM technology can be found on Ramona Optics’ YouTube Channel.
As part of their third Phase IIB award, Drs. Harfouche and Horstmeyer established a partnership with Duke University, relocating the company to Durham, North Carolina and renaming the company Ramona Optics, Inc. (named after the street that three of the early employees grew up on (Ramona St., in Palo Alto, California), as well as one of the early company advisers). Duke University investigators were the first users of the MCAM, providing vital feedback about the instrument. Dr. Harfouche is Chief Technology Officer of Ramona Optics, and Dr. Horstmeyer is its Scientific Officer, as well as Director of the Computational Optics Lab and Assistant Professor of Biomedical Engineering at Duke University. Ramona Optics has grown to have 14 full-time employees. Following significant success in U.S.-based institutions, the technology is now being sold and used by researchers worldwide (Canada, Germany, Korea, Switzerland, and Taiwan at this time—with Italy soon to follow). Ramona Optics has demonstrated that the MCAM technology, which relies on data acquired from dozens of micro-cameras, can be combined digitally into composite frames whose total pixel counts significantly exceed the pixel counts of standard microscope systems.1,2
Currently, Ramona Optics is commercializing one primary product, the MCAM Kestrel. Dr. Harfouche said, “We have shown it works and have gained excitement in the research community, and now the hard work begins, where you have to make sure it works repeatedly and in a way that meets the demands of the researchers who want to quickly acquire these measurements.” The team is currently developing a second higher-resolution product for live cell imaging applications.
The ORIP SBIR funding enabled Ramona Optics to serve a broader community of researchers who need to capture better and faster microscope data. The company’s customers include both academic and government laboratories, as well as larger pharmaceutical companies. Its customers span all aspects of science and biological investigation. Dr. Harfouche explained that new technology often starts at the university level, gains traction and credibility, and translates to industrial or commercial areas. The company has begun to see that transition with its video capabilities. Investigators now can start to understand the effects of drugs and chemicals in a high-throughput manner at speeds and resolutions that are otherwise impossible to achieve with a standard microscope. Dr. Harfouche reiterated that the largest use case has been with microscopic imaging of entire 96-well plates, with software automatically performing key measurements and assays, such as providing a detailed organism’s skeletal model, quantifying tail angle and eye size, and tracking high-speed organism activity and fluorescence response. More recent developments allow the technology to provide 3D perspectives at a high resolution.
Numerous recent peer-reviewed publications2-4 have highlighted the MCAM technology, and several partners have been involved in its development. Dr. Harfouche remarked, “The most amazing aspect about this technology is that we have been able to find so many different applications for it beyond the original call from ORIP to study zebrafish.” Ramona Optics has developed a technology capable of studying fruit flies (Drosophila melanogaster) and videoing ants (Formica rufa) working in a colony, scanning high-resolution pathology slides in a high-throughput manner, and examining organoids. These latter complex cell groups more closely mimic the 3D aspect of the human body and are used in high-throughput cancer studies and medicinal chemistry research.
Some zebrafish research laboratories are using the MCAM technology to investigate different toxins that could affect human health or to conduct drug discovery studies. This technology has had the greatest impact in the zebrafish research community, which was the first market Ramona Optics explored. Dr. Harfouche said, “It’s been nice to be a part of a company recognized for its microscope and well known in the zebrafish community.” Ramona Optics also has examined cell cultures derived from human cells and has partnered with preclinical research laboratories that directly culture these cells and use the MCAM to study cell growth and live cell dynamics for drug and compound discovery purposes.
Ramona Optics has published data that demonstrate the utility of the MCAM technology in research, such as its ability to quantify the visuomotor response of zebrafish or to record immune system development within zebrafish embryos at extremely high throughput.5 The goal is to use these data to highlight the MCAM’s usability and applicability within various research settings.
Dr. Horstmeyer explained that although the MCAM technology can pack 100 cameras into a compact array, others in the field are working on different approaches to capture images more rapidly. As the field begins to incorporate artificial intelligence and machine learning into the standard microscopy workflow, Ramona Optics must consider two particular areas to make sure that it stays up to date. The first area will be to develop new approaches for rapidly analyzing the data to free biologists from having to spend enormous amounts of time examining images frame by frame. The second area will be to continue to push the boundaries of optical resolution. Ramona Optics is continuously looking for new and more demanding applications in biology, especially at subcellular resolution.
Dr. Horstmeyer advises other startup companies applying for small business grants for the first time to be patient and to reach out to and discuss their technologies with a NIH Small Business program official, whose role it is to help companies by recommending programs that may be of interest, conveying institute or center (IC) program information and research topics priorities, determining whether a proposal is a good fit for a specific ICs opportunity, and clarifying opportunity requirements.
For additional information on ORIP’s small business programs, visit its Small Business webpages, where such resources as fact sheets, short videos and staff contacts are available. ORIP SBIR/STTR eligibility requirements can be explored through the interactive Small Business Program Selection webpage.
References
1 Thomson EE, Harfouche M, Kim K, et al. Gigapixel imaging with a novel multi-camera array microscope. Elife. 2022;11:e74988. doi:10.7554/eLife.74988.
2 Harfouche M, Kim K, Zhou KC, et al. Imaging across multiple spatial scales with the multi-camera array microscope. Optica. 2023;10(4):471–480.
3 Herrera KJ, Panier T, Guggiana-Nilo D, et al. Larval zebrafish use olfactory detection of sodium and chloride to avoid salt water. Curr Biol. 2021;31(4):782–793.e3. doi:10.1016/j.cub.2020.11.051.
4 Zhou KC, Harfouche M, Cooke CL, et al. Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second. Nat Photonics. 2023;17(5):442–450. doi:10.1038/s41566-023-01171-7.
5 Efromson J, Ferrero G, Bègue A, et al. Automated, high-throughput quantification of EGFP-expressing neutrophils in zebrafish by machine learning and a highly-parallelized microscope. PLoS ONE 2023;18(12):e0295711. doi:10.1371/journal.pone.0295711.