Quantitative Imaging Publications
Correct cell/cell interactions and motion dynamics are fundamental in tissue homeostasis, and defects in these cellular processes cause diseases. Therefore, there is strong interest in identifying factors, including drug candidates that affect cell/cell interactions and motion dynamics. However, existing quantitative tools for systematically interrogating complex motion phenotypes in timelapse datasets are limited. We present Motion Sensing Superpixels (MOSES), a computational framework that measures and characterises biological motion with a unique superpixel 'mesh' formulation. Using published datasets, MOSES demonstrates single-cell tracking capability and more advanced population quantification than Particle Image Velocimetry approaches. From > 190 co-culture videos, MOSES motion-mapped the interactions between human esophageal squamous epithelial and columnar cells mimicking the esophageal squamous-columnar junction, a site where Barrett's esophagus and esophageal adenocarcinoma often arise clinically. MOSES is a powerful tool that will facilitate unbiased, systematic analysis of cellular dynamics from high-content time-lapse imaging screens with little prior knowledge and few assumptions.
Digital pathology and image analysis potentially provide greater accuracy, reproducibility and standardisation of pathology-based trial entry criteria and endpoints, alongside extracting new insights from both existing and novel features. Image analysis has great potential to identify, extract and quantify features in greater detail in comparison to pathologist assessment, which may produce improved prediction models or perform tasks beyond manual capability. In this article, we provide an overview of the utility of such technologies in clinical trials and provide a discussion of the potential applications, current challenges, limitations and remaining unanswered questions that require addressing prior to routine adoption in such studies. We reiterate the value of central review of pathology in clinical trials, and discuss inherent logistical, cost and performance advantages of using a digital approach. The current and emerging regulatory landscape is outlined. The role of digital platforms and remote learning to improve the training and performance of clinical trial pathologists is discussed. The impact of image analysis on quantitative tissue morphometrics in key areas such as standardisation of immunohistochemical stain interpretation, assessment of tumour cellularity prior to molecular analytical applications and the assessment of novel histological features is described. The standardisation of digital image production, establishment of criteria for digital pathology use in pre-clinical and clinical studies, establishment of performance criteria for image analysis algorithms and liaison with regulatory bodies to facilitate incorporation of image analysis applications into clinical practice are key issues to be addressed to improve digital pathology incorporation into clinical trials.
Clinical success of immunotherapy is driving the need for new prognostic and predictive assays to inform patient selection and stratification. This requirement can be met by a combination of computational pathology and artificial intelligence. Here, we critically assess computational approaches supporting the development of a standardized methodology in the assessment of immune-oncology biomarkers, such as PD-L1 and immune cell infiltrates. We examine immunoprofiling through spatial analysis of tumor-immune cell interactions and multiplexing technologies as a predictor of patient response to cancer treatment. Further, we discuss how integrated bioinformatics can enable the amalgamation of complex morphological phenotypes with the multiomics datasets that drive precision medicine. We provide an outline to machine learning (ML) and artificial intelligence tools and illustrate fields of application in immune-oncology, such as pattern-recognition in large and complex datasets and deep learning approaches for survival analysis. Synergies of surgical pathology and computational analyses are expected to improve patient stratification in immuno-oncology. We propose that future clinical demands will be best met by (1) dedicated research at the interface of pathology and bioinformatics, supported by professional societies, and (2) the integration of data sciences and digital image analysis in the professional education of pathologists.
Advances in optical microscopy, biosensors and cell culturing technologies have transformed live cell imaging. Thanks to these advances live cell imaging plays an increasingly important role in basic biology research as well as at all stages of drug development. Image analysis methods are needed to extract quantitative information from these vast and complex data sets. The aim of this review is to provide an overview of available image analysis methods for live cell imaging, in particular required preprocessing image segmentation, cell tracking and data visualisation methods. The potential opportunities recent advances in machine learning, especially deep learning, and computer vision provide are being discussed. This review includes overview of the different available software packages and toolkits.
© Springer International Publishing Switzerland 2014. We present a robust and high-throughput computational method for cell segmentation using multiplexed immunohistopathology images. The major challenges in obtaining an accurate cell segmentation from tissue samples are due to (i) complex cell and tissue morphology, (ii) different sources of variability including non-homogeneous staining and microscope specific noise, and (iii) tissue quality. Here we present a fast method that uses cell shape and scale information via unsupervised machine learning to enhance and improve general purpose segmentation methods. The proposed method is well suited for tissue cytology because it captures the the morphological and shape heterogeneity in different cell populations.We discuss our segmentation framework for analysing approximately one hundred images of lung and colon cancer and we restrict our analysis to epithelial cells.
As histology patterns vary depending on different tissue types, it is typically necessary to adapt and optimize segmentation algorithms to these tissue type-specific applications. Here we present an unsupervised method that utilizes cell shape cues to achieve this task-specific optimization by introducing a shape ranking function. The proposed algorithm is part of our Layers™ toolkit for image and data analysis for multiplexed immunohistopathology images. To the best of our knowledge, this is the first time that this type of methodology is proposed for segmentation and ranking in cell tissue samples. Our new cell ranking scheme takes into account both shape and scale information and provides information about the quality of the segmentation. First, we introduce cell-shape descriptor that can effectively discriminate the cell-type's morphology. Secondly, we formulate a hierarchical-segmentation as a dynamic optimization problem, where cells are subdivided if they improve a segmentation quality criteria. Third, we propose a numerically efficient algorithm to solve this dynamic optimization problem. Our approach is generic, since we don't assume any particular cell morphology and can be applied to different segmentation problems. We show results in segmenting and ranking thousands of cells from multiplexing images and we compare our method with well established segmentation techniques, obtaining very encouraging results. © 2013 IEEE.
Limitations on the number of unique protein and DNA molecules that can be characterized microscopically in a single tissue specimen impede advances in understanding the biological basis of health and disease. Here we present a multiplexed fluorescence microscopy method (MxIF) for quantitative, single-cell, and subcellular characterization of multiple analytes in formalin-fixed paraffin-embedded tissue. Chemical inactivation of fluorescent dyes after each image acquisition round allows reuse of common dyes in iterative staining and imaging cycles. The mild inactivation chemistry is compatible with total and phosphoprotein detection, as well as DNA FISH. Accurate computational registration of sequential images is achieved by aligning nuclear counterstain-derived fiducial points. Individual cells, plasma membrane, cytoplasm, nucleus, tumor, and stromal regions are segmented to achieve cellular and subcellular quantification of multiplexed targets. In a comparison of pathologist scoring of diaminobenzidine staining of serial sections and automated MxIF scoring of a single section, human epidermal growth factor receptor 2, estrogen receptor, p53, and androgen receptor staining by diaminobenzidine and MxIF methods yielded similar results. Single-cell staining patterns of 61 protein antigens by MxIF in 747 colorectal cancer subjects reveals extensive tumor heterogeneity, and cluster analysis of divergent signaling through ERK1/2, S6 kinase 1, and 4E binding protein 1 provides insights into the spatial organization of mechanistic target of rapamycin and MAPK signal transduction. Our results suggest MxIF should be broadly applicable to problems in the fields of basic biological research, drug discovery and development, and clinical diagnostics.
Automated segmentation and quantification of cellular and subcellular components in multiplexed images has allowed for a combination of both spatial and protein expression information to become available for analysis. However, performing analyses across multiple patients and tissue types continues to be a challenge, as well as the greater challenge of tissue classification itself. We propose a model of tissues as interconnected networks of epithelial cells whose connectivity is determined by their size, specific expression levels, and proximity to other cells. These Biomarker Enhanced Tissue Networks (BETN) reflect both the individual nature of the cells and the complex cell to cell relationships within the tissue. Performing a simple analysis of such tissue networks managed to successfully classify epithelial cells from stromal cells across multiple patients and tissue types. Further experiments show that significant information about the structure and nature of tissues can also be extracted through analysis of the networks, which will hopefully move towards the eventual goal of true tissue classification. © 2012 IEEE.
We present an algorithm to adjust the contrast of individual dyes from colour (red-green-blue) images of dye mixtures. Our technique is based on first decomposing the colour image into individual dye components, then adjusting each of the dye components and finally mixing the individual dyes to generate colour images. Specifically in this paper, we digitally adjust the staining proportions of hematoxylin and eosin (H&E) chromogenic dyes in tissue images. We formulate the physical dye absorption process as a non-negative mixing equation, and solve the individual components using non-negative matrix factorisation (NMF). Our NMF formulation includes camera dark current in addition to the mixing proportions and the individual H and E components. The novelty of our approach is to adjust the dye proportions while preserving the color of nonlinear dye interactions, such as pigments and red blood cells. In this paper we present results for only H&E images, our technique can easily be extended to other staining techniques.
While high-content screening is already playing an important role in drug discovery, a growing number of academic laboratories are applying these techniques to conduct a system-level analysis of biological processes. In this context more complex assays and model systems are being imaged at higher throughput. Examples include co-culture assays, tissues, and entire model systems, as for example zebrafish. This summary presents examples of methods and algorithms that have been developed at GE Global Research that facilitate the quantitative analysis of such complex specimens. 1 © 2011 IEEE.
A growing number of screening applications require the automated monitoring of cell populations enabled by cell segmentation and tracking algorithms in a high-throughput, high-content environment. Building upon the tracks generated by such algorithms, we derive biologically relevant features and demonstrate a range of biological studies made possible by such quantitative measures. In the first, we introduce a combination of quantitative features that characterize cell apoptosis and arrest. In the second, we automatically measure the effect of motility-promoting serums. In the third, we show that proper dosage levels can be automatically determined for studying protein translocations. These results provide large-scale quantitative validation of biological experiments and demonstrate that our framework provides a valuable tool for high-throughout biological studies. © 2011 IEEE.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6892 LNCS (PART 2), pp. 343-351. | Citations: 4 (Scopus) | Show Abstract | Read more2011. Non-parametric Population Analysis of Cellular Phenotypes
Methods to quantify cellular-level phenotypic differences between genetic groups are a key tool in genomics research. In disease processes such as cancer, phenotypic changes at the cellular level frequently manifest in the modification of cell population profiles. These changes are hard to detect due the ambiguity in identifying distinct cell phenotypes within a population. We present a methodology which enables the detection of such changes by generating a phenotypic signature of cell populations in a data-derived feature-space. Further, this signature is used to estimate a model for the redistribution of phenotypes that was induced by the genetic change. Results are presented on an experiment involving deletion of a tumor-suppressor gene dominant in breast cancer, where the methodology is used to detect changes in nuclear morphology between control and knockout groups. © 2011 Springer-Verlag.
Recognizing the presence of object classes in an image, or image classification, has become an increasingly important topic of interest. Equally important, however, is also the capability to locate these object classes in the image. We consider in this paper an approach to these two related problems with the primary goal of minimizing the training requirements so as to allow for ease of adding new object classes, as opposed to approaches that favor training a suite of object-specific classifiers. To this end, we provide the analysis of an exemplar-based approach that leverages unsupervised clustering for classification purpose, and sliding window matching for localization. While such exemplar based approach by itself is brittle towards intraclass and viewpoint variations, we achieve robustness by introducing a novel Conditional Random Field model that facilitates a straightforward accept/reject decision of the localized object classes. Performance of our approach on the PASCAL Visual Object Challenge 2007 dataset demonstrates its efficacy. © 2011 Springer-Verlag.
A growing number of screening applications require the automated monitoring of cell populations in a high-throughput, high-content environment. These applications depend on accurate cell tracking of individual cells that display various behaviors including mitosis, merging, rapid movement, and entering and leaving the field of view. Many approaches to cell tracking have been developed in the past, but most are quite complex, require extensive post-processing, and are parameter intensive. To overcome such issues, we present a general, consistent, and extensible tracking approach that explicitly models cell behaviors in a graph-theoretic framework. We introduce a way of extending the standard minimum-cost flow algorithm to account for mitosis and merging events through a coupling operation on particular edges. We then show how the resulting graph can be efficiently solved using algorithms such as linear programming to choose the edges of the graph that observe the constraints while leading to the lowest overall cost. This tracking algorithm relies on accurate denoising and segmentation steps for which we use a wavelet-based approach that is able to accurately segment cells even in images with very low contrast-to-noise. In addition, the framework is able to measure and correct for microscope defocusing and stage shift. We applied the algorithms on nearly 6000 images of 400,000 cells representing 32,000 tracks taken from five separate datasets, each composed of multiple wells. Our algorithm was able to segment and track cells and detect different cell behaviors with an accuracy of over 99%. This overall framework enables accurate quantitative analysis of cell events and provides a valuable tool for high-throughput biological studies.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6801 LNCS pp. 398-410. | Citations: 2 (Scopus) | Show Abstract | Read more2011. Identifying Nuclear Phenotypes Using Semi-supervised Metric Learning
In systems-based approaches for studying processes such as cancer and development, identifying and characterizing individual cells within a tissue is the first step towards understanding the large-scale effects that emerge from the interactions between cells. To this end, nuclear morphology is an important phenotype to characterize the physiological and differentiated state of a cell. This study focuses on using nuclear morphology to identify cellular phenotypes in thick tissue sections imaged using 3D fluorescence microscopy. The limited label information, heterogeneous feature set describing a nucleus, and existence of sub-populations within cell-types makes this a difficult learning problem. To address these issues, a technique is presented to learn a distance metric from labeled data which is locally adaptive to account for heterogeneity in the data. Additionally, a label propagation technique is used to improve the quality of the learned metric by expanding the training set using unlabeled data. Results are presented on images of tumor stroma in breast cancer, where the framework is used to identify fibroblasts, macrophages and endothelial cells - three major stromal cells involved in carcinogenesis. © 2011 Springer-Verlag.
We propose a shape model fitting algorithm that uses linear programming optimization. Most shape model fitting approaches (such as ASM, AAM) are based on gradient-descent-like local search optimization and usually suffer from local minima. In contrast, linear programming (LP) techniques achieve globally optimal solution for linear problems. In , a linear programming scheme based on successive convexification was proposed for matching static object shape in images among cluttered background and achieved very good performance. In this paper, we rigorously derive the linear formulation of the shape model fitting problem in the LP scheme and propose an LP shape model fitting algorithm (LPSM). In the experiments, we compared the performance of our LPSM with the LP graph matching algorithm(LPGM), ASM, and a CONDENSATION based ASM algorithm on a test set of PUT database. The experiments show that LPSM can achieve higher shape fitting accuracy. We also evaluated its performance on the fitting of some real world face images collected from internet. The results show that LPSM can handle various appearance outliers and can avoid local minima problem very well, as the fitting is carried out by LP optimization with l 1 norm robust cost function. © 2011 IEEE.
Recent advances in visual tracking methods allow following a given object or individual in presence of significant clutter or partial occlusions in a single or a set of overlapping camera views. The question of when person detections in different views or at different time instants can be linked to the same individual is of fundamental importance to the video analysis in large-scale network of cameras. This is the person reidentification problem. The paper focuses on algorithms that use the overall appearance of an individual as opposed to passive biometrics such as face and gait. Methods that effectively address the challenges associated with changes in illumination, pose, and clothing appearance variation are discussed. More specifically, the development of a set of models that capture the overall appearance of an individual and can effectively be used for information retrieval are reviewed. Some of them provide a holistic description of a person, and some others require an intermediate step where specific body parts need to be identified. Some are designed to extract appearance features over time, and some others can operate reliably also on single images. The paper discusses algorithms for speeding up the computation of signatures. In particular it describes very fast procedures for computing co-occurrence matrices by leveraging a generalization of the integral representation of images. The algorithms are deployed and tested in a camera network comprising of three cameras with non-overlapping field of views, where a multi-camera multi-target tracker links the tracks in different cameras by reidentifying the same people appearing in different views. © 2011 Springer-Verlag.
Med Image Comput Comput Assist Interv, 14 (Pt 2), pp. 343-351. | Citations: 5 (Web of Science Lite) | Show Abstract2011. Non-parametric population analysis of cellular phenotypes.
Methods to quantify cellular-level phenotypic differences between genetic groups are a key tool in genomics research. In disease processes such as cancer, phenotypic changes at the cellular level frequently manifest in the modification of cell population profiles. These changes are hard to detect due the ambiguity in identifying distinct cell phenotypes within a population. We present a methodology which enables the detection of such changes by generating a phenotypic signature of cell populations in a data-derived feature-space. Further, this signature is used to estimate a model for the redistribution of phenotypes that was induced by the genetic change. Results are presented on an experiment involving deletion of a tumor-suppressor gene dominant in breast cancer, where the methodology is used to detect changes in nuclear morphology between control and knockout groups.
Inf Process Med Imaging, 22 pp. 398-410. | Citations: 3 (Web of Science Lite) | Show Abstract2011. Identifying nuclear phenotypes using semi-supervised metric learning.
In systems-based approaches for studying processes such as cancer and development, identifying and characterizing individual cells within a tissue is the first step towards understanding the large-scale effects that emerge from the interactions between cells. To this end, nuclear morphology is an important phenotype to characterize the physiological and differentiated state of a cell. This study focuses on using nuclear morphology to identify cellular phenotypes in thick tissue sections imaged using 3D fluorescence microscopy. The limited label information, heterogeneous feature set describing a nucleus, and existence of subpopulations within cell-types makes this a difficult learning problem. To address these issues, a technique is presented to learn a distance metric from labeled data which is locally adaptive to account for heterogeneity in the data. Additionally, a label propagation technique is used to improve the quality of the learned metric by expanding the training set using unlabeled data. Results are presented on images of tumor stroma in breast cancer, where the framework is used to identify fibroblasts, macrophages and endothelial cells--three major stromal cells involved in carcinogenesis.
The systems-level analysis of complex biological processes requires methods that enable the quantification of a broad range of phenotypical alterations, the precise localization of signaling events, and the ability to correlate such signaling events in the context of the spatial organization of the biological specimen. The goal of this review is to illustrate that, when combined with modern imaging platforms and labeling techniques, automated image analysis methods can provide such quantitative information. The article attempts to review necessary image analysis techniques as well as applications that utilize these techniques to provide the data that will enable systems-level biology. The text includes a review of image registration and image segmentation methods, as well as algorithms that enable the analysis of cellular architecture, cell morphology, and tissue organization. Various methods that enable the analysis of dynamic events are also presented.
Image focus quality is of utmost importance in digital microscopes because the pathologist cannot accurately characterize the tissue state without focused images. We propose to train a classifier to measure the focus quality of microscopy scans based on an extensive set of image features. However, classifiers rely heavily on the quality and quantity of the training data, and collecting annotated data is tedious and expensive. We therefore propose a new method to automatically generate large amounts of training data using image stacks. Our experiments demonstrate that a classifier trained with the image stacks performs comparably with one trained with manually annotated data. The classifier is able to accurately detect out-of-focus regions, provide focus quality feedback to the user, and identify potential problems of the microscopy design.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6362 LNCS (PART 2), pp. 446-453. | Citations: 2 (Scopus) | Show Abstract | Read more2010. Automated Training Data Generation for Microscopy Focus Classification
Image focus quality is of utmost importance in digital microscopes because the pathologist cannot accurately characterize the tissue state without focused images. We propose to train a classifier to measure the focus quality of microscopy scans based on an extensive set of image features. However, classifiers rely heavily on the quality and quantity of the training data, and collecting annotated data is tedious and expensive. We therefore propose a new method to automatically generate large amounts of training data using image stacks. Our experiments demonstrate that a classifier trained with the image stacks performs comparably with one trained with manually annotated data. The classifier is able to accurately detect out-of-focus regions, provide focus quality feedback to the user, and identify potential problems of the microscopy design. © 2010 Springer-Verlag.
We present a study of the spatial variation of nuclear morphology of stromal and cancer-associated fibroblasts in the mouse mammary gland. The work is part of a framework being developed for the analysis of the tumor microenvironment in breast cancer. Recent research has uncovered the role of stromal cells in promoting tumor growth and progression. In specific, studies have indicated that stromal fibroblasts - formerly considered to be passive entities in the extra-cellular matrix - play an active role in the progression of tumor in mammary tissue. We have focused on the analysis of the nuclear morphology of fibroblasts, which several studies have shown to be a critical phenotype in cancer. An essential component of our approach is that the nuclear morphology is studied within the 3D spatial context of the tissue, thus enabling us to pose questions about how the locus of a cell relates to its morphology, and possibly to its function. In order to make quantitative comparisons between nuclear populations, we build statistical shape models of cell populations and infer differences between the populations through these models. We present our observation on both normal and tumor tissues from the mouse mammary gland. ©2010 IEEE.
Total publications on this page: 23
Total citations for publications on this page: 482