<p> Visual knowledge plays an important role in many highly skilled applications, such as medical diagnosis, geospatial image analysis and pathology diagnosis. Medical practitioners are able to interpret and reason about diagnostic images based on not only primitive-level image features such as color, texture, and spatial distribution but also their experience and tacit knowledge which are seldom articulated explicitly. This reasoning process is dynamic and closely related to real-time human cognition. Due to a lack of visual knowledge management and sharing tools, it is difficult to capture and transfer such tacit and hard-won expertise to novices. Moreover, many mission-critical applications require the ability to process such tacit visual knowledge in real time. Precisely how to index this visual knowledge computationally and systematically still poses a challenge to the computing community.</p><p> My dissertation research results in novel computational approaches for highthroughput visual knowledge analysis and retrieval from large-scale databases using latest technologies in big data ecosystems. To provide a better understanding of visual reasoning, human gaze patterns are qualitatively measured spatially and temporally to model observers&rsquo; cognitive process. These gaze patterns are then indexed in a NoSQL distributed database as a visual knowledge repository, which is accessed using various unique retrieval methods developed through this dissertation work. To provide meaningful retrievals in real time, deep-learning methods for automatic annotation of visual activities and streaming similarity comparisons are developed under a gaze-streaming framework using Apache Spark. </p><p> This research has several potential applications that offer a broader impact among the scientific community and in the practical world. First, the proposed framework can be adapted for different domains, such as fine arts, life sciences, etc. with minimal effort to capture human reasoning processes. Second, with its real-time visual knowledge search function, this framework can be used for training novices in the interpretation of domain images, by helping them learn experts&rsquo; reasoning processes. Third, by helping researchers to understand human visual reasoning, it may shed light on human semantics modeling. Finally, integrating reasoning process with multimedia data, future retrieval of media could embed human perceptual reasoning for database search beyond traditional content-based media retrievals.</p><p>

<p> In volumetric image analysis and visualization, challenges have be induced by the increasing size of volume over recent years. Rendering and interacting with a volume with reduced size is preferable and highly needed. The primary concern in producing such downsized volumetric images is to preserve the important structures and those of the user's interest, such as boundaries between materials. Typical volume reduction approaches usually perform uniform subsampling without the awareness of user-specified parameters such as the opacity and color transfer functions. However, it is also handy for the algorithm to have "global'' encoding and control over the entire volume, meanwhile revealing some features of the data while it is being downsized. This thesis aims at providing a means of such type, extended from the famous seam carving operator that has been used widely in the task of image and video retargeting. </p><p> Our work applies and extends the seam carving algorithm for videos proposed by Rubinstein et al. to downsize three-dimensional volumetric images. This extended technique computes and removes from the volume two-dimensional seams, or what we name and define as sheets, to reduce the size of the volume with minimum loss of important details measured by gradient. We aim at learning through experimentation the visual quality of seam carved volumetric images, making improvements based on feedback and potentially paving ways towards applications. With the great flexibility of the graph cut formulation, we implement in our algorithm the existing backward and forward energy optimization, and add extensions including isosurface protection and the encoding of the opacity transfer function. </p><p> At the visual level, experimental results tell us when applied alone with fixed parameters, volumetric seam carving outperforms trivial approaches in preserving important structures only for part of the datasets, on which discussions are included at the best knowledge of the author.</p><p>

<p> Autosomal Dominant Polycystic Kidney Disease (ADPKD) is among the most prevalent life-threatening genetic conditions. Despite this, no approved medical therapies exist to treat the disease. Until the recent past, no methods could reliably measure the course of the disease far in advance of end stage renal disease (ESRD). As normal tissue is progressively destroyed or blocked by enlarging cysts, remaining nephrons compensate in a process called hyperfiltration. This beneficial physiological response confounds tests of renal function. Thus, potential interventions could not be tested against a reliable measurement of disease progression. </p><p> However, progressive changes are visually apparent on medical imaging examinations throughout the course of ADPKD. The search for ADPKD proxy biomarkers is now focused on quantitative imaging, or the extraction of information from medical images for purposes of diagnosis or disease tracking. Recent studies from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)- sponsored Consortium for Radiologic Imaging Studies of Polycystic Kidney Disease (CRISP) showed Total Kidney Volume (TKV) is a usable quantitative imaging biomarker which can track disease in the early, asymptomatic phase and register measurable changes in as little as 12 months. These findings launched several new trials into potential ADPKD therapies. </p><p> Advanced analysis of polycystic kidney images, however, has never been done. The method CRISP used to extract TKV was stereology, an efficient means to estimate volume. However, stereology was tradi- tionally a dead end for further advanced analysis. TKV is useful for clinical trials and large population-based studies, but cannot accurately predict disease progression or stratify risk due to known out- lier cases. Thus, the utility of TKV for individual patient prognosis is limited. This work builds upon stereology data, describing a reliable and accurate new semi-automatic method to fully segment images us- ing only labeled stereology grids. Then, two new second generation quantitative imaging biomarkers are introduced and analyzed: Cyst- Parenchyma Surface Area (CPSA) and cyst concentration. These new physiologically motivated biomarkers will complement or potentially replace TKV in efforts to bring quantitative imaging to individual patients. </p><p> The goal of this body of work is to enable a pathway for efficient advanced image analysis in ADPKD, never before attempted in this dis- order, and to define new quantitative imaging biomarkers which will complement or replace existing ones in hopes of making individualized disease tracking for ADPKD patients a reality.</p>

<p> Eye exam can be as efficacious as physical one in determining health concerns. Retina screening can be the very first clue to detecting a variety of hidden health issues including pre-diabetes and diabetes. Through the process of clinical diagnosis and prognosis; ophthalmologists rely heavily on the binary segmented version of retina fundus image; where the accuracy of segmented vessels, optic disc and abnormal lesions extremely affects the diagnosis accuracy which in turn affect the subsequent clinical treatment steps. This thesis proposes an automated retinal fundus image segmentation system composed of three segmentation subsystems follow same core segmentation algorithm. Despite of broad difference in features and characteristics; retinal vessels, optic disc and exudate lesions are extracted by each subsystem without the need for texture analysis or synthesis. For sake of compact diagnosis and complete clinical insight, our proposed system can detect these anatomical structures in one session with high accuracy even in pathological retina images. </p><p> The proposed system uses a robust hybrid segmentation algorithm combines adaptive fuzzy thresholding and mathematical morphology. The proposed system is validated using four benchmark datasets: DRIVE and STARE (vessels), DRISHTI-GS (optic disc), and DIARETDB1 (exudates lesions). Competitive segmentation performance is achieved, outperforming a variety of up-to-date systems and demonstrating the capacity to deal with other heterogenous anatomical structures. </p><p>