This thesis presents novel methods for creating and improving hierarchical hidden Markov models. The work centers around transforming a traditional tree structured hierarchical hidden Markov model (HHMM) into an equivalent model that reuses repeated sub-trees. This process temporarily breaks the tree structure constraint in order to leverage the benefits of combining repeated sub-trees. These benefits include lowered cost of testing and an increased accuracy of the final model-thus providing the model with greater performance. The result is called a merged and simplified hierarchical hidden Markov model (MSHHMM). The thesis goes on to detail four techniques for improving the performance of MSHHMMs when applied to information extraction tasks, in terms of accuracy and computational cost. Briefly, these techniques are: a new formula for calculating the approximate probability of previously unseen events; pattern generalisation to transform observations, thus increasing testing speed and prediction accuracy; restructuring states to focus on state transitions; and an automated flattening technique for reducing the complexity of HHMMs. The basic model and four improvements are evaluated by applying them to the well-known information extraction tasks of Reference Tagging and Text Chunking. In both tasks, MSHHMMs show consistently good performance across varying sizes of training data. In the case of Reference Tagging, the accuracy of the MSHHMM is comparable to other methods. However, when the volume of training data is limited, MSHHMMs maintain high accuracy whereas other methods show a significant decrease. These accuracy gains were achieved without any significant increase in processing time. For the Text Chunking task the accuracy of the MSHHMM was again comparable to other methods. However, the other methods incurred much higher processing delays compared to the MSHHMM. The results of these practical experiments demonstrate the benefits of the new method-increased accuracy, lower computation costs, and better performance.

microRNAs (miRNAs) are small (20-23 nt), non-coding single stranded RNA molecules that play an important role in post-transcriptional regulation of protein-coding genes. miRNAs have been found in all animal lineages, and have been implicated as critical regulators during development in multiple species. The echinoderms, Strongylocentrotus purpuratus (sea urchin) and Patiria miniata (sea star) are excellent model organisms for studying development due to their well-characterized transcriptional gene networks, ease of working with their embryos in the laboratory and phylogenetic position as invertebrate deuterostomes. Literature on miRNAs in echinoderm embryogenesis is limited. It has been shown that RNAi genes are developmentally expressed and regulated in sea urchin embryos, but no study in the sea urchin has examined the expression of miRNAs. The goal of my work has been to study miRNA regulation in echinoderm developmental gene networks. I have identified developmentally regulated miRNAs in sea urchin and sea star embryos, using a combination of computational and wet lab experimental techniques. I developed a probabilistic model (named HHMMiR) based on hierarchical hidden Markov models (HHMMs) to classify genomic hairpins into miRNA precursors and random stem-loop structures. I then extended this model to make an efficient decoder by introduction of explicit state duration densities. We used the Illumina Genome Analyzer to sequence small RNA libraries in mixed stage population of embryos from one to three days after fertilization of S. purpuratus and P. miniata. We developed a computational pipeline for analysis of these miRNAseq data to reveal the miRNA populations in both species, and study their differential expression. We also used northern blots and whole mount in situ hybridization experimental techniques to study the temporal and spatial expression patterns of some of these miRNAs in sea urchin embryos. By knocking down the major components of the miRNA biogenesis pathway, we studied the global effects of miRNAs on embryo morphology and differentiation genes. The biogenesis genes selected for this purpose are the RNAse III enzyme, Dicer and Argonaute. Dicer is necessary for the processing of mature miRNAs from hairpin structures while Ago is a necessary part of the RISC (RNA interference silencing complex) assembly, which is required for the miRNA to hybridize to its target mRNA site. Knocking down these genes hinders normal development of the sea urchin embryo and leads to loss of the larval skeleton, a novel phenotype not seen in sea stars, as well as abnormal gastrulation. Comparison of differentiation gene marker expression between control and Ago knocked down sea urchin embryos shows interesting patterns of expansion and suppression of adjoining some embryonic territories, while ingression of larval skeletogenesis progenitors does not occur.