Patterning, morphogenesis, and cell divisions are distinct processes during development yet are concurrent and likely highly integrated. However, it has been challenging to investigate them as a whole. Recent advances in imaging and labeling tools make it possible to observe live tissues with high coverage and resolution. In this dissertation work, we developed a novel imaging platform that allowed us to fully capture the early neural tube formation process in live zebrafish embryos at cellular resolution. Importantly, these datasets allow us to reliably track single neural progenitors. These tracks carry information on the history of cell movement, shape change, division, and gene expression all together. By comparing tracks of different progenitor fates, we found they show a spatially noisy response to Sonic hedgehog (Shh) and become specified in a positionally mixed manner, in surprising contrast to the "French Flag" morphogen patterning model. Both cell movement and division contribute to cell mixing. In addition, we decoupled the temporal and genetic regulatory network (GRN) noises in Shh interpretation using tracks that carry both Shh signaling and cell fate reporters. Our tracks suggest that, after specification, progenitors undergo sorting to self-assemble a sharp pattern. Consistent with this hypothesis, we found ectopically induced progenitors move to correct locations. Furthermore, we show that proper adhesion is required for cell sorting to happen (Chapters 2 and 3). In the cleavage stage embryos, the cells on the surface undergo shape changes followed by lineage separation and differentiation. We quantitatively measured this morphogenesis process and tracked cell divisions. By applying a mathematical model we uncover a predictive, and perhaps general link between cell division orientation, mechanical interaction, and the morphogenetic behavior of the whole surface layer (Chapter 4). Finally, we discuss the concepts and tools of cell tracking including a multi-color cell labeling method we developed by modifying the "Brainbow" system (Chapter 5). Together this dissertation showcases the importance and promise of live observation based, quantitative and integrated analysis in our understanding of complex multi-cellular developmental processes.