The objective of this research is to develop innovative techniques for modeling and control of network congestion. Most existing network controls have discontinuous actions, but such discontinuity in control actions is commonly omitted in analytical models, and instead continuous models were widely adopted in the literature. This approximation works well under certain conditions, but it does cause significant discrepancy in creating robust, responsive control solutions for congestion management. In this dissertation, I investigated three major topics. I proposed a generic discontinuous congestion control model and its design methodology to guarantee asymptotic stability and eliminate traffic oscillation, based on the sliding mode control (SMC) theory. My scheme shows that discontinuity plays a crucial role in optimization of the I-D based congestion control algorithms. When properly modeled, the simple I-D control laws can be made highly robust to parameter and model uncertainties. I discussed applicability of this model to some existing flow or congestion control schemes, e.g. XON/XOFF, rate and window based AIMD, RED, etc. It can also be effectively applied to design of detection and defense of distributed denial of service (DDoS) attacks. DDoS management can be considered a special case of the flow control problem. Based on my generic discontinuous congestion control model, I developed a backward-propagation feedback control strategy for DDoS detection and defense. It not only prevents DDoS attacks but also provides smooth traffic and bounded queue size. Another application of the congestion control algorithms is design of private group communication networks. I proposed a new technique for protection of group communications by concealment of sender-recipient pairs. The basic approach is to fragment and disperse encrypted messages into packets to be transported along different paths, so that the adversary cannot efficiently determine the source/recipient of a message without correct ordering of all packets. Packet flows among nodes are made balanced, to eliminate traffic patterns related to group activities. I proposed a sliding window-based flow control scheme to control transmission of payload and dummy packets. My algorithms allow flexible tradeoff between traffic concealment and performance requirement.