Writing underlies a vast landscape of cultural artifacts, from poetry to journalism to scientific papers. While technology has been used to reduce the cognitive load of writing with accurate next word prediction, recent developments in natural language generation may prove able to go beyond predicting what we were going to write anyway, and give us new ideas relevant to a particular writing task. This proposal, of computers giving writers valuable ideas, is quite new in the history of writing tools, and has so far proven illusory. Existing systems that address story continuation, which present writers with options for the next sentence in their story, has continually found that suggested sentences are nonsensical, inconsistent with what's already written, or a deviation from the writer's intended direction. Thus, it's not understood if---and if so, how---generative language technologies can support writers with complex writing tasks. I address this challenge by focusing on more specific goals than story continuation, and demonstrate that the methods I develop generate coherent, cogent suggestions that writers are able to use in a variety of settings and writing tasks. In this thesis, I consider writing tasks that are constrained by some external expectation, such as the logic of a metaphor or the details of a technical topic, but also require creativity to write a sentence or paragraph that is novel, surprising, and engaging to read. I introduce a design space, based on the cognitive process model of writing, that reveals how constrained, creative writing tasks are not supported by current writing support tools. I then present methods, embedded in systems, to support two challenging constrained, creative writing tasks. With `Metaphoria', I present a method to aid in metaphor writing by generating metaphorical connections between two concepts. With `Sparks', I present a method to aid in science writing by generating sentences that make a connection between a technical topic and typical reader interests. These systems demonstrate that computation has the power to support constrained, creative tasks, and outline how they aid in inspiration, translation, and perspective. Finally, through a qualitative study with a range of creative writers, I uncover the social dynamics that modulate how writers respond to such generative writing support. Collectively, this work demonstrates new methods for using technology to support creative writers, and presents theoretical results that describe both how and why writers make use of such technologies.