How does the visual cortex represent and process color? Experimental evidence from macaque monkey suggests that cells selective for color are organized into small, spatially separated blobs in V1, and stripes in V2. This organization is strikingly different from that of orientation and ocular dominance maps, which consist of large, spatially contiguous patterns. In this dissertation, a self-organizing model of the early visual cortex is constructed using natural color image input. The modeled V1 develops realistic color-selective receptive fields, ocular dominance stripes, orientation maps, and color-selective regions, while the modeled V2 also creates realistic colorselective and orientation-selective neurons. V1 color-selective regions are generally located in the center of ocular dominance stripes as they are in biological maps; the model predicts that color-selective regions become more widespread in both cortical regions when the amount of color in the training images is increased. The model also predicts that in V1 there are three types of color-selective regions (red-selective, greenselective, and blue-selective), and that a unique cortical activation pattern exists for each of the HSV colors. In both V1 and V2, when regions of different color-selectivity are located nearby, bands of color form with gradually changing color preferences. The model also develops lateral connections between cells that are selective for similar orientations, matching previous experimental results, and predicts that cells selective for color primarily connect to other cells with similar chromatic preferences. Thus the model replicates the known data on the organization of color preferences in V1 and V2, provides a detailed explanation for how this structure develops and functions, and leads to concrete predictions to test in future experiments.