Background: Visual food stimuli have been shown to influence desire to eat and may influence overall energy intake. The purpose of this study was to determine the influence, if any, that time of day has on the neural response to visual food stimuli, as measured by functional magnetic resonance imaging (fMRI). Methods: Using a crossover design, 15 healthy women were scanned using fMRI while presented with low- and high-energy pictures of food, once in the morning (6:30–8:30 am) and once in the evening (5:00–7:00 pm). Diets were identical on both days of the fMRI scans and were verified using weighed food records. Pictures used were based on the work of Sweet et al. (2012). Visual analog scales were used to record subjective perception of hunger and preoccupation with food prior to each fMRI scan. Results: Nine brain regions showed significantly higher activation for high energy stimuli compared to low energy stimuli (p < 0.05). Six areas of the brain showed lower activation in the evening to both high and low energy stimuli including parts of some reward pathways (p < 0.05). Subjectively, participants reported no difference in hunger by time of day (F(1, 14) = 1.84, p = 0.19), but felt they could eat more (F = 4.83, p = 0.04) and were more preoccupied with thoughts of food (F = 5.51, p = 0.03) in the evening compared to the morning. Conclusions: High energy food stimuli tended to produce greater fMRI responses than low energy foods in specific areas of the brain, regardless of time of day. However, evening scans showed a lower response to both food categories in some areas of the brain compared to the morning. These data may have clinical implications for fMRI measurement and practical implications for sensitivity to food cues and eating behavior.