This work presents a new technique for automatic detection of marine vehicles in images and video of open sea. Users of such system include border guards, military, port safety, flow management, and sanctuary protection personnel. The source of images and video is a digital camera or a camcorder which is placed on a buoy or stationary mounted in a harbor facility. The system is intended to work autonomously, taking images of the surrounding ocean surface and analyzing them for the presence of marine vehicles. The goal of the system is to detect an approximate window around the ship. The proposed computer vision-based algorithm combines a horizon detection method with edge detection and postprocessing. Several datasets of still images are used to evaluate the performance of the proposed technique. For video sequences the original algorithm is further enhanced with a tracking algorithm that uses Kalman filter. A separate dataset of 30 video sequences 10 seconds each is used to test its performance. Promising results of the detection of ships are discussed and necessary improvements for achieving better performance are suggested.