After a century of sporadic advances in equipment, tools, materials, and methods, the US construction industry still faces a low rate of productivity growth. To improve the productivity of any site activity, it is important to rapidly record relevant data about utilized resources and processes, as well as about the output quantities produced by these activities. There is sufficient evidence to suggest that activity-level productivity measurement is the premise for making any productivity improvement decision. To date, certain aspects of productivity measurement, such as input/output quantities, are partially automated through advanced project control systems. However, measuring the process of construction activities for productivity improvement remains an elusive goal for most construction companies. This is mostly due to the massive manual effort embedded in these data collection methods. Digital cameras are inexpensive devices that are widely used in the construction industry as an effective site observation method. This opens the door for conducting scientific method studies on complex operations through examining recorded videos. However, in the absence of an efficient video interpretation method, tedious manual reviewing is currently still required to extract productivity information from the recorded videos. This research aims to develop a computational methodology to rapidly and intelligently interpret construction videos into productivity information. It determines what elements can represent the steps and information flows in construction video interpretation. It identifies, develops, and evaluates computer vision algorithms to enable reliable visual recognition and tracking of construction resources in typical construction environments. It develops methods to enable context aware video computing. A software prototype, the Construction Video Analyzer, was developed and implemented based on this conceptual methodology. The proposed methodology was validated through using the developed prototype system to analyze five construction video sequences that record various types of construction operations. The Construction Video Analyzer was able to interpret these videos into productivity information with an accuracy that was close to manual analysis, without the limitations of onsite human observation. The developed methodology provides site management with a tool that can rapidly collect productivity data with greatly reduced manual efforts. / text