This research was to develop a real-time, modifiable, tangible interactive tabletop system for participatory urban design. The targeting user group was those stakeholders in urban design charrettes. Previous system solutions overlooked the importance of the modifiable tangible medium in the situation of reaching spatial-temporal consensus. These design issues impeded communication between the stakeholders and the professionals. Users of these systems had difficulties expressing ideas to professionals during the collaborative design process. Literature in evolving technology in the smart city context, collaborative urban design, embodied interaction, and depth-sensing was referred to guide the system design. Based on the review, this research identified the pivotal role of a shapeable and tangible medium in the system. The prototype system unified the modifiable, realistic model with its digital equivalent in urban analytics in real-time. By integrating tangible interaction, depth-sensing, and large touch screen tabletop, an intuitive, immersive decision-making interface for non-professional stakeholders could be created. During the system implementation, system elements centering ‘tangible interoperability’ were documented along the system pipeline. A heuristic evaluation, a method of usability inspection, was conducted to assess and to guide the future system design. The result was promising and inspiring. In the end, challenges and directions of system design were discussed. The contribution of this research included: discovering direction, centering tangibility, implementing a prototype, and documenting elements in each stage along the system pipeline of designing a modifiable tangible interactive tabletop system for the urban design charrette.