Time- and rate-dependent materials such as polymers, bituminous materials, and soft materials clearly display all four fundamental responses (i.e. viscoelasticity, viscoplasticity, viscodamage, and healing) where contribution of each response strongly depends on the temperature and loading conditions. This study proposes a new general thermodynamic-based framework to specifically derive thermo-viscoelastic, thermo-viscoplastic, thermo-viscodamage, and micro-damage healing constitutive models for bituminous materials and asphalt mixes. The developed thermodynamic-based framework is general and can be applied for constitutive modeling of different materials such as bituminous materials, soft materials, polymers, and biomaterials. This framework is build on the basis of assuming a form for the Helmohelotz free energy function (i.e. knowing how the material stores energy) and a form for the rate of entropy production (i.e. knowing how the material dissipates energy). However, the focus in this work is placed on constitutive modeling of bituminous materials and asphalt mixes. A viscoplastic softening model is proposed to model the distinct viscoplastic softening response of asphalt mixes subjected to cyclic loading conditions. A systematic procedure for identification of the constitutive model parameters based on optimized experimental effort is proposed. It is shown that this procedure is simple and straightforward and yields unique values for the model material parameters. Subsequently, the proposed model is validated against an extensive experimental data including creep, creep-recovery, repeated creep-recovery, dynamic modulus, constant strain rate, cyclic stress controlled, and cyclic strain controlled tests in both tension and compression and over a wide range of temperatures, stress levels, strain rates, loading/unloading periods, loading frequencies, and confinement levels. It is shown that the model is capable of predicting time-, rate-, and temperature-dependent of asphalt mixes subjected to different loading conditions.