Abstract This research work focuses on how to improve the toughness of heat affected zones (HAZs) of low heat input welds in the case of high strength thermomechanically processed (TMCP) and recrystallization controlled rolled and accelerated cooled (RCR) plates with yield strengths of 355–500 MPa. Experimental work was aimed at the investigation of the intragranular nucleation of acicular ferrite or bainite in hot-rolled plates and the evaluation of the Charpy V and CTOD toughness of the most critical sub-zones of the weld HAZ using simulated specimens with a cooling time t8/5 = 5 s. The zones studied were the coarse grained HAZ (CGHAZ), the intercritically reheated coarse-grained HAZ (ICCGHAZ) and the intercritical HAZ (ICHAZ), the metallographical analyses consisted of microstructural investigations complemented with hardness measurements. Optical, scanning and transmission electron microscopy techniques were employed together with image and electron backscatter diffraction (EBSD) analysis. The test results showed that the toughness of the various sub-zones of the HAZ is improved by promoting intragranularly nucleated ferritic-bainitic (acicular) microstructure in both the CGHAZ and in the base plate. In this way, the sub-zones subjected to intercritical thermal cycles (the ICCGHAZ and the ICHAZ) develop evenly distributed M-A constituents between ferrite and bainite laths. These favourable microstructures can be achieved by using titanium killing or by avoiding niobium microalloying by using copper plus nickel alloying instead. In the laboratory experiments titanium killed steel, containing titanium-manganese oxide/manganese sulphide inclusions with a number density of 300–750 particles/mm2, develops a largely acicular ferritic microstructure in the base plate provided the austenite grain size is greater than about 120 μm and the cooling rate is in the range 6–11 °C/s down to 500 °C. Under plate mill conditions, no significant amount of acicular ferrite could be obtained, because it was not possible to achieve austenite grain sizes larger than about 70 μm after rolling. However, a significant fraction of acicular ferritic-bainitic microstructure was achieved in the CGHAZ, when the austenite grain size exceeded 90 μm. When achieved, a uniform distribution of M-A particles in an acicular ferritic-bainitic microstructure improves toughness. Cracks nucleate at numerous sites on M-A/ferrite boundaries or bainite packet interfaces, but they are initially arrested in the acicular matrix. Crack growth finally occurs by linking of the numerous arrested microcracks.