In the field of industrial automation there is a huge demand for real time communication networks and there are several different protocols like EtherCAT, PROFINET IO, SERCOS competing each other in the market. Many of the products in this industry are subjected to hard real-time communication requirements. The purpose of this thesis is in introducing tools to automatically test the various requirements that are helpful in deciding the performance of real time communication systems. Developing a well-defined test framework is one of the important tasks in this thesis project. The end users of the test framework can focus more on managing and analyzing the results from the framework instead of the design process. The thesis work presents the test requirements, design of test system and automating the measurement process by selecting appropriate hardware. The report also explains the design of commands, method used for communication between different systems and also discusses the different methods that could be applied for measuring the performance, the limitations of some of these methods when applied to the framework. The architecture and the working of the framework is covered in later chapters. The framework uses EtherCAT master communication stack developed in-house at ABB. EtherCAT is just used as a pilot test case, but in general the framework could be applied to other Ethernet based industrial communication protocols with suitable hardware or software modifications.