Context: The problem this thesis address is concerned with experimental software engineering and is associated with validity of the studies with students as subjects. Often in software engineering experiments students are used because students are easily available and are willing to participate in experiment studies, professionals are rarely available to participate in research study and there are several benefits of using students as research subject.The validity of experiments studied with students are questioned because students are less experienced and skilled compared to professionals. Objectives: The goal of this research is to find evidence whether or not the research studies with students as subjects can be trusted to get valid results. Identify factors that influence the results of experiment studies with students. Identify the ways to improve the validity of experiment studies with students. Methods: In this systematic literature review a number of article sources are used, including Compendex, Inspec, IEEE Xplore, ACM Digital Library, and Springer Link. Experiment studies in software engineering are selected after reading titles and abstracts to decide whether the articles are relevant to the subject. Snowball sampling is done to identify several experiment studies that address same research problems. Results: It is found that graduate students serve as good proxies to study professional practices compared to undergraduate students, because there are significant differences between result/findings of studies with undergraduate students as subjects and studies with professionals as subjects. The following factors influencing the results of experiment studies are identified: Number of subjects, Task related knowledge, Motivation levels of subjects, Training provided, Task related experience and Incentives or grades given for participation in the experiment. This study also identifies some problems with the way of conducting and reporting experiments in software engineering. Addressing these problems could improve the validity of experiment studies with students. Conclusion: The results indicate that students can be used to study professional practices but they cannot be said true for all areas and problems of software engineering because the results of experiment studies with different type of subjects in different area and problems are varying. Which means that whether or not students could be used to study a professional practice depends upon the research problems being studied, type of students subjects chosen for the study, difference between students chosen and professionals to whom the results/findings will be generalized. We conclude that graduate students serve as good proxies to study professional practices compared to undergraduate students. The cumulative experience or overall industry or academic experience of subjects does not seem to be affecting the result/findings of experiment studies unless the overall experience includes task related experience. Even though the study identified above mentioned factors that influences the results/findings of experiment studies there still might be few factors which the study might have failed to identify due lack of data. We suggested improvements for reporting that could improve the comparability of studies. / Often in software engineering experiments students are used because students are easily available and are willing to participate in experiment studies, where as professionals are rarely available to participate in research study. This thesis investigates whether it is ok to use students as experiments subjects while studying professional practices. Thesis also investigates the factors which influences the results of experiment studies with students. The derived knowledge from the thesis can assists researchers improve the way of conducting and reporting software engineering experiments, It also aids in improving the comparability of experiment studies.
| Identifer | oai:union.ndltd.org:UPSALLA1/oai:DiVA.org:bth-4687 |
| Date | January 2012 |
| Creators | Kotakonda, Sharat Chandra, Engu, Ranjith |
| Publisher | Blekinge Tekniska Högskola, Sektionen för datavetenskap och kommunikation, Blekinge Tekniska Högskola, Sektionen för datavetenskap och kommunikation |
| Source Sets | DiVA Archive at Upsalla University |
| Language | English |
| Detected Language | English |
| Type | Student thesis, info:eu-repo/semantics/bachelorThesis, text |
| Format | application/pdf |
| Rights | info:eu-repo/semantics/openAccess |
Page generated in 0.0033 seconds