Open innovation that Henry Chesbrough introduced in 2003 promotes the usage of the input of outsiders to strengthen internal innovation processes and the search for outside commercialization opportunities for what is developed internally. Open innovation has enabled both academics and practitioners to design innovation strategies based on the reality of our connected world. Although the literature has identified and explored a variety of benefits and costs, to the best of our knowledge, no study has reviewed the benefits and costs of open innovation in terms of their importance for strategic performance. To conduct such a study, we need to take into account two main issues. First, the number of benefits and costs of open innovation are multifold; so, to have a comprehensive comparison, a large number of benefits and costs must be compared. Second, to have a fair comparison, benefits and costs must be compared in terms of different performance criteria, including financial and non-financial. Concerning the issues above, we will face a complex process of exploring benefits and costs. In this regard, we use multiple criterion decision-making (MCDM) methods that have shown promising solutions to complex exploratory problems. In particular, we present how using a stochastic multi-criteria clustering algorithm that is one of the recently introduced MCDM methods can bring promising results when it comes to exploring the strategic importance of benefits and costs of open innovation. Since there is no comprehensive understanding of the nature of the benefits and costs of open innovation, the proposed model aims to cluster them into hierarchical groups to help researchers identify the most crucial benefits and costs concerning different dimensions of performance. In addition, the model is able to deal with uncertainties related to technical parameters such as criteria weights and preference thresholds. We apply the model in the context of open innovation for universities concerning their research collaboration with industries. An online survey was conducted to collect experts' opinions on the open-innovation benefits and costs of university-industry research collaboration, given different performance dimensions. The results obtained through the cluster analysis specify that university researchers collaborate with industry mainly because of knowledge-related and research-related reasons rather than economic reasons. This research also indicates that the most important benefits of university-industry research collaboration for universities are implementing the learnings, increased know-how, accessing specialized infrastructures, accessing a greater idea and knowledge base, sensing and seizing new technological trends, and keeping the employees engaged. In addition, the results show that the most important costs are the lack of necessary resources to monitor activities between university and industry, an increased resistance to change among employees, conflict of interest (different missions), an increased employees' tendency to avoid using the knowledge that they do not create themselves, paying time costs associated with bureaucracy rules, and loss of focus. The research's findings enable researchers to analyze open innovation's related issues for universities more effectively and define their research projects on these issues in line with the priorities of universities.