Confidence distributions (CDs), which provide evidence across all levels of significance, are receiving increasing attention, especially in meta-analysis. Meta-analyses allow independent study results to be combined to produce one overall conclusion and are particularly useful in public health and medicine. For studies with binary outcomes that are rare, many traditional meta-analysis methods often fail (Sutton et al. 2002; Efthimiou 2018; Liu et al. 2018; Liu 2019; Hunter and Schmidt 2000; Kontopantelis et al. 2013). Zabriskie et al. (2021b) develop a permutation-based method to analyze such data when study treatment effects vary beyond what is expected by chance. In this work, we prove that this method can be considered a CD. Additionally, we develop two new metrics to assess a CD's relative performance.