How to Separate Real Collaborative Coding Projects from Noise on GitHub?

GitHub hosts over 10 million repositories: a gold mine for software researchers, but also a minefield of noisy projects. These are repos created for personal purposes, like configuration files or script collections, that don’t engage in real collaboration. When researchers pick projects to study, mixing such "noise" with genuine collaborative projects (e.g., machine‑learning frameworks) can skew results. To tackle this, the authors refined the Semi‑Automatic Decision Tree Method (SADTM), an existing technique for classifying collaborative projects. The new approach is more general and precise: tested on the GHTorrent dataset (2012‑2020), it showed greater stability, delivering consistent outcomes across different datasets. Moreover, its precision is high, with an F‑measure ranging from 0.78 to 0.89. In practice, the method better filters out noise and selects truly collaborative projects, allowing scholars to build high‑quality samples without fearing bias. This means future GitHub research can rely on cleaner, more reliable data, speeding up the discovery of new trends in open‑source software.

The SADTM method relies on a semi‑automatic decision tree: it blends predefined rules with supervised learning. Rules include metrics such as the number of unique contributors, commit frequency, presence of typical development files (README, LICENSE, build scripts), and the temporal distribution of activity. The model trains a classifier (e.g., Random Forest) on a manually labeled sample of repositories known to be collaborative or noisy. During prediction, the tree first applies "hard" rules (e.g., more than five active contributors) to quickly filter obvious cases, then passes borderline cases to the classifier. The proposed optimization introduces normalization of counts to account for the exponential growth of repositories over time, reducing bias toward newer projects. Additionally, a 10‑fold cross‑validation mechanism was added to assess model stability across different temporal slices of the GHTorrent dataset. Results show an average F‑measure of 0.84, with lower variance compared to previous methods, indicating greater robustness. The approach scales: it can run on millions of repositories using distributed computing infrastructures (Spark or Hadoop), keeping processing times reasonable (about 2‑3 hours for the entire 2012‑2020 dataset). This makes the method practical for anyone building clean datasets for software mining or open‑source evolution studies.