Cluster Benchmarking Challenge Call: Published or Accepted Benchmarking Papers of Clustering

The Cluster Benchmarking Task Force of the International Federation of Classification Societies (IFCS) is calling for benchmarking papers on clustering methods that have been published or accepted for publication after January 1, 2019. This call, which is scientifically sponsored by the IFCS Executive Committee, is part of a challenge connected with the 2022 IFCS conference in Porto, Portugal, July 19-23.

To achieve a cumulative building of knowledge on clustering and classification, careful attention to benchmarking (performance comparison of methods) is very important. New methods of data pre-processing, new data-analytic techniques, and new methods of output post-processing, should be extensively and carefully compared with existing alternatives, and existing methods should be part of neutral comparison studies.

The Task Force has written a white paper (for a preprint, see: https://arxiv.org/abs/1809.10496) that addresses the theoretical and conceptual underpinnings of benchmarking in cluster analysis, and some practicalities for performing sound benchmarking studies. As part of a challenge, the Task Force is now calling for authors to contribute published or accepted benchmarking papers on cluster analysis.

Rules for submission:

  • Contributions for the challenge must be e-mailed to Iven.VanMechelen@kuleuven.be .
  • A contribution should be an English language paper on a benchmarking study in the context of cluster analysis that has been published or accepted for publication in a peer-reviewed, international publication outlet.
  • Contributions (which may include, but are not limited to journal articles, book chapters, and proceedings articles) should be published or accepted for publication after January 1, 2019. For accepted papers, a proof of acceptance (e.g., e-mail of acceptance) should also be submitted.
  • A contributor should be an author of the paper.
  • The benchmarking study may compare methods of data pre-processing, clustering (potentially but not necessarily including the estimation of cluster parameters such as the number of clusters), and/or output post-processing, and may use empirical or simulated data.
  • The deadline for submissions is March 15, 2022, 12:00 pm CET.

Competition guidelines:

  • Contributions to the challenge will be evaluated by the IFCS Task Force on Benchmarking.
  • Major evaluation criteria will be:
    • the technical correctness and clarity of the paper,
    • that the study shows a critical reflection on principles on which a sound benchmarking study should be based (which may be either in line or at odds with the principles outlined in the white paper of the Task Force),
    • a detailed specification of the choices made in the benchmarking study at issue along with a justification of these based on the principles referred to above,
    • the degree of neutrality of the author(s) with regard to the methods under study.
  • Authors of contributions that are rated as excellent will be invited for a presentation in a special session at the 2022 IFCS conference in Porto, Portugal (such a presentation will not conflict with giving another presentation at that conference). Authors of the selected contributions will be notified no later than April 20, 2022. Authors of selected contributions who cannot attend the conference in person may exceptionally give an online presentation.
  • During the closing session of the IFCS conference in Porto one of the selected contributions will be proclaimed winner of the challenge. The winning author(s) will receive a plaque and a CRC/Chapman and Hall book voucher.
  • Contact for questions: Iven.VanMechelen@kuleuven.be .

All researchers interested in clustering and classification are most welcome to participate in this challenge!

Task Force members:
Anne-Laure Boulesteix, Nema Dean, Isabelle Guyon, Christian Hennig, Friedrich Leisch, Douglas Steinley, Iven Van Mechelen, and Matthijs Warrens.