The Cluster Benchmarking Task Force of the International Federation of Classification Societies (IFCS) is calling for analyses of a selected data set in their recently launched Cluster Benchmark Data Repository. This call is part of a challenge connected with the IFCS 2017 conference in Tokyo.

The aim of the repository is to stimulate better practice in benchmarking (performance comparison of methods) for cluster analysis by providing a unique resource comprised of a wide variety of well-documented, high quality datasets and simulation routines for use in practical benchmarking. The repository will include datasets with as well as datasets without given “true” clusterings. A unique feature of the repository is that each dataset will be supplied with comprehensive meta-data, including documentation on the specific nature of the clustering problem and on characteristics that useful clusters should fulfill (with scientific justification).

Within the repository one data set has been singled out, namely a data set on baseline and outcome assessment of low back pain patients. This data set, along with associated meta-data and variable descriptions, can be downloaded here. The Task Force is now calling for individuals to contribute cluster analyses of this data set as part of a challenge.

Rules for submission:

  • Contributions for the challenge must be e-mailed to Iven Van Mechelen
  • A contribution should comprise a report on a cluster analysis of some part of the target data set along with an evaluation of its result and files with the cluster assignments and code.
  • More detailed information on the required format can be found in the file README_challenge2.docx.
  • The deadline for submissions is May 15, 2017, 5pm CET.

Competition guidelines:

  • Contributions to the challenge will be evaluated by the IFCS Task Force on Benchmarking.
  • Apart from technical correctness and clarity of the report, major evaluation criteria will be linking choices in the analysis and the evaluation of its result to the available meta-data, and quality of the reflection of what constitutes a good clustering.
  • Up to 8 contributions will be selected for a short presentation during one or two invited sessions at the 2017 IFCS conference in Tokyo, Japan, August 8-10. The authors of the selected contributions will be notified no later than May 31, 2017. Authors of selected contributions who could not attend the conference will be invited to prepare a few slides on their analysis and the associated results.
  • During the closing session of the IFCS conference one or two of the selected contributions will be proclaimed winners of the challenge. The winners will receive a plaque and a book award.

Contact for questions: Iven.VanMechelen@kuleuven.be

All researchers interested in clustering and classification are most welcome to participate in this challenge!

Taskforce members: Anne-Laure Boulesteix, Rainer Dangl, Nema Dean, Isabelle Guyon, Christian Hennig, Friedrich Leisch, Douglas Steinley, and Iven Van Mechelen