Call for challenge cases

Important: Phase 1 is now closed. Please see the Phase 2 Call for Solutions.

This year, SPLC is introducing a challenge track. The idea of the challenge track is to provide participants with a set of case studies that tackle relevant SPL problems and challenge the state of the art. The challenge track happens in two phases. In the first phase, there will be a call for cases. Submitted cases will be reviewed by the challenge co-chairs to ensure that all required information is clearly described. Accepted cases will be part of the official conference proceedings. Authors of the accepted cases are strongly encouraged to attend the challenge track and participate in the discussion.  In the second phase, there will be a call for solutions to the accepted cases. Submitted case solutions will be peer-reviewed by the challenge program committee. Accepted solutions will also be part of the official conference proceedings. Authors of accepted solutions must present their papers during the conference.

We are currently in Phase 1 of the challenge, and the call for cases is open. A case description can be (1) a particular data set with specific questions or (2) a call for a solution to a particular problem in a given context. The following information is needed for both types of case descriptions:

For data sets with specific questions:

• • What is your dataset and how was it obtained?
  • What is its size?
  • How can this data be accessed?
  • Do you provide tools to process the data?
  • Provide at least one concrete question you want participants to answer. You can provide multiple questions.
  • For each question, provide the criteria for evaluating an answer/solution.

For call to solutions:

• What is the concrete problem you want participants to solve?
• How can a solution be evaluated? The following are some ideas on how to specify evaluation criteria:
  • Concrete evaluation metrics (e.g., precision, recall, accuracy etc. depending on the problem)
  • Concrete test cases participants can evaluate their solution against (e.g, provide input and expected output and participants are expected to provide a solution that gets from one point to the other)
  • A list of systems to evaluate their solution against (e.g., a list of C systems that have a large number of nested #ifdefs, because your problem only makes sense in the context of higher-order variability)
  • A reference implementation to compare against, according to particular metrics.

Additional requirements for both types of cases:

• The description should contain the URL of a public repository or artifact page that contains all the instructions needed to get started with the case study.
• Optional: Case authors may include a list of 5 names of senior PhD students or junior researchers (postdocs or junior professors) who have the expertise required to evaluate submitted solutions. The case authors themselves may be part of this list. The challenge track co-chairs will consider this list when creating the SPLC challenge program committee, which will be announced by February 9, 2018.

The challenge track co-chairs will select a small, but representative set of case studies to be used for the challenge. After the description of accepted cases is finalized, a call for solutions will be released. Please note that case authors cannot submit a solution to their own case study.

SUBMISSION INFORMATION

If you have a suitable case study, we encourage you to write a short description with all the necessary details and submit this description through EasyChair (Challenge Track).  Case study proposals have a minimum length of 2 pages and a maximum length of 6 pages, including all references and figures.

The ACM styles have changed recently, and all authors should use the official 2017 ACM Master article template.

Latex users are indicated to use the “sigconf” option, so they are recommended to use the template that can be found in “sample-sigconf.tex”. In this way, the following latex code can be placed at the start of the latex document:

\documentclass[sigconf]{acmart}

\acmConference[SPLC'18]{22nd International Conference on Software Product Line}{10--14 September, 2018}{Gothenburg, Sweden}

IMPORTANT DATES (AOE TIME)

The following describes the important dates for the whole timeline of the challenge track.

• • • Case submission deadline: Friday January 12, 2018Tuesday January 16, 2018
    • Case notification: Friday January 19, 2018Tuesday January 23, 2018
    • Camera ready case descriptions deadline: Friday February 2, 2018
    • Call for solutions released: Friday February 9, 2018
    • Solution submission deadline: Friday May 25, 2018
    • Challenge notification: Friday June 15, 2018
    • Camera ready solution papers due: Monday July 2, 2018

  CHAIRS