After a long effort, the review process is now approaching its end. We have received 169 submissions, nearly all of which were sufficiently good that they could be considered for publication. Using a large number of volunteer reviews, we now have at between four and six reviews for all submissions. The program co-chairs and the conference chair have already begun to examine all review results, and are now working hard at making the decisions on which papers to accept, and how to place papers in sessions.
This problem is harder than it looks at first glance. We (obviously) aim to have the "optimal" conference program based on the submissions and reviews we have received. What does that mean (in a non-scientific, off-my-head, not empirically validated or researched way)?
- Exactly "the best papers" are accepted
Ideally, if the conference has space for (say) 60 papers, the 60 best papers will be selected. However, what does "best" mean? Do we simply use the average rating, or a normalized rating that ignores outliers (as done, for example, in ice skating)? Or should we check outliers personally, read the paper, and decide if the outlier should be discarded or not? How do we factor in higher or lower reviewer familiarity? Also, if the last "best papers" belong to a group with the same net rating, how do we decide which ones to take and which ones to reject with regrets?
As you can see, a one-dimensional numerical rating may not be good enough. - The conference program should ideally offer at least one session at any time slot that has something that interests each concrete attendee
Since the interest areas of attendees vary significantly, it is easy to see that this is hard (if not impossible) to achieve for "each attendee". We try to provide a program that has parallel sessions with no "obvious" overlap; for example, if there are two sessions that address "ideas for CS1/2", they will not be placed at the same time slot. However, some combined interests are difficult to predict! - Avoid parallel presentations of the same presenter in different sessions
This is normally easy to do at first glance, but actually does involve some work in cross-checking all entries scheduled in parallel to avoid that the same person or author "team" has two papers in parallel sessions. Even then, we cannot easily tell if the two papers will actually be presented by the same author... - Reduce the number of people who want to "change sessions" during a session
In principle, we support attendees who want to swap sessions during a session, for example by attending the first presentation in session A and then change for paper 2 to session B. The session chairs will strive to make this possible by scheduling papers to start at the same time slots in all sessions. Due to the unrest this causes (people rushing in and out during a session, looking for a free chair, etc.), we want to reduce the amount of "cross-session traffic" - but again, we cannot easily predict all attendees' interest areas. - Provide engaging and well-matched sessions
By this, I mean that we want to avoid "hodgepodge" session with at least one "outlier" presentation. For example, two papers present using Lego MindStorms, and the third paper discusses how one can simulate real processors in software. This should be avoided to the extent possible, as it may confuse the attendees, and may also reduce the interest in the complete session: people who are interested in one topic may not care for the other, and either leave early (or come in late), or not show up at all.
In some cases, this will also contradict - to a degree - the "best paper" criterion above: sometimes a paper which is good or very good, but not quite in the "top N" paper range, should be chosen to "flesh out" a session, instead of putting in a (potentially "better") submission that simply does not fit in. Of course, we strive to keep this to the absolute minimum possible!
Dr. Guido Rößling, March 2, 2011