More than a decade of RecSys
The eleventh edition of the ACM Conference on Recommender Systems (RecSys) took place earlier this year in Como, Italy. In the time since, a number of summaries of the conference have been published by some of the attendees, e.g. Neal Lathia, Ola Gustafsson, Gustavo Penha, Marco Creatura to name a few (note, I am not trying to make an extensive list of summaries here, there already is one over at the ACM Recsys Medium channel). In this post, I will not focus so much on the last RecSys (RecSys’17), as on the (just over a) decade of recommender systems conferences leading up to the week in Como.
Growing interest
Last year, the ACM Conference on Recommender Systems celebrated its tenth birthday. Keeping with the trend from the last few years, the conference attracted the largest number of participants to date (560+). This year, the conference, again, grew to a record breaking 630 participants. Apart from the attendess, other numbers continuosly grow as well, e.g. there were 23 sessions, 12 workshops, 5 tutorials, etc., etc. Most of these numbers are available on the conference website or in the image below, so I won’t iterate them all here. (If you’re looking for even more details, the frontmatter material of each year’s proceedings usually contains more information).
Thinking of the growth of the conference, I wanted to see how the research-focused parts of ACM RecSys have developed, i.e. the number of full papers presented, the number of research sessions, and how the topics covered in the papers have changed over time.
Paper numbers over time
To do so, I collected all full paper titles and research paper session (excluding panels, industry sessions, tutorial, keynotes) titles at each ACM RecSys conference since 2007 (the first one). This generated a collection of 82 session titles and 281 paper titles. The graphs below show the number of full papers, the number of sessions, and the number of full papers per session per year at every RecSys conference since the conference series began in 2007.
The numbers only include full (8+ page) research papers, even though some conference years have seen short papers presented during the same sessions as full papers, e.g. 2013, 2017, whereas other conference years have had short papers presented as posters. Still, it is interesting to see that even though the number of sessions has almost tripled, the number of (full) papers has only increased by 60%. When taking into consideration the fact that 20 short paper were presented in the main sessions in 2017, the numbers are not as surprising.
Another interesting aspect is that, until 2014, RecSys was a single track conference. From 2015 and onwards, there have been two parallel tracks through most of the program. Still, even though the parallel tracks allow for more presentations, the number of full papers presented in the years the conference has had two tracks is lower than the two years before (2012, 2013).
The general trend in RecSys over the last few years has been higher submission numbers of short papers, whereas the number of full papers has not changed as significantly. This is something that can, in turn, affect the number of full papers presented per session, i.e. since 2015 there has been a steady decline of full papers per sessions (2.36 in 2017, vs. 4 in many of the prior years). Additional changes to the program that have had effects over the years is whether or not the tuturials have been organized during the main conference (like in 2017), or whether they were held in parallel to the workshops (2014) or other parts of the program.
Significance?
So, does the the number of full papers vs. short papers have any actual significance? Looking at the nature of short papers, they are supposed to report on early stage work, where the ideas presented should be novel, thought-provoking, and need not be as extensively evaluated as full papers (at least according to the Call for papers). Full papers, on the other hand, should be mature, well-executed and implemented, and should be evaluated very meticulously. However, in recent years, short papers have come to be evaluated almost as rigorously as short papers, leading to the acceptance rate of short paper becoming lower than that of full papers (16.4% in 2014 for short vs. 20.8% for long).
So what is the significance of this then? Well, there is of course no simple answer, but one can speculate that mature work has increasingly been written in short paper format instead of long, which has led to quite mature works being submitted as short paper instead of, perhaps more fittingly, full papers (in the light of the much higher acceptance rates for short papers in the past, this could have been a way to increase the chances of getting the paper accepted to the conference). NB: these are my very personal, subjective, observations and should be valued as such.
Having said this, the ACM RecSys Steering Committee is aware of these issues and have discussed this. There are suggestions on how to move forward to “re-establish” the RecSys full paper as the most valued publication at RecSys.
Paper titles over time
Another aspect of how the conference has developed over time can be seen by looking at the topics covered by the papers presented at the conference. The topics should reflect the trends within the research topics, but perhaps also other things such as geographical locations (granted, RecSys has ventured outside of Europe & North America only once, Hong Kong in 2013, so the geographical trends might be not be a factor here), etc.
Below, are word clouds of the words contained in the titles and sessions, year-by-year from 2007 to 2017. The size of the words in the word clouds represents the number of times they appear in either the paper or session titles respectively. The session titles represent the sub-topics of papers in each given year.
Session titles
It is interesting to see the word algorithms pop up in the session titles every few years, sometimes taking precedence even over recommendation/recommender, whereas in other years it does not appear at all (2014). The significance of this is probably quite low as there usually are relatively few sessions, each with a title that sometimes needs to fit quite diverse paper topics (e.g. Emerging recommendation domains in 2011).
Paper titles
Papers titles perhaps better reflect the curent trends in the recommender systems research field than session titles. Progressions of the ket topics at RecSys can be seen by, e.g. the words rating, matrix, factorization, being phased out and new terminology such as deep (2016), convolutional, neural, recurrent, and networks making an appearance (2017). This echos the developments in machine learning, where topics related to deep learning have become very popular in recent years.
(NB: subjective opinions again:) It once used to be that RecSys could just as well have been called The Conference on Movie Rating Prediction, especially in the years during and after the Netflix Prize. It is very nice to see that the field has gotten past this and is more and more exploring other facets of recommendation. This is, as with most things, a double sided coin. In the past, you could attend any session at RecSys and find practically every talk was somehow related to your own research, this is not the case today. As the field has matured, so have the many variations and applications of recommender systems. It is refreshing to see that even as the community matures, it does not become stale.
RecSys has always been a research conference which has enjoyed high attendance from industry. In the last few years, industry/practitioner attendance has surpassed academic/research attendance. In the past years (and especially in 2017), there have been voices saying that the academic papers and presentations at RecSys are iterating (or very slightly changing) already existing works, whereas the industry papers and presentations are more novel and interesting. Perhaps the industry presentations are more interesting, and likely more digesteable in the format of a presentation. However, even though the majority of attendees come from industry, RecSys is still a research conference – and in being so it is an outlet for research-focused presentations and papers.
Summary
In summary, ACM RecSys, and the general reseach field of recommender systems have gotten quite some way since the inception of the conference. The conference has kept up with what’s going in in the field, and has grown to be the premier venue for research on recommender systems. Even though it has an academic focus, it has been attracting a large number of industry attendees (not only listening to the talks, but also presenting industry-driven research - in both the industry sessions as well as the traditional research sessions).
It’ll be interesting to see in which directions the research/practitioner community that creates the conference will take it in the future.
(Disclaimer: In the last ~5 years, I’ve been, in one way or another, involved in the conference organization. My opinions are likely biased due to this. These opinions are mine and should not be misintepreted as those of ACM RecSys.)
