Archived entries for General

Conference Deadlines: Is there a better way?

n the day to day life of a grad student, there are few things more important than meeting conference submission deadlines. These immovable objects tower over the academic landscape, casting their long shadows on other responsibilities and priorities, deepening as they approach. They are unavoidable and all-important: fellowships, scholarships, and applications for further study are distinguished by the student’s success in publishing at respected conferences and in journals. These venues are how we are best able to take part in and influence the intellectual discussions occurring in our field.

Despite the importance of these dates, however, the majority of us are waiting until the last minute to submit our work. This graph shows logins over time for the CHI 2011 deadline, which recently passed. This is shown on the login page of the Precision Conference submission system. From my experience, this trend is typical across other conferences and times of year.

CHI 2011: Submission system logins over time

CHI 2011: Submission system logins over time, deadline Sept. 24th

Am I misreading the data? Is the spike in logins representative of authors taking as much time as is available to polish their completed manuscripts? Anecdotal evidence and personal experience tells me this isn’t the case. What’s in fact happening is a panicked push for completion and submission, often down to the final minutes. [1]

I am guilty of this behaviour. It happens every time; my best intentions are swamped by the practical reality that writing high-quality papers with accurately represented data and thoroughly considered conclusions is extremely time-consuming. I continually underestimate the time required, which is exacerbated by the parallel challenge of effectively collaborating on these documents with multiple authors. A particularly stressful bout of this experience has prompted me to ask: is there a better way?

Considerations

After discussing this idea with several fellow researchers at a recent conference, the most common response was straightforward: the individual needs to take responsibility for their own behaviour. If you can’t acknowledge and plan for deadlines accordingly, maybe you’re in the wrong game. It is up to the individual student to map out their time to accommodate a proposal, research design, data collection and analysis, drafting, and polishing well in advance of the deadline. This cycle takes in the range of 3 months to 1 or more years to do properly, depending on the topic of study.

If the individual can do this without supervision, then they will almost certainly be successful. For the rest of us mere mortals, the overlapping responsibilities of research, teaching, coursework, and life make such a disciplined approach a bit of a pipe dream. I believe that the graph above indicates that this structured approach does not match the reality of the situation for the great majority of us.

Supervisors and research groups have a role in guiding students along this path. Undergraduate classes often build in this kind of guidance in the form of iterative submissions: an outline followed by a draft, and finally the completed work. This enables opportunities for revision and feedback, and ensures that the majority of the work is done before the final deadline is in sight. However, this kind of hand-holding disappears fairly quickly after a graduate student begins to take on their own research.

I am not laying blame. If grad students have a lot of responsibilities, it’s clear that their supervisors often have far greater demands on their time: grant-writing, supervision, teaching, and their own research, to name a few. Given this, different supervisors will be able to hold their students to a well-paced schedule to differing degrees, but it is unrealistic to expect them to micro-manage graduate research at this level. It truly is an individual responsibility, and one which each of us grapples with in his own way.

However, the apparently widespread last minute nature of conference submissions often leads to reduced clarity and increased errors in the submitted papers, and subsequent rejection. [2] It also places immense stress on one area of performance at the expense of a more balanced approach. I think in any intellectual undertaking the process is as important as the outcome, and that the current structure often reduces the quality of both. Given this, I believe the monolithic deadline system is not as obvious a solution as it seems. The question isn’t who’s to blame, but how we as a community can do better.

Alternatives

I propose two alternatives for consideration. The first is concerned with a restructuring of the emphasis which is placed on different publication venues. The second suggests that we may learn a more sane alternative to publication methods from agile approaches to software development.

Conferences vs. Journals

One senior researcher I spoke with believes our particular community (HCI specifically, along with wider applied computing fields) places far too much emphasis on conferences, with journals running second. His perspective was that conferences should be a place to present work in progress with early results, and to network and “have a chat.” In contrast, journals should be where the “real research” is published, in long form and of archival quality. Journal submissions take place over a longer time period and offer the opportunity for researchers to work alongside editors to position the work and make revisions based on their feedback. Journal articles also enjoy the input of layout and graphics professionals to assist with production.

I do not have enough experience with journal publications to comment on this very thoroughly, but when I compare my own experience publishing an article in IEEE Computer Graphics & Applications with my experience submitting to conferences, the difference is significant. The former was far more enjoyable and less stressful than the latter, and I believe the work was accordingly superior.

A few open questions:

  • How did conference publications take on such significance?
  • How does this compare to other fields?
  • Is our community best served by this emphasis?

Agile Development: Iteration and Incremental Improvement

Before beginning grad school, I worked in web development for several years. In that world, outcomes were more important than deadlines, and iterative, incremental improvements were valued more highly than large feature releases. This model of development, enabled by the network and the model of software-as-a-service, broke from traditional commercial software practices, where large feature releases were tailored to specific economically-imposed deadlines, and were inevitably followed by numerous bug patches.

If we consider research to be like software development, conference submissions are more like the old, broken way of building software. One deadline a year demands researchers corral their findings into a tightly defined format and submit on time, whether the bugs are ironed out or not. Fortunately, the review system finds and eliminates these bugs, but it doesn’t offer a way for the researcher to correct and re-submit. Iterations of the research may find their way into alternative conferences, but such opportunities are often limited. [3]

What would research modeled on agile development processes look like? It would be pragmatic, adaptive, flexible to shifting requirements, and performed in small iterative increments. I think this is actually a pretty good fit for a lot of research that happens in HCI, design and technology, where researchers are constantly asking: How can we answer these questions? What are the requirements? Does it work? Which evaluation method suits best? To my mind, however, our conference publication model doesn’t match this dynamic process, imposing a static deadline and a single filter on a diverse and unpredictable activity.

  • How would an iterative and flexible approach to research be reflected in submission and review processes?
  • What types of research is software development a good model for? Which do not fit?
  • Would a more open model of research, where the iterations are made as evident and accessible as the final product, improve our results? How would this be moderated?

Conclusions

In many ways, the current system works. The proceedings for high-level conferences are full of insightful papers that advance their respective fields, and these contributions offer a way of evaluating a given researcher’s place in the community. However, the current structure takes a significant toll on student quality of life, and limits the potential for a wider range of contributions. Each publication window missed represents a precious lost opportunity for graduate students to share their research. I believe we can do better, and that this is a conversation worth having.

Footnotes

1: It would be interesting to see the data from the submission system of last login for each submission, indicating the point at which all work was completed. My hunch is that it wouldn’t look much different.

2: The hard deadline may also have the unintended side-effect of limiting creativity and insight in published work. If you’re anything like me, the day or week after a deadline is when your most lucid insights into the problem crystallize. When you’re bent over a keyboard under the cruel tyranny of the clock, the data seems unyielding and your conception of the argument rigid.

3: This is at least in part due to the review cycle, which doesn’t indicate acceptance or rejection until months after submission. In the meantime, the author is unable to submit to alternative venues due to the requirement made by conference organizers that the work not be under simultaneous review.

Architect Magazine recognizes North House with a 2010 R&D Award


orth House has been awarded a 2010 R&D Award by Architect Magazine. These jury-selected awards are based on criteria of performance, aesthetics, and progressive thinking. Way to go Team North!

Full write-up on our research group blog: Human-Centered Systems for Sustainable Living.

Participants needed for research study

arly in September, I will be running a research study assessing the design requirements for ambient and artistic visualization of residential resource use. I am currently recruiting participants for the study. Individuals wishing to participate must be 19+. The study will take approximately 1 hour to complete, and participants will be entered into a lottery for a $300 gift certificate in recognition of their contribution.

If you or someone you know might be interested in participating, please let me know! Contact Johnny Rodgers at jgr3 at sfu dot ca.. Clicking on the advertisement below will load a PDF version of the document. Feel free to post and circulate!

AdvertisementForParticipants

Takeout sushi, minus the styrofoam

love sushi. As there are about 10 great places within a ten minute walk of our apartment in Vancouver, my fiancée and I often get takeout. However, after every delicious meal, a great wave of guilt rolls over me for the immense pile of unrecyclable trash that takeout sushi entails: styrofoam clamshells, throwaway chopsticks, plastic tubs of soy sauce, and napkins all stacked up in a hideous shrine to convenient disposability!

P1040636

After a while, this was really getting to me, so I decided to try taking my own containers to the restaurant: some Glad sandwich containers and leftover plastic tubs from the salad bar. To my surprise, they had no problem with this. I was expecting hesitancy due to some rule or another about sterilization of food containers. Instead, they gladly took my tupperware, and returned it full of sushi, miso, tofu, and rice, all neatly tied up in the canvas bag I’d brought it in.

P1040638

We’ve now been doing this for a few months, and have had the same no-questions-asked experience at three of our favourite sushi restaurants. Guilt-free tempura yam roll never tasted so good.

Motivating engagement through aesthetics and joy

have a growing hunch about the relationship between intrinsic motivation, engagement, aesthetic interest, and joy. In the past month, I’ve presented my work at several conferences, where I’ve had the opportunity to discuss my ideas about aesthetic and artistic visualization of residential resource use with diverse audiences (people in business, marketing, graphics, visualization, computer science, and human-computer interaction). My essential argument is that ambient and artistic visualization techniques reduce the attention required of residents while increasing aesthetic interest and maintaining coherence with the home. For certain coarse-grained data, they represent an engaging alternative to traditional feedback techniques. Moreover, in contrast to these traditional techniques (numbers and graphs) they seem to engage people who are otherwise uninterested in thinking about resource consumption.

Nuage Vert
Power-Aware Cord

Compare an ambient indicator of time-of-use energy billing — which could be as simple as a glowing LED — to a chart on your computer providing the same information. Which requires more attention? Which integrates into the home more unobtrusively? Which most clearly distills the complexity of the data into a manageable piece of information? This simple example helps to situate ambient and artistic visualization in the spectrum of feedback approaches for residential resource use feedback. For a more in-depth treatment, including related work upon which these ideas are based, see my proposal at the Graphics Interface poster proceedings.

Two themes have emerged from these conversations. First, the examples I’ve given of ambient and artistic visualization have drawn curiosity and spontaneous expressions of enjoyment. Nobody ooh’s and aah’s when I show the Kill-A-Watt or Google PowerMeter. However, people across audiences express delight and curiosity when I show examples of ambient and artistic feedback. From fascination with the Power-Aware Cord to wonder at the Nuage Vert green cloud project, these visualizations tend to evoke joyful expressions of interest, and a desire to know more about what they communicate.

The second theme is that these kinds of feedback seem to sidestep people’s hardwired interrogation of the data itself in order to encourage engagement with the ideas behind the visualization. As Ingrid Fetell puts it, “I think it’s because our emotions react to aesthetics before they process content. Even when the aesthetics and content are dissonant, the aesthetics guide our reactions, I guess because in most circumstances, aesthetics are an accurate shortcut to understanding content.” Unfortunately, lots of misleading infographics and unnecessarily prettified visualizations take advantage of this shortcut. However, when aesthetic appeal is considered as an essential element of information visualization, as opposed to visual icing, the results will be more engaging and can promote emotional engagement with the data more readily than traditional techniques. For a subject as personal as the relationship between resource use and behaviour, triggering an emotional response may be the best way to promote awareness and self-reflection, and create the space needed for change. This is specifically the case when operating within the artistic/ambient/casual InfoVis paradigm that I’ve been exploring.

The first major prototype I’ve built in this area is the Ambient Canvas, which was deployed in West House. This display uses strings of LEDs mounted behind the kitchen backsplash to convey feedback on residential resource use through shifting patterns and varying intensities of light. It is not designed for precise feedback on resource use data, but rather to increase awareness of resource flows at a more generalized level: not 13.4 kWh of energy used, but 2/3 of your daily average.

Ambient Canvas

The Ambient Canvas in West House. I collaborated with Rob MacKenzie and Chris Brandson to design and build the display, with supervision from Dr. Lyn Bartram.

Ambient Canvas Example Configuration

The Ambient Canvas can be configured to provide feedback on energy use. First, a baseline of typical use is established using collected sensor data. Then, as residents go about their daily activities, the LED strings light up and fill the Canvas to indicate cumulative use against the baseline. The intent is to enable residents to gain awareness of their energy use and adjust their activities over time, continually ‘competing’ against their own self-adjusting baseline of use.

I’ve been asked what design guidelines I’ve used when prototyping these kinds of visualizations, and how I’ve “operationalized my aesthetic criteria.” That, of course, is a tricky question to answer. To the first, I emphasized the use of colour, motion, and light. These seem to be a common and effective element in expressing data through ambient and artistic means. To the second, I believe that’s another thesis topic in its own right! However, this question has prompted me to look at my own aesthetic assumptions more critically. Intuition plays a great role in art and design, and certain people to seem to have a natural aesthetic sensibility to draw upon when approaching creative problems. Nevertheless, attempting to isolate and explore some of the variables in this aesthetic toolkit is a worthwhile endeavour. Ingrid Fetell is attempting to do just this with her project, Aesthetics of Joy:

Aesthetics of Joy explores the intersection between design and positive emotion. The project draws on insights from neuroscience and evolutionary psychology to explain the universal, hardwired triggers of joy, and suggest ways that designers can evoke these in their work. These “aesthetics of joy” can be applied to the design of objects, spaces, and experiences to enhance our emotional health and well-being and create more moments of spontaneous delight in the world.

As is obvious if you’ve read this far, Fetell’s efforts are significantly influencing my conception of these ideas. She further explores this concept as it relates to behaviour change, with reference to the psychological relationship between our rational and emotional minds:

“Lately I’ve been reading the book Switch, by Chip and Dan Heath, about the psychology of behavior change. In the book, the authors reference a construct developed by psychologist Jonathan Haidt to explain the way our rational and emotional sides deal with each other. The emotional brain is like an elephant while the rational brain sits on top like its rider. The rider (our rational side) provides direction, while the elephant (emotion) provides the motivation and force that gets us to act. The rider looks like the boss, sitting up on top of the elephant, reins in hand. But the elephant is so massive that unless it goes along voluntarily, it’s hard for the rider to get his way.”

Pragmatic information visualization techniques clearly target the rider in this metaphor. This is appropriate and effective for the tasks which pragmatic InfoVis is good at. But when attempting to change people’s minds about how they use resources in their own home, the rider isn’t the only one who needs convincing. We need to persuade the elephant.

So my hunch boils down to this. Beautiful, engaging, aesthetically informed visualization of resource use in the home taps into positive emotional responses, lending intrinsic motivation (in the form of curiosity and delight) to residents’ efforts to change their behaviour. This approach avoids several obstacles to traditional feedback mechanisms, which can be time-consuming, intimidating to those who do not access the world numerically, and depressing to those who do not wish to think about resource consumption. Ambient and artistic visualization can speak to something joyful rather than something mundane, and can therefore open a communication pathway to an underserved audience for residential resource use feedback.


© All material copyright of the HCSSL Research Group and its respective authors, 2009-10.
This site is proudly powered by Wordpress and uses a modified version of the Modern Clix theme.
Daily Dropcaps by Jessica Hische licensed under CC BY-NC-ND 3.0.

Simon Fraser University School of Interactive Arts & Technology