Tasting My Proverbial Academic Foot

01 Dec 2011

tl;dr: Researchers I really respect read a rambling, ranting, accidentally-public review of their paper I wrote for class and then blogged about it.

Apparently private-but-really-public memos are all the rage this week, so I shouldn't have been surprised when I received an interesting tweet from Mike Freedman last night.  Mike informed me that Dave Andersen had posted a reply to a paper review I'd recently posted on a private blog as homework for Ion Stoica's cloud computing seminar I'm enrolled in at Berkeley.  This was surprising in part because I wasn't aware of any public links to my personal review blog (having disabled Google indexing and public references) except for the link apparently buried on the course site. Apparently, my roommate mentioned our class to Mike, his old advisor, and I'd recently de-anonymized myself when tweaking my Blogger profile for this blog. However, the tweet was even more surprising because Dave and Mike were co-authors on the paper that Dave blogged about, which I had reviewed unnecessarily harshly.  My review of this paper (describing a system called COPS that provides a variant of causal consistency called causal+ consistency) was definitely not intended for public consumption--and certainly not by the authors, who I genuinely greatly respect.

To comment directly on Dave's post, the substantive critical portions of my review stemmed from what I incorrectly perceived as a claimed major contribution of the paper: the development of causal+ consistency, which has existed in various flavors in related work but wasn't previously formally defined.  As Dave clarifies in his post, defining causal+ is a contribution, but the paper's main contribution is the design of a causal+ system, including technical details like garbage collection and making it run fast.  This is an interesting distinction that I'm slowly learning about between the systems community, with which I am more familiar, and the database community.  To paraphrase Eric Brewer in CS262 lecture, traditionally, the database community focuses on providing useful invariants, then building systems that obey those invariants.  On the other hand, the OS/systems community builds from the ground up, providing useful abstraction and modularity along the way.  Brewer and Joe Hellerstein make a rough categorization between OS/systems research ("elegance of system") and database research ("elegance of semantics") styles along these lines.  COPS is a huge win in terms of system elegance compared to prior work--it screams, it's scalable, and it's practical--even if the semantics which derailed my brief review have been previously explored (if not formally stated).  As I determine a field for my dissertation, it's important that I continue aligning and calibrating my rapidly-evolving tastes with the values of the research community I will join.

Dave quotes me mercifully in his post, but I had some unnecessarily strong words in my "(surprisingly angry)" review that I wouldn't stand by publicly or even privately.  In my (would-be) private class reviews and discussions--often written quickly, in the eleventh hour--I'm typically pretty harsh and at times deliberately inflammatory. This helps me get feedback from my advisors and peers about the uncertainties and opinions I have about research in addition to encouraging debate--in private. I'm an admirer of the philosophy that one should put a stake firmly in the ground and see what's there, but be open and ready to move the stake just as easily. This is one of the great parts of academia: ideas are currency, yet judgments are rarely irreversible, so making judgments about ideas and rethinking them is both fun and encouraged!  In the context of a write-once-read-never paper review between me and my instructor, I lean towards making over-the-top statements and often voice my doubts about our field and what to expect from my own research. Additionally, in full disclosure, the fact that I'm working on similar data consistency issues to their paper probably didn't help this particular review (although, to be fair, we're looking at how far we can push existing models--but more on that soon).  I'm not the most socially adept person I know, but I definitely wouldn't have knowingly publicly published this review.  You can imagine my horror when I learned the COPS team had read it.

Public content on the internet is serious business.  There are too many examples of public content being posted that really shouldn't be posted (even if it's a conscious decision: in CS, Mihai Patrascu's job offer debacle comes shudderingly to mind).  In the internet era, the "newspaper rule" of not doing anything you wouldn't want on the front page of the New York Times applies to all content you post; a hyperlink can spread virally.  Moreover, the internet (more or less) lasts forever, so there's no easy "undo" or "delete" operations (unless we eventually have some combination of ubiquitous attestations/TPMs, DIFC, and leases).  I personally make a concerted effort to control what content I release publicly, and this error is not one I'll forget. I'm well-aware that security through obscurity isn't security, but this experience truly brings the lesson home.

Ironically, this semester's classes ended yesterday, and I was planning to shut down the blog shortly thereafter.  After this incident, I considered keeping the post publicly readable, but Dave has picked out the salient points and re-posted what little is worth reposting (in addition to answering my questions!).  I'm a big proponent of publicly baring weaknesses in one's own research and have recently thought hard about posting the blind paper reviews that my work receives and independently providing lists of deficiencies for each publication I write.  However, because this review wasn't intended to be public (and, of course, for the sake of my academic career), I've decided to take it down. (Dave has graciously agreed to remove the copy of my post that he uploaded after I removed my blog.)

After a panicked late-night phone call with my advisor, a powwow with two wonderful, anonymous late-night inhabitants of Soda Hall, and some sleep, I still feel embarrassed, but I think I've learned a few good lessons here.  I trust that Dave, Mike, and the COPS team won't hold this against me in five years if I make a trip around the academic job circuit.  And, secretly, I look forward to the day when naïve first-year Ph.D. students write ridiculous commentaries on my work, especially if they're as bombastic I was in my review.
You can follow me on Twitter here.