peer.qmd

---
author: Neil Ernst
date: Oct 2023
title: Peer Review
format:
  revealjs:
    theme: solarized
    smaller: true
    scrollable: true
    incremental: true
    footer: "©️ Neil Ernst"
---

# Learning Objectives

* Appreciate the role peer review plays in science. 
* Learn to understand Reviewer 2.
* Grasp the intricacies of the conference review model.
* Insight into the Journal review model.
* Learn tips and tricks for reviewing efficiently.

# Why Peer Review (and what can't it do)

* Gatekeeping
* Quality checks
* The [NIPS experiment](http://blog.mrtz.org/2014/12/15/the-nips-experiment.html)

# The conference model

Bidding, discussion, rebuttal. 

Review criteria.

# The journal model

- Revision letters. 
- LatexDiff
- Letter to the editor explaining differences. Neil's example from ArchDocs.
- Polite, professional, to the point. 

# Writing Good reviews

* Focus on the paper not the author
* Provide actionable advice that can improve the paper
* Identify good and bad parts of the paper

# Review templates

Typical for my venues:

- Soundness
- Novelty
- Significance
- Verifiability
- Presentation

----
* [ICSE](https://2020.icse-conferences.org/track/icse-2020-papers?#Call-for-Papers)
* [NeurIPS](https://nips.cc/Conferences/2020/CallForPapers)
* [STOC](http://acm-stoc.org/stoc2020/STOC-2020-cfp.pdf)
* [USENIX](https://www.usenix.org/conference/usenixsecurity24/submission-policies-and-instructions)


What should the criteria be? What about other venues or different types of papers?

# Other resources:

* [RetractionWatch](https://retractionwatch.com)
* [Ramanujan](https://en.wikipedia.org/wiki/Srinivasa_Ramanujan#Mathematicians'_views_of_Ramanujan)
* [Rebuttal tips](https://andreas-zeller.info/2012/10/01/patterns-for-writing-good-rebuttals.html)

# Activity

* Read 2 papers on our HotCRP site. Enter your reviews. Engage in the discussion by posting at least 2 comments on each paper in addition to your review. Decisions are Nov 29.
* Discussion question: what do YOU want from a review? 
* Post your favourite critique (Mentimeter).

# Readings (before class)

* Neil's [preprint on peer review.](https://arxiv.org/abs/2009.01209) 

# Optional Readings

There are lot of other readings in the mandatory reading. 

- [How to Write a Rebuttal](https://andreas-zeller.info/2012/10/01/patterns-for-writing-good-rebuttals.html)
- https://violentmetaphors.com/2013/12/13/how-to-become-good-at-peer-review-a-guide-for-young-scientists/