AI Tech Inspire

arXiv vs Double‑Blind at ICLR: What Really Happens and How to Navigate It

Seen a paper on arXiv weeks before it shows up as an anonymous ICLR submission? You’re not imagining things. The modern ML publishing stack blends public preprints with double-blind review, and that mix can feel contradictory. At AI Tech Inspire, this recurring question keeps surfacing because it speaks to a real friction point: how do you establish priority fast without undermining anonymity?

What sparked the question? Key facts

  • An early-career researcher noticed multiple ICLR submissions that were already visible on arXiv.
  • They wondered whether prepublishing breaks ICLR’s double-blind review.
  • There’s a tension between fast priority via preprints and maintaining valid double-blind submissions.
  • To avoid issues, the researcher has not prepublished, but worries this could hurt novelty claims or risk desk rejection.
  • It’s easy to find a preprint by Googling a title or abstract, which feels at odds with the spirit of anonymity.
  • Overall sentiment: the situation is frustrating and confusing.

The short version: double-blind is a best-effort system

In today’s ML world, double-blind review is best-effort anonymity, not total secrecy. Top conferences, including ICLR, typically allow authors to post non-anonymous preprints (e.g., on arXiv) while still requiring double-blind submissions for review. Reviewers are instructed not to actively search for author identities, and authors are expected to keep the submitted manuscript anonymized (no acknowledgments, no self-identifying phrases, careful self-citations). In parallel, the community accepts that some reviewers may still infer authorship based on prior talks, code repos, or recognizable writing style.

ICLR also runs reviews on OpenReview, where submissions are public and anonymous during review. That public visibility raises the stakes for how authors handle communication, but it doesn’t prohibit arXiv preprints. The underlying idea: fast, open exchange of ideas can coexist with a process that tries to minimize bias.

Key takeaway: Posting to arXiv is generally allowed; it doesn’t automatically violate double-blind. The policy goal is to reduce, not eliminate, identity signals.

Does prepublishing actually break anonymity?

Not by default. Seeing a similar work on arXiv doesn’t automatically “break” double-blind. Double-blind aims to limit bias, not to guarantee that identities are unknowable. Reviewers are asked to avoid searching for identities; if they accidentally know, they’re expected to proceed and judge the work on its merits.

What about desk rejection fears? Under typical conference guidance, having a preprint is not a reason for desk rejection. Desk rejections usually relate to format violations, scope, or severe policy breaches (e.g., clear de-anonymization within the submission). Always check the current Call for Papers, but in recent ICLR cycles, non-anonymous arXiv preprints have been explicitly permitted.

Why preprint anyway? The incentives are real

  • Priority and timestamping: arXiv provides a clear clock for novelty claims.
  • Community feedback: Early visibility invites comments, replication attempts, and bug spotting.
  • Recruiting and visibility: Labs and practitioners track arXiv to identify promising directions.
  • Parallel development: Rapid fields often see near-simultaneous ideas. Public preprints make credit and independence easier to track.

For developers and engineers, preprints are a superpower: you can start experimenting earlier, stress-test claims on your own stack (say, PyTorch or TensorFlow), and assess feasibility with your infrastructure (e.g., CUDA capabilities or deployment constraints). You don’t have to wait for camera-ready versions to start building.

What about the Googleable title problem?

Yes, if you paste the exact title or distinctive abstract phrases into a search engine, you’ll probably land on the preprint. The double-blind norm asks reviewers not to do that. In practice, reviewers are also community members—they may have already seen the preprint on social feeds, newsletters, or lab talks. It’s imperfect, but the field leans on the professional norm: judge the submission, not the authors.

For authors, the actionable part is straightforward: keep the submission cleanly anonymized and avoid unnecessary promotion that calls attention to authorship during the review window. For reviewers, the discipline is equally simple: don’t search, and if you already know, evaluate the work fairly.

A practical playbook for authors

  • Check the latest policy: Before you submit, re-read the current ICLR instructions on anonymity, preprints, and social media. Policies evolve.
  • Keep the submission strictly anonymized: Remove acknowledgments, institution logos, and self-revealing text. Use neutral citations for prior work, including your own (Author et al., 202X without signaling “this is us”).
  • Time your arXiv post intentionally: Many groups post right before or at submission time. If you prefer, delay the preprint until after submission—but remember that others may post concurrent work in the meantime.
  • Minimize publicity during review: Avoid broad promotion (e.g., thread-per-day social blasts). Quietly hosting a code repo is different from a marketing push.
  • Separate code artifacts from identity where possible: If you must share code during review, consider a neutral org or a private link. After decisions, you can migrate to your lab’s main account.
  • Document concurrent work carefully: If a very similar arXiv appears during review, acknowledge it factually in the camera-ready. That’s normal in fast-moving areas (think Hugging Face model hubs or Stable Diffusion-style diffusion variants where ideas propagate quickly).

One thing not to do: don’t play title games to evade search. Changing titles to hide identity is counterproductive and can confuse readers post-acceptance. Clarity > gamesmanship.

For reviewers and practitioners

  • Resist the search reflex: If you’re reviewing, don’t Ctrl + F unique phrases on the open web. Evaluate the content you received.
  • Focus on claims and evidence: Can you reproduce the central result with a quick PyTorch or TensorFlow script? Is the compute profile reasonable for your CUDA-class GPUs? Would this integrate into your serving stack or an GPT-powered pipeline?
  • Track delta over baselines: Novelty often shows up as careful ablations and comparisons—not just a new name for a known pattern.

Why this matters to engineers

For builders, arXiv isn’t just academic decor. It’s a roadmap of near-future tools. Preprints often drop reference code, datasets, or model checkpoints. That means:

  • You can prototype features months earlier and benchmark against your workloads.
  • You can estimate deployment costs sooner (memory footprints, inference latency, failure modes).
  • You can make better buy vs. build decisions before competitors catch up.

At AI Tech Inspire, we’ve seen teams turn preprint insights into internal demos within days, not months. If a method is likely to land at ICLR, it’s probably worth a weekend spike to gauge ROI.

Open questions the community is still debating

  • Stricter norms or more openness? Should conferences lean into fully open identities (no double-blind) or enforce tighter publicity rules during review?
  • Artifact-first cultures: Would standardized artifact evaluation or masked repositories reduce bias without slowing dissemination?
  • Infrastructure support: Could platforms like arXiv or OpenReview offer “masked mode” linking artifacts to submissions without revealing authors until decision?

None of these trade-offs are easy. The speed of ML pushes toward openness; the human tendency toward bias pulls toward anonymity. The current compromise is messy but workable when participants respect the spirit of the rules.

Bottom line

Prepublishing on arXiv and double-blind review at ICLR coexist through norms: authors anonymize submissions and avoid overt promotion during review; reviewers avoid identity searches and judge the work. Yes, you’ll sometimes recognize a paper. No, that alone doesn’t invalidate the process. For researchers, choose a preprint timing strategy that fits your risk tolerance. For engineers, use preprints as an early signal and a playground for rapid validation. The friction is real—but so are the benefits of faster, broader idea flow.

Recommended Resources

As an Amazon Associate, I earn from qualifying purchases.