Guide for Reviewers

Reviewer Process And Guidelines For Quality Reviews

7/8/11

A. Receive Invitation To Review

You will receive an invitation to review a manuscript by email. This will be addressed to the email address we have on file for you. An invitation to review will always come from "Ophthalmology Editorial Office" with the subject as "Reviewer Invitation for Manuscript XXXXX" with the XXXXX being the assigned manuscript number. The email will always contain your username and password, title page and abstract of submission and links to accept or decline the invitation to review. Occasionally the links malfunction and you may need to log on and accept or decline as outlined below.

If you do not have time to do a review, please turn down the invitation quickly so we can invite another reviewer. It is unfair to authors if we drag out the process and wait a month for your review only to learn that with apologies, you can’t do it. Please make a prompt decision to accept or turn down the invitation and live up as best you can to your decision. If you do not respond to the email within 5 days, you will be "uninvited" automatically so that we may keep the length of time it takes to get reviews to a minimum

Important Note

It is your responsibility to keep your email address as well as other contact information current. If you need to change an email address and/or institution and address log in through your current username and password, change your contact information via the “change details” link and then select "update" before closing your screen.

If requesting your username and password, be sure to use the same email address every time. If you are confident that you are a registered user and are unable to retrieve your username and password, do not register again; contact the editorial office at 443-287-2445. Otherwise there will be multiple entries for your name which will not allow you to track your manuscripts and reviews.

B. Log In And Respond

At http://ees.elsevier.com/ophtha click on "Log in" and fill in the User Name and Password as provided in your invitation letter. Then click "Reviewer Log In", which will bring you to your Reviewer Main Menu. If you don't click "Reviewer Log In", the default log in is as an author and you will not be able to access the information you need to respond or complete the review.

On the Main Menu for Reviewers you will see three categories:

  • New Reviewer Invitations - these have not yet been viewed or responded to by you
  • Pending Assignments - these you agreed to review, but have not completed
  • Completed Assignments - these you have completed, and submitted your review back to the Editorial Office. You can view completed assignments indefinitely.

Click on "New Reviewer Invitations" on the left hand side and you will see a box that reads:

View Abstract/Title Page
Agree to Review
Decline to Review

You should review the abstract and title page for conflict of interest as well as whether it is within your expertise. You then click on "Agree" or "Decline". If you decline, you are done and will not be given access to the manuscript.

C. Access Manuscript

If you agree to review, you generate another automatic “thank you” email that gives you access to the full manuscript file for review purposes. This email advises you of the actual due date (generally 14 days from your date of acceptance) and your user name and password. If you logged out of the system and come back, you should again go to http://ees.elsevier.com/ophtha, log in as a reviewer as before, but now you will see that the given manuscript has left the "New Reviewer Invitations" and has dropped down into your “Pending Assignments”. If you are still in the EES system, you will see an immediate link to your Pending Assignments.

D. Submitting Your Review

After you read the manuscript, you must go to http://ees.elsevier.com/ophtha/, log in as a reviewer and find the manuscript in your pending manuscripts. Click on "Submit Recommendation" and the review form will open.

You have three options while entering your review into the electronic system. If you are in the middle of writing the review and wish to exit and come back later, you can click the "Save and Submit Later" button which will save whatever you have done in your Pending Assignments. If you finish and want a paper record you can click on proof and print. When you are finished and wish to submit just click on "Proceed". This will give you a final look at what you've done and let you either edit or proceed to submitting it to the Editorial Office. Be sure to fill in information for at least the first four items shown below, confidential comments to the editor are appreciated, but not mandatory.

Across from that you are asked to insert a manuscript rating, based on the following.

1. Recommendation — At the top you are asked to give your recommendation with the following choices available in a drop down menu:

Accept as is, Accept minor changes, Moderate Revision, Extensive Revision or Reject.

2. Manuscript Rating — Across from the recommendation, you are asked to insert a manuscript rating of the overall quality of a manuscript, based on the following:

The review form includes a location for the rating of the manuscript by the reviewer. Ophthalmology now uses ratings from (1–100, with 100 as the highest rating). Below are "guidelines" for rating manuscripts, such ratings are obviously subjective and imprecise. Reviewers should exercise their own judgment as to what issue or issues justify their final recommendation. Revision may move a manuscript up the scale and greatly improve its likelihood for acceptance.

80–100 = Excellent Manuscript — decision is usually an accept as is or accept with minor changes — very few manuscripts will meet this standard on first submission.

Good science, follows scientific method, no major flaws, exciting new information (breakthrough), timely, clinically relevant, well-planned, data clearly presented, logical analysis, appropriate statistics, discussion and conclusions flow logically from data, good grammar and spelling, format appropriate for Journal, follows instructions, excellent readability, no conflict of interest or ethical problems, title and length appropriate, citations comprehensive but not excessive, tag as "lead" article, may justify an accompanying editorial.

60–79 = Above average manuscript — many accepted papers are in this category, decision is usually accept with minor changes or moderate revision.

General qualities typical of most papers in Ophthalmology, usually only minor to moderate revisions required, re-review usually not necessary, no major flaws in concept or logic, no ethical or conflict of interest problems, adds some new information with clinical relevance, readable, clinically relevant, appropriate analysis, or minor flaws correctable without major effort, language good, formatting appropriate or easily fixed, citations reasonable.

40–59 = Average manuscript — many of these manuscripts could move on to be accepted or can be rejected depending on space or priority.

Good science but not exciting, not new, issues already well described in literature, moderate or extensive revisions required, re-review may be necessary, average language and readability, often too long, appropriate analysis, citations reasonable.

20–39 = Below average manuscript — most likely to be rejected, some high priority topics survive one or more revisions to publication but are subject to space considerations.

Revision unlikely to lead to acceptance, major revisions recommended, re-review necessary, flawed study organization, data collection, methods, analysis inadequate or inappropriate, low clinical relevance, disconnect between data and conclusions, no new information, poor language, incomplete or excessive citations.

1–19 = Unacceptable manuscript — eject Hopeless, flawed beyond repair, Not appropriate for Journal No new information, poorly conceived, data inadequate, analysis flawed, language unacceptable, no justification for additional peer-review effort, citations inadequate, inappropriate.

3. Comments for Authors— There is a box for you to type in or cut and paste your comments for the authors.

4. Priority Rating — This is within the box for Confidential Comments to the Editor. You are asked to fill in a priority rating which is your estimation of the "average" reader's interest in the topic of the manuscript reviewed. Many excellent papers may have limited appeal to the average reader and may be more appropriate for alternative, specialty journals. The "average reader" of Ophthalmology is assumed to be board qualified or certified ophthalmologist in private or group practice doing comprehensive vision care.

1: highest priority, 2: high priority, 3: midline, 4: low priority and 5: lowest priority.

5. Confidential Comments to the Editor— On the bottom is another box for comments to the Editor. These comments are seen only by the Editorial Office and Editor and do not go to authors or other reviewers.

E. After Your Review

You will receive an acknowledgment of receipt and thank you from the Editorial Office which will remind you to complete the CME Credit for Reviewers Form. You can complete this form and forward it to the Editorial Office by email at vdoyle@jhmi.edu. Further information regarding CME credit for reviewing is below.

You will also be able to see the other reviewer's comments to the author(s) and the final decision by logging in and going to completed reviews.

F. CME Credit

Those who serve as reviewers of manuscripts for the American Academy of Ophthalmology journal Ophthalmology may be claim up to three category 1 CME credits per review, up to a maximum of five reviews or 15 CME credits per calendar year. These credits are given only for the original review (not reviews of revisions) and only for manuscripts. The thank you for reviewing letter which you will receive once your review is in the system, will also provide you with the link to the CME form which you can then forward it to the Editorial Office by email (vdoyle@jhmi.edu) or fax it to 443-287-2448. There it will be reviewed and then forwarded to the American Academy of Ophthalmology, for inclusion with other CME credits the AAO tracks for members. If you are not a member of AAO, a certificate will be sent to you.

Those who serve as reviewers for the journal, Ophthalmology, may claim up to three AMA PRA category 1 CME credits™ per review, up to a maximum of five reviews or 15 CME credits per calendar year. CME credit may be earned for reviews of original contributions to the medical literature that require multiple reviewers, and which are at a depth and scope that require an in-depth knowledge of both the literature and the evidence base. The Editor of Ophthalmology will determine which articles qualify, and which reviews are of sufficient quality (see Confidential Reviewer Ratings below as well as Guidelines for Quality Reviews) and timeliness to be eligible for credit.

These credits are given only for the original review (not re-reviews or reviews of correspondence letters or replies) s) and only for manuscripts. The “thank you for reviewing” email which is sent once a review is completed will also provide you with the link to the CME form which you can then forward it to the Editorial Office by email (vdoyle@jhmi.edu) or fax it to 717-384-8436. There it will be reviewed and then forwarded to the American Academy of Ophthalmology, for inclusion with other CME credits the AAO tracks for members. If you are not a member of AAO, a certificate will be sent to you.

Confidential Reviewer Rating by Editor — The reviewer pool currently includes over 4000 individuals. Review ratings (1–100) for previously assigned manuscripts help the editors to assign new manuscripts.

80–100 = Valuable new insight, appropriately objective and detailed — Offers fair, constructive criticisms that benefit the author and the Journal, involved significant personal knowledge or effort to authenticate data, may have sought additional input from colleagues regarding data analysis for expertise outside his/her own field, well written without inappropriate remarks, attention to all sections of the manuscript, no conflict of interest, remarks and recommendations consistent, follows instructions, met deadlines.

60–79 = Very useful review with some new insight — Fair and constructive with benefit to author and Journal, clear opinion as to whether to accept/revise/reject, remarks and recommendations consistent, no inappropriate remarks or conflict of interest evident, appropriate attention to entire manuscript, follows instructions, met deadlines.

40–59 = Brief but useful comments — Fair but minimally insightful or constructive criticisms that might improve the manuscript, consistency between remarks and recommendations, no conflict of interest, Followed instructions, met deadlines.

20–39 = Abbreviated or hypercritical comment of marginal use — REVIEW INSUFFICIENT FOR CME CREDIT — Wandering, non-focused discussion with no constructive criticisms, inconsistencies between remarks and recommendations, late return of review or incomplete reviewer sheet.

1–19 = Useless review: Inappropriate remarks, neither in-depth nor adequate - REVIEW INSUFFICIENT FOR CME CREDIT — No constructive criticism, not clear that reviewer understood the topic, no attempt to acquire help with topics where expertise lacking (may have over-extended own expertise), obvious conflict of interest, Hypercritical or unfair, return excessively late, so as to be inconsequential to review process.

Guidelines For Quality Reviews

We understand that there are tremendous demands on your time.

If a paper is clearly flawed, feel free to be brief, offer a few constructive comments and criticisms and stop. I recommend you reserve more of your time for better papers that you plan to recommend for revision and possible acceptance.

In your confidential comments, please advise me if the readers will enjoy the paper. Will it have broad or narrow appeal? Is the paper too technical or specialized? You are probably an expert in the topic. Authors tend to write with their expert colleagues in mind. However, the readers are 50% general or comprehensive ophthalmologists and the remaining 50% are divided among about 10 specialties e.g., retina, cornea, etc. So, a paper on a specialty related topic will be read maybe by 5% same specialty experts; the remaining readers need to be attracted to the paper by an interesting and clear abstract, brevity, and a clear and simple clinical message. Thank you for helping to point this out to authors.

Here are some hints on things to focus on when doing your review.

Be Polite

Most papers are not going to be accepted and authors will not be pleased with the decision letters. At the least, we must be polite with our comments and offer succinct and constructive advice and criticism. When you can, embed a compliment or nice comment. Where you criticize, consider how you would feel reading the criticism as an author. Thank you.

Allow Initially For Language Issues

Many authors are writing in a second or even third language. The writing must be excellent when the paper is finalized, but we can review papers with poor writing as long as the science is understandable. Please be sure to mention in confidential comments to the editors if writing assistance is needed. On a case by case basis, if you have time, in your comments to authors, offer suggestions for improved writing. If you do not have time, simply make a polite comment that writing assistance will be needed. Importantly, since we are going to criticize the writing, please do your very best to write well, succinctly, clearly and correctly. It is awkward for us to share poorly written comments that criticize the writing.

The Science

Please consider issues such as:

  • Is the study design clear? Is there a clearly stated a priori hypothesis? Is there a clearly stated primary outcome? Is the outcome a good one – for example, is it clinically relevant to patients? For surrogate outcomes such as fewer bacterial counts in a conjunctival swab as a predictor of post op endophthalmitis, is there reliable data linking the surrogate outcome to a clinically relevant one?
  • Be wary of composite outcomes. You might look at “Composite End Points in Randomized Trials. There Is No Free Lunch.” George Tomlinson, PhD; Allan S. Detsky, MD, PhD. JAMA. 2010;303(3):267-268.
  • Is the sample size adequate? For negative studies, are the authors powered to rule out a clinically relevant difference if one in fact exists?
  • Although we would love to focus on large randomized clinical trial manuscripts about important eye diseases, there are not enough of these to fill each issue. I expect to see a large number of case series manuscripts. Here are some general comments about case series papers to consider:
  • Prospective is better than retrospective. Larger sample sizes are better than smaller. Longer follow up is better than shorter.
  • For all papers, it is important to explain inclusion and exclusion criteria. Readers will want to apply the results and recommendations to their patients. We need to know who was in the study (eligible) and not in it (not eligible or excluded) to understand if the patient we are seeing now might have been eligible to be in the study. If they are, the study results, if valid, may apply. If not, there is less chance the results will reliably apply.
  • Is there good completeness of follow up? For patients lost to follow up, are baseline case mix features similar to those reported on?
  • Common errors in case series reporting include the use of “final” outcomes or the last follow up data. This is prone to bias since patients followed longer tend to be different from those not followed. When possible, authors should report outcomes at set time points such as one, two, or five years. Learn from the paper by Jabs: Improving the Reporting of Clinical Case Series , 21 March 2005 . Douglas A. Jabs. American Journal of Ophthalmology. May 2005 (Vol. 139, Issue 5, Pages 900-905)
  • Watch for regression to the mean.
  • Is there IRB approval? Are there other ethical or regulatory issues? Conflict of interest issues?
  • Is there appropriate use of words e.g., “safe?” To detect an unexpected serious adverse event that occurs 1% of the time a sample size of 300 is needed. A sample size of 100 can find 3% rates. Consider an editorial in this journal: Safe and effective. Andrew P Schachat, Wiley A Chambers, Thomas J Liesegang, Daniel A Albert. Ophthalmology November 2003 (Vol. 110, Issue 11, Pages 2073-2074).
  • Are claims overstated? Is there marketing or “hype” embedded in the text? The data should be clearly spelled out but it is best for the readers to interpret it without the benefit of embedded “spin” from the authors.
  • Is the content in correct sections of the manuscript? So, are discussion comments in the results section or are methods and results comments mixed up?
  • For experimental studies, is the material understandable to the non-scientist readers? Is there adequate detail in the methods section that would allow someone skilled in the field to replicate the work?
  • Is there judicious use of space (tables and figures)? Tables and figures take a lot of space. Are they as clear as they can be? Are all needed? Material should not be duplicated. If the authors give data in a table, it need not be reiterated in the text or vice versa.
  • References should include pertinent material and need not be encyclopedic. In general, our papers are not review articles and it is not necessary to list every paper on the subject. Did the authors select the appropriate material to cite? Note that when the authors are claiming priority such as “the first case of …” it is not adequate simply to say “we did a pub med search…” More detail on the depth and breadth of the literature review strategy that was employed should be included.
  • Reviewing papers is time consuming so we thank you for your efforts and advice. Good reviews help us improve papers and in turn, allow us to provide better information to the readers and eventually, improve patient care and outcomes. Thank you in advance for your help and advice.