Reviewers should preserve their anonymity. The Conference Management System in which the authors uploaded their contribution (Conftool) already ensures that reviews are blind. The authors should avoid providing any hint about who they are: names, affiliations, reference to papers and projects, acknowledgement should be omitted.
Reviewers have the responsibility to keep the submissions confidential, including papers rejected from the event it has been submitted to. If reviewers are assigned a paper with a possible conflict of interest, they should return the paper immediately to their Area Chair or Technical Commission President.
Details about the review procedure and the ISPRS policy on preprints in public repositories are available at the following link: http://www.isprs.org/documents/orangebook/app4.aspx.
Citations to these papers are not required and such reference omission is not ground for rejection.
In addition, especially with the increase in popularity of open preprints, the authors of a paper may be known to the reviewer. Posting to arXiv is not considered as a violation of anonymity. If they know or guess who the authors are, experts can still review the paper unless they consider they cannot perform a unbiased evaluation.
An important general principle is to treat papers fairly whether or not you know (or guess) who wrote them. The reviewers should not try to find who the authors are by searching the Web for preprints.
Reviews are available to other reviewers, AC, PC, and the authors of the papers.
If the reviewer thinks a paper may be a dual submission or suspected of plagiarism, please contact the Program Chairs.
The evaluation sheet is composed on two main parts: (1) an overall recommendation score (50% of the total), and (2) a list of criteria (50% of the total), for which each reviewer should provide a score and a detailed evaluation (see below). The advantage of such a list is that all criteria are of similar importance. They are weighted equally so as to provide a final mark that is consistent whatever the scope of the paper (method-driven or application-driven).
The overall recommendation (and the associated confidence score) and the technical evaluation of all the reviewers are taken into account so as to first decide for acceptance/rejection, and then, for accepted papers, for the selection between long oral, short oral or interactive presentation.
Five main questions compose the review form. The criteria slightly differ depending whether a full paper or an abstract is submitted.
o Full paper: Does the paper contain original/innovative work (on theory and/or methods and/or applications)? Has related work been properly credited?
o Abstract: Does the abstract indicate original/innovative work (on theory and/or methods and/or applications)? Is there any mention of existing approaches?
- Scientific formulation:
o Full paper: Are the hypothesis clearly stated? Are the research questions correctly formulated? Are the employed methods appropriate? Are equations / methodological framework correctly presented / justified?
o Abstract: Is the abstract informative? Does it contain information about the hypotheses, the research questions, the methods employed and their relevance?
- Experiments and validation:
o Full paper: Do the authors properly reflect on the achieved performance of the proposed methods or procedures? Are facts and interpretation duly separated? Are results assessed in a qualitative and quantitative way? Are the interpretations and conclusions justified by the data?
o Abstract: Do the authors properly reflect on the expected performance of the proposed methods or procedures?
- Relevance: Does the subject fit within the call for papers? Do the results have a practical relevance? Will results impact other research?
o Full paper: Is the paper well written and well structured? Are figures well readable? Are the references complete? Is the paper formatted according to the author guidelines?
o Abstract: Is the abstract well written? Do the authors provide some references and an illustration of their work? Is the abstract formatted according to the author guidelines?
In addition to rating the papers according to the five criteria, reviewers are asked to explain their evaluation in a detailed and clear manner and justify the above scores. The comments should be specific since discussions are more helpful than scores, both for authors and for the final decision. We recommend pointing out strengths and weaknesses of the submitted contribution. In addition, reviewers should provide suggestions for improvement for the camera-ready paper and use an objective and constructive writing style. References to significant missing papers should be provided, excluding as much as possible papers from reviewers’ group and their close collaborators.
Minor comments should also be provided (typos, erroneous sentences, additional references to be added, problems with the paper format) since authors are bound not to proof read their paper before sending their camera-ready version (due to limited time frame). Minor flaws that can be easily corrected should not be a reason to reject a paper.
Innovation and stimulation are core elements in an ISPRS event. Acceptance or rejection decisions should not be advocated solely by the performance of the proposed methods. It is not mandatory for the authors to propose methods that exceed the accuracies of the state-of-the-art or that have been tested on specific benchmark datasets. Nevertheless, papers should be technically correct and should propose a contribution to their field.
The reviewer has the responsibility to protect the confidentiality of the ideas exhibited in the papers he/she reviews. The work is considered new or proprietary by the authors. The reviewer is not allowed to show the paper, any results, code or supplementary material to anyone else, unless he/she has asked them to write a review, or to help with the review. The reviewer should not use ideas/code to develop his/her own ideas/code.
Once accepted, abstracts will not be reviewed again. Therefore, the degree of maturity of the work should be clearly assessed in the reviewer score sheet. This is absolutely necessary so as to build a consistent program, especially if some abstract-based works are targeted to be included in oral sessions. Suggestions on improvements, method comparison, references to existing works should be provided with the review so that the authors can take them into account.
Finally, the reviewers should note that at the stage of the submission experimental results may not be available and fully analyzed. Therefore, the reviewers should not focus their evaluation on the ”Experiments and validation” criterion.