Standards of Physical Review Letters


     This entry relates experiences which give
insight into the standards of Physical Review Letters (PRL),
which is the self-proclaimed "world's foremost physics letters journal"
(a quote from the American Physical Society (APS) website).
To state the conclusion in advance, I have found the standards of PRL
to be much lower than they should be.  The journal seems to have no
interest in correcting erroneous work, nor in injecting any genuine peer
review.

    The story which I shall relate is perhaps the most bizarre of my
forty-odd years of professional experience.  It is long because the evidence
is complicated.  Perhaps few will care to read all of it in detail.  
Those who do not may prefer to skip to a summary at the end.

The story starts with three authors whom I shall call "S", "A", and "D",
who published a paper in the 2010 Physical Review Letters.  This paper,
together with its authors, will be called "SAD". 

The initials "S", "A", and "D" are not in 1-1 correspondence with the
actual initials of the authors.  During all of the incidents to be
described, "D" was an Assistant Professor at a good university,
subsequently promoted to Associate Professor with tenure.  The lead
author "S" is a graduate student of "D". "A" is another graduate student
with a different advisor.  The main characters will be "S" and "D";
"A" appears only as an author of SAD.  Subsequent papers about the same
topic were written only by "S" and "D" and will be called SD1, SD2, etc.

    I decided to substitute these  "pseudoinitials" for the
authors' real initials to decouple the personal story from my main aim
of commenting on the standards of PRL.  Apart from this 
substitution, nothing in the story has been altered. 

    Readers who look into the matter will have no difficulty guessing
the true identities of "S", "A", and "D", but I hope that maintaining
this minimal level of anonymity may help the reader focus on the broader
picture.   It does seem sad that individuals can act as disgracefully as
the "SAD" (or later, "SD") group has, but they are only individuals. 
What is truly sad is that such behavior seems to be accepted, and
in some ways actively enabled, by various prominent representatives of
the physics community such as PRL.


In January, 2011, I became interested in the SAD paper published in
Physics Review Letters.  The paper starts with various vaguely described
but tantalizing claims such as:
"WV [the theory of weak values] can be subsumed as a special case
of the CV ["contextual values"] formalism".
Its abstract gives a more specific claim:
    "We introduce contextual values as a generalization of the eigenvalues of
an observable that takes into account both the system observable
and a general measurement procedure.  This technique leads to
a natural definition of a general conditioned average that
converges uniquely to the quantum weak value in the minimal disturbance limit
[emphasis mine]."
When I speak of  the "main claim", or "claim" of SAD, the italized quote is what I shall mean.

Since no definition of  "minimal disturbance limit" was given in the paper, 
I initially interpreted  this term according to the normal meaning
of its words.  I assumed that the authors meant what would normally
be called a "weak limit"---a measurement depending on a "weakness" parameter g
which for sufficiently small g would result in an arbitrarily small disturbance
to the system being measured.  This turned out to be an incorrect interpretation,
but there was no way that the authors' actual meaning  could be deduced
from the paper, and I did not learn this for some time. 
Under my original interpretation, the quoted claim looked as if it might be
in contradiction to a recent paper of mine.  Of course, I had to look into
the possible contradiction.

Still under the impression that "minimal disturbance limit" was synonymous
with "weak limit", I found a counterexample to the claim quoted above
from SAD's abstract.  I also found that the "contexual values" concept
had been implicit in the literature for a long time. 
I wrote a paper relating contextual values to existing literature
and presenting the counterexample.  Before placing it in the arXiv,
I sent it to SAD asking for their comments. 
I was anxious to correct any errors before making it public.

In February, 2011, they sent me a reply which in retrospect I see
that I did not fully understand.  (This was not entirely my fault
because the reply was not clearly written.)  In particular,
I did not pick up on the fact that they were using "minimal disturbance limit"
in a technical sense which has little to do with the ordinary meanings
of its words.  A recent book of Wiseman and Milburn
(Quantum Measurement and Control, Cambridge U. Press, 2009,
of which I had never heard) defines "minimally disturbing measurements"
as ones whose measurement operators are positive. This is the only instance
of anything similar to this usage of  the term  which I have seen anywhere
in the literature apart from SAD and subsequent papers of its authors.
It turns out that SAD was using "minimal disturbance limit" in
a technical sense similar to that of the Wiseman/Milburn book,
but not identical.   No real definition is given in SAD,
and the Wiseman/Milburn book is not even in their reference list. 

This should never have gotten past the referee. 
I don't see how a typical reader of the paper could possibly be expected
to understand the above claim.  And if the claim is not clearly stated,
what is the point of publishing it?  And what justification can there be
for refusing to clarify it?

SAD's reply also included an attempted proof of the quoted claim. 
After I questioned parts of it, they sent me a second proof. 
The second proof was definitely incorrect because I found a counterexample
to one of its steps. However, at that time, I did not have a counterexample
to the claim itself.  I sent the counterexample to SAD,
but they never acknowledged it.  

    Over the next few months I sent them perhaps four or five queries
about various aspects of the SAD paper, but all were ignored.  It later
turned out that the authors were deliberately ignoring all my correspondence.  
(Up to the point where the SAD began ignoring correspondence,
all correspondence on both sides had been civil as, I believe, has been
all correspondence on my side.)

    After they had ignored the counterexamples and also hints that they
should retract the main claim of SAD (italicized in the quote above),
I submitted a  "Comment" paper to Physical Review Letters (PRL).
It pointed out a major error in their mathematics, namely the assumption
that *any* collection of unitary operators U(g) indexed by a real parameter g
can be written as U(g) = exp(iHg), with H a Hermitian operator. 
(This is true only for 1-parameter group U, i.e., satisfying
U(g+h) = U(g)U(h), a condition which would rarely be satisfied
in their context.)
 
    As is their policy, PRL sent it to the authors for their comments. 
The authors  essentially ignored the error, instead seeming to pretend
that they had intended all along to assume that g -> U(g) was a
1-parameter group. They apparently didn't  think it worthwhile to correct this
important error or misstatement.

    They also pointed out that their undefined "minimal disturbance limit"
was not identical to a "weak limit" as I had thought.  Although they certainly
should have defined this unusual usage in SAD (or at least given a reference to
their intended definition), this was a valid objection to the Comment.

    In light of this new information, I resubmitted
a revised comment.  I think that the last two paragraphs are worth quoting
separately:
    "The assumption U_j (g) = \exp (ig G_j) is so strong
    that the essence of the hypothesis for [SAD's] claim
    to have established (7) [the main claim of SAD] is probably that
    the measurement operators be positive (i.e., that U_j be trivial). 
    However, this is still an interesting hypothesis. My
    attempts to prove (7) or find a counterexample
    under the assumption that the measurement operators are positive
    have been unsuccessful, nor have I been able to obtain a correct proof
    from the authors. ...
    Those thinking of building on the work of 
    [SAD] or citing it should be aware that the status of (7)
    may be uncertain. The authors would do a service to other workers
    in the field by removing this uncertainty, either by a retraction or
    making public a precise definition of their "minimal uncertainty limit"
    and a peer-reviewable proof that (7) does follow in this
    minimal uncertainty limit."

    The main points are that

(1)  There is at least one serious  mathematical error in the statement of
     the main claim of SAD,   namely that it applies to an  *arbitrary* unitary U(g)
    (without assuming that g --> U(g) is a 1-parameter  unitary group).

(2)  The main claim of SAD is unproved. The authors have been unable
    or unwilling to furnish a proof.  It seems to me that this is
    of paramount importance!  A result for which there is no proof
    has been claimed and published in the self-proclaimed
    "world's foremost physics letters journal".

    To allow an unproved claim to remain in the literature is unethical,
according to standards of the American Mathematical Society (which I found
via a link from an APS web page, so one assumes that it should apply to
the APS as well):
    [From the ethical standards of the American Mathematical Society]:

    "... mathematicians have certain responsibilities, which include
    the following:

    ... To publish full details of results which are announced without
    unreasonable delay, because claiming a result in advance of its
    having been achieved with reasonable certainty injures the
    community by restraining those working toward the same goal.

    ... To correct in a timely way or withdraw work that is erroneous."
  
The rejection letter for the comment ignored point (1), and the Editor
was not at all bothered by point (2): 
    Dear Dr. Parrott,

    We have considered your response to my letter and your resubmission.
    It seems that you have tried to understand the assumptions
    and claims of [SAD], but "lacking knowledge of [SAD]'s precise assumptions,
    it would be impossible to say for sure what they thought they had proved,"
    so you do not know if Eq. (7) is correct "because the answer would have
    to depend on their unknown definition of 'minimal disturbance limit'."

    It may well be that [SAD] should have been clearer in stating their
    assumptions and in outlining the steps to arrive at (7). It may well
    be that had they done so, you would be able to prove their analysis
    was wrong. But these are hypotheticals. The bottom line is that we
    cannot publish a Comment which does not demonstrate that the original
    Letter was fundamentally wrong, even if the reason for that is that
    the Letter was too vague to pin down.

    [... The rest is essentially irrelevant.]

    [signed by an Editor]     
 
    PRL seems to maintain that its authors have no obligation
to justify their published claims.  I had formally requested from the authors
a definition of the "minimal disturbance limit" hypothesis of SAD's claim
and also a proof of the claim.  (The SAD paper intimated that
the claim followed from routine power series manipulation, which I had been
unable to duplicate.)  The authors had ignored the request. 
If PRL is at all concerned with the integrity of what it publishes,
it should provide some means of notifying potential readers that what
has been published has been credibly questioned, questions which the authors
have essentially refused to answer.

    According to the above PRL rejection letter, a "Comment" can
appear in PRL only if it proves that a claim is "fundamentally wrong",
and this has to be done within PRL's strict 1-page limit for "Comment"
papers!  The authors have no obligation to prove that the claim was correct! 

    Moreover, PRL apparently considers it entirely proper for the authors
to withhold a critical definition necessary to disprove the claim.  I was
95% certain that I could guess the authors' definition of "minimal disturbance limit"
from their reply to my first comment, but in view of my initial misinterpretation of
this definition, I wanted to be sure before making any definite statements about
 the correctness of SAD's main conclusion (7).
 
    I proceeded to look for a counterexample based on my best guess
at the meaning of "minimal disturbance limit" (which turned out to be
essentially correct).  Eventually, I found one.

    Before continuing, I would like to interject a personal note. 
I run considerable risk to my reputation by raising these issues publicly. 
It is so unusual that people are bound to wonder if my account can be trusted,
or even if I might be some kind of nut.  I hope that recounting the following
may allay some doubts.

    I said above that I thought that the SAD group has acted not just
unprofessionally, but disgracefully, and that PRL was enabling this behavior. 
By refusing to answer civil questions about their published work,
they have wasted much time of others.  However, not all of
"S", "A", and "D" may be equally culpable.  For example, "S" is
a graduate student who may well be financially dependent on "D"
(who has a large NSF grant).  

    As I submitted what I thought at the time was a definitive 
counterexample, I was uneasy about its possible effect on the career of "S".   
I wondered if he could be under some explicit or implicit pressure to act
unprofessionally.  Whatever the fallout from the counterexample, "D" would
amply deserve whatever he got, but "S" might not.  I did not want to
take the chance that "S" might not be a willing accomplice to "D".

    So, I decided to give "S" a chance to bail out, along with the
rest of the SAD group.  Before placing the counterexample in the arXiv
and submitting a new "Comment" to PRL,  I wrote him that I had found
a counterexample and offering the chance for either the SAD group or for
"S" alone (in case the group refused) to submit an Erratum  before the
counterexample was made public.  Since they would not have seen the
counterexample, they would have no need to mention it.  To all appearances,
they would be ethical authors correcting a mistake. 

    Of course, that would probably preclude publication of the counteraxample
on which I had worked so hard, but as a retired person,
I have no need to publish. After a few days without a response,
I appended the counterexample to an existing 30-odd page arXiv paper
analyzing SAD and submitted a third Comment to PRL.  
    [ I should emphasize that when this Comment was submitted,
    my only information about possible proofs of the main claim of SAD
    were the two incorrect proofs which SAD had sent me in February,
    after which they ignored all correspondence, including counterexamples
    to those  incorrect proofs.  Based on this sole information, I had to
    assume that SAD had no proof of their claim and that they knew this.
    Subsequently the "SD" group (seemingly, "A" has dropped
    out after SAD was published) has muddied the waters by adding strong
    additional hypotheses to the main claim of SAD and claiming that they
    could prove it under these additional hypotheses.  They further seem
    to claim, or intimate, that these additional hypotheses were assumed
    from the start and somehow omitted or downplayed in SAD due to
    lack of space.  (I know that is false because the incorrect
`    proofs that they sent me in Februrary did not assume these 
    additional hypotheses.)  ]
    This is important for judging the integrity of PRL.  At this point,
    to the best of my knowledge (and to the best knowledge of PRL,
    unless they had information which they did not share with me)
    PRL knew (or could and should have known) that PRL had published claims
    in SAD for which the authors had no proof. ]
 
    The cover letter for the submission informed the Editor that
the Comment might prove superfluous if SAD first submitted an Erratum:
[Exerpts from my cover letter for the third "Comment" submission]

"...    I don't know if you are aware that [SAD]'s lead author
["S"] is a graduate student whose advisor is one of the other autors [sic],
["D"].  It seems to me that ["D"] must shoulder a large share of
the responsibility for the excessive hype and poor exposition
of the PRL article.  It is easy to see how a graduate student might
get carried away by an approach which on careful examination proved
questionable, but one would expect his advisor to bring him down to earth
and to insist on a sober evaluation of results, claims, and hopes.

    And it bothers me that the more senior members of [SAD] may be
jeopardizing ["S"]'s career by insisting on counterproductive behavior
such as refusing to respond to civil requests for basic information
like definitions. For this reason, I wrote to ["S"] last week
(April 26) letting him know that I had found a counterexample
which I was writing up for the arXiv and  that if [SAD] or himself
submitted an erratum before it appeared, it would not be necessary
to mention the counterexample (because they would not have seen it). 
So, if they did submit an erratum before the counterexample appeared
in arXiv:1102.4407v4 (May 5), they will have fulfilled a professional
responsibility, albeit belatedly.  (Of course, they should have submitted
an erratum months ago when they realized that they could not prove
their claim that (6) implies (7) in the "minimal disturbance limit.) 

I assume they will eventually feel obligated to submit an erratum
after seeing the counterexample.  If so, I assume that my Comment
will probably never be published.  At least that is the  impression I get
from a document explaining the policies of PRL concerning Errata and
Comments.  That will be fine with me as long as they
correct the errors which have cost so much time (for me, and to a lesser
extent for PRL), and which if uncorrected have the potential of causing
others much more time. ...

Stephen Parrott "
I was astounded when the Editor's reply contained the following:
"  ...

I should tell you that Dr. ["D"] wrote me a concerned note about your
letter to ["S"]. I have no plans to intervene between you and them.
But you should know that a paragraph beginning "You have a small
window of time. . ." might be viewed by anyone as a threat, no matter
how well meaning you intended to be. If you wish more free advice, I
would suggest you refrain from any further such 'advice' to ["S"],
and avoid any inferences about behavior.
"
     The sneering tone seems unprofessional, to say the least. 
The judgment  that I had somehow made an improper threat is
beyond my comprehension. Had I said "If you do not submit an erratum,
I will place a counterexample in the arXiv", that could possibly be considered
a threat, though not an improper one. But I did not even say that. 
I simply informed "S" that I would place the counterexample in the arXiv
within a few days.  I would have done so no matter what they did,
and never did I intimate otherwise.  They were free to do what they wished
with this information.   Were I in their position, I would be grateful
for this additional information.

    For the editor to issue any value judgment on my letter to "S"
was unprofessional.  I had not sent him a copy; his response was based
entirely on whatever "D" had sent him, and he had no way to verify
its accuracy or completeness.  That letter was private correspondence
which was entirely outside the purview of PRL.  It was this Editor's response
that made me realize that I might be dealing with editorial bias,
apart from the Editor's previously evident lack of concern about the integrity
of what PRL publishes.

    For reference, I reprint the relevant parts of my letter to "S", including
the paragraph which the Editor unaccountably characterized as a "threat":

Dear Mr. "S":

I have found a counterexample to [SAD]'s assertion that (6) implies
(7) for positive measurement operators, with the ``weak limit'' defined
by my (105). I am fine-tuning the writeup and expect to submit it to the
arXiv in a few days.

...

You have a small window of time, almost certainly no more than
a week, and perhaps as little as a day or two, to submit an erratum to PRL
before my counterexample goes public in the arXiv, and perhaps a few days
more before my Comment paper is resubmitted.

You may not be able to persuade [SAD] to do this because the other
members of the group have far less to lose than you. In that case, you may
want to consider the possiblity of doing it yourself. Since you have not
seen my counterexample and have no way to know if it is correct,
you would need only say that since publication of [SAD], a gap in its proof
that (6) implies (7) in the "minimal disturbance limit" has been discovered,
and you no longer make that claim. After the counterexample appears
in the arXiv, you would almost certainly have to mention it.

...

Sincerely yours,

Stephen Parrott

    Returning to the main narrative, recall that I have just posted
my counterexample in the arXiv and submitted a third "Comment" to PRL including
a reference to this counterexample which is worked out in detail in the arXiv. 
This "Comment" was rejected because the counterexample was not included
in the Comment itself (which would have been impossible because
the counterexample was too long for the 1-page limit for a "Comment").

    I then refined the counterexample so that its essence could
be described in one page and verified mentally, and rewrote and submitted
a fourth attempt at a "Comment" to PRL.  `

    The authors were asked to reply, and a letter from "S" and "D"
(but perhaps significantly, not signed by "A")
said that they could not adequately do so in the 1-page limit
for "Reply" papers, but that they had posted a more lengthy reply in
a new arXiv paper.  The Editor responded that unless I could refute the
criticisms in that arXiv paper, the Comment would be rejected. 
[From SD's reply letter to PRL  ]

"... we have now posted a paper on the arXiv [citation] devoted entirely
to this issue [the counterexample submitted in the Comment].  Just as
Dr. Parrott cannot put his objection into a one page summary, we cannot
put our reply into one page.  ...

In our new paper that can be taken as a reply to Dr. Parrott's latest
comment we do the following:

1) Review the basics of the contextual value formalism.

2) Analyze Dr. Parrott's counterexample and demonstrate that it does
not satisfy our pseudo-inverse prescription for constructing the contextual values.
...

4) Give a discussion of the pseudo-inverse prescription and discuss why
it is both mathematically and physically important."
...
[signed "S" and "D"]
SD's arXiv reply will henceforth be abbreviated as SD1, to distinguish
it from later papers of SD.

The essence of the Editor's response was:
"  ... We would be willing to reconsider your Comment further only if you can
make a convincing case in one PRL page that the original Letter was
fundamentally flawed in light of all the information available. Given
the extensive postings by you and ["S"] et al, it is hard to see how
that could be accomplished. ...
In other words, we can reconsider your Comment only if you can rebut
all of the criticism of ["S"] et al reflected in their response via
minor modifications to your current Comment.
     
  [signed by the Editor] "
    Recall that the stated reason for rejecting my third "Comment"
submission was that it referred to a counterexample fully worked out
in the arXiv, but too long for a 1-page Comment.  However, the Editor
is perfectly willing to accept at face value SD's claim to have refuted
the counterexample in an arXiv paper.  This reinforced a growing suspicion
that editorial bias might be playing a role in my efforts to inject
some genuine peer review into PRL.

    Item 2) of SD's reply injects a new element into the controversy.
They now claim that SAD specifies a new hypothesis, their so-called
"pseudo-inverse prescription" (which the counterexample of the Comment
does not satisfy, though a later counterexample does).  Before examining
this new claim, I must say a bit more about the logical structure
of the SAD paper.

    First, it defines "contextual values", which it notes are
not necessarily unique.  Then SAD states:
    "...we propose that the physically sensible choice of 
    CV [contexual values] is the least redundant set uniquely related
    to the eigenvalues through the Moore-Penrose pseudoinverse."
    [The strange language (e.g. "least redundant
    set") probably carries meaning only to the authors.  Subsequent
    developments made clear what the authors actually meant. Such
    logically meaningless language should never have gotten past the
    referee.]
    This is what the above item 2) calls the "pseudo-inverse" prescription,
and the quote just given is the only  substantive reference to it in SAD.
(A following paragraph describes a  method of computing the
Moore-Penrose pseudoinverse, but this is standard mathematical material which
has nothing to do with the pseudo-inverse prescription.)
No explanation is given as to why this should be the "physically sensible
choice", and no indication is given that this should be a hypothesis for
further statements in the paper.

    Then SAD obtains what it characterizes as its "main result",
a so-called "conditioned average" given by its equation (6).  The
pseudo-inverse prescription is not necessary for the derivation
of (6). 

    Next follows a section entitled "Weak values" which attempts
to justify the only nontrivial claim of its abstract, namely that its
"general conditioned average" "converges uniquely to the quantum weak value
[SAD's equation (7)] in the minimal disturbance limit".  This section does
not mention the pseudo-inverse prescription. 

    How can a reader possibly guess that the "pseudo-inverse prescription",
introduced in such a vague way above, is to be taken as a hypothesis for
(7) but not for (6)?  This should never have gotten past a referee who read
the paper with understanding.  Even if one accepts the questionable claim that
the pseudo-inverse prescription should be taken as a hypothesis for (7), 
it seems unquestionable that the SAD paper does not clearly state this. 
Is this not worth a "Comment", to prevent future readers from being misled?

    When I posted the counterexample to (7), I did consider whether
the pseudo-inverse prescription might possibly be a hypothesis, but rejected
the possibility because the incorrect proofs which SAD had sent me in February
did not require the pseudo-inverse prescription as a hypothesis.  So,
I knew to a certainty that when SAD was submitted, the authors did not
assume the pseudo-inverse prescription as a hypothesis for (7).  They added
it later only after they realized that their initial proof was wrong. 

    Morever, when SD finally explained in their arXiv reply SD1
why they consider the pseudo-inverse prescription "mathematically and
physically important", the explanation (which happens to be questionable,
but that's another issue) has nothing to do with weak values.  It has
nothing to do with the reasoning of SAD leading to (7). 

   Their arXiv reply adds other nontrivial
hypotheses for (7) which are unequivocally not even mentioned in SAD.     
I hesitate to subject the reader to these technical details,
but I had to include them in order to understand what happened next.  The
bottom line is that their reply to the submitted "Comment" claims that
(7) can be proved under additional hypotheses not given in SAD, but 
seems to pretend that these additional hypotheses were already clearly
stated in SAD.

    Finally, after months of exchanges, PRL sent the Comment to a referee.
My cover letter for the first "Comment" submitted back in February,
specifically requested that if the Comment were to be refereed,
the referee not be one who had approved the SAD paper. 
That was because it was clear to me that the referee could not
possibly have read SAD with understanding; if he had, he could never have
approved it in anything close to its published form.  At a minimum,
he should have insisted on clear definitions and statements of hypotheses.
It seems obvious that if a referee were to be consulted, for confidence in his report,
it should be one who did not have a vested interest
in justifying his initial positive recommendation.

    Nevertheless, the Editor did send the Comment to a referee
who had originally approved SAD.  I know this because his report
said so.  I would like to include the report verbatim, but I will not
because the Editor specifically asked me not to, which raises copyright issues.
However, I will paraphrase parts of it and include a  quote which can
be considered "fair use" under the copyright law.
[Paraphrase of parts of the referee's report on the "Comment"]

    The referee expresses reservations about the SAD paper and
related work of other authors.  He encourages me to publish my

" findings about the concept of weak values and contextual values as an ordinary article.
In this respect I agree with the Editor [emphasis mine], [name of Editor],
that this would be the more appropriate solution."
The emphasized phrase, "I agree with the Editor",  is troubling.  It suggests that the Editor
may have (intentionally or not) improperly influenced the referee, perhaps subtly
indicating the kind of report which he hopes for.  These editors have a lot
of power, and should scrupulously avoid any appearance of "leading" referees. 
They should seek a referee's opinion without interjecting their own.
The report continues:
"It is basically because of the lack of self-containedness of the
comment, which make it practically impossible for a reader to gain any

useful insights concerning the problem. She/He would have to read
additional manuscripts published as E-prints, as well as Author
replies and Referee comments, which are obviously inaccessible to the
readers, in order to understand the discussion. It is for those
reasons, that I cannot recommend the publication of the comment in
Physical Review Letters."

This is simply wrong.  It is true that a reader unfamiliar with SAD cannot expect
to follow the technical details of the counterexample given in the Comment. 
But it would be completely unreasonable to demand that a Comment limited to
only one page should summarize the original paper sufficiently that a reader unfamiliar
with it could  follow the technical details.  Contrary to what the referee says,
a reader who was already familiar with SAD should be able to grasp the essence
of the counterexample.  More importantly, readers not already familiar with SAD
(which would include almost all readers) might find it useful to know that
a nontrivial claim of SAD had been credibly questioned.

     They would find it even more useful to know that an independent
referee had checked the counterexample, if he had.  (The  counterexample was
extremely simple, involving only 2x2 diagonal matrices.)  Unfortunately,
this referee ignored the issue of the correctness of the counterexample.
But that is the essential issue!  If the counterexample is incorrect,
then obviously PRL should not publish it in a Comment.  If it is correct,
then PRL should have a professional obligation to inform readers that
a published  claim has turned out to be false.

    Because the referee's report ignored the main issue, it is obviously
deficient.  It should never have been accepted by a conscientious editor.
He should have consulted another referee at that point.

    Recall that the Editor had previously written:
"We would be willing to reconsider your Comment further only if you can
make a convincing case in one PRL page that the original Letter was
fundamentally flawed in light of all the information available."
    The mathematics of the Comment's counterexample has never been
questioned, even by the authors.  The only issue is whether SAD clearly states
the pseudo-inverse prescription (along with the other strong hypotheses
added later in SD1) as a hypothesis for its nontrivial claim (7). 
The referee either did not recognize or ignored this crucial issue. 
If a referee ruled that SAD did not clearly state these additional
assumptions as hypotheses for (7), then would not this constitute
clear evidence that the SAD paper was "fundamentally flawed". 
Why did the Editor not insist that the referee address this crucial point? 
Or can it be that PRL's standards do not preclude publishing claims
that require very strong, unstated hypotheses to prove?

    I was dissatisfied not only with this outcome, but
more importantly, the way that it was obtained.  In view of the
intemperate remarks of the Editor noted above, I wondered if editorial
bias might be playing a role.  So, I decided to appeal, mainly to
determine the true standards of PRL.  The result truly surprised me,
based on my previous experiences with PRL,which had involved only this Editor.

    The appeal went to a Divisional Associate Editor (DAE), who had the
integrity to sign his name (which of course I won't reveal).  It is my
impression that Divisional Associate Editors are not APS employees like
the Editor who handled the Comment, but volunteers somewhat analogous
to referees for papers.

    Based on the previous superficial treatment by one Editor, I was very surprised
when this DAE submitted a report which made clear that he had spent
a lot of time and effort trying to understand the substance of the matter.
It also made me realize how difficult the mathematical issues (which seem
almost trivial to me) could appear to a competent person not already
intimately familiar with the problem.

     His report is lengthy  and technical, and I would not reproduce
it in full even if there were not copyright issues.  But I can see no
harm in quoting the first few paragraphs, which only show the integrity
of this individual:

  "Under normal circumstances when dealing with an appeal, I would read
 the manuscript first so as to form an independent opinion of it. Doing
so on this occasion (and also reading the original manuscript, of
course) did not give me a clear understanding of the issues and in no
way prepared me for the 70 pages of correspondence, which was enough
to make my head spin.

In all my time as a board member and editor for four journals, this
case has been the most stamina-sapping. I have tried very hard to
understand the problem here. I have read the original paper and the
comment (both several times) and also referred to the arXiv papers. I
have not published research explicitly on weak measurements, but I
have followed the field since its inception and one thing must, I
think, be quite clear: If I, as an informed even specialist reader,
have had this much difficulty then there is no way that our general
readership will be able to follow the subtleties by reading a one page
comment and reply.

If the above paragraph presents what must be my recommendation, it
does not suffice as a report and (understandably) would not satisfy
the author. I felt that it was necessary to try to understand the
roots of the problem that seems to have been the source of so much
acrimony. Most of my time, and the remaining parts of this long report
are devoted to this task. I do hope that this effort has not been
wasted.

[The report goes on for about 2 pages to examine technical details.
The DAE makes some mistakes, but with admirable honesty admits
that his understanding is limited.  He concludes:]

"My suspicion (I dare state it no more strongly than that) is that it is here
[referring to a previous statement] that the problem lies."
    Before commenting on this, I want to make clear that
I greatly appreciate the time and effort that the DAE devoted to this.
Up to now, my dealings with PRL (i.e., with one particular Editor) had
given no hint that PRL had the least interest in correcting errors
in what it had published.  The DAE's report shows integrity, even where
it may falter on technical details.  However the following criticisms
should not go unnoted.

(1)  The DAE is right that readers not already familiar with
the SAD paper (which is probably almost all readers) will not be able
to follow the technical details of the counterexample. 
This is normal.  I do not recall ever understanding a PRL "Comment"
for a paper which I had not previously read in detail.

    But that does not  imply that such a Comment is useless
and unfit for publication.  Its utility lies in alerting the reader that
a nontrivial claim of SAD is credibly questioned.   It would contribute
to the reliability of the literature by deterring those who might be tempted
to build on the claim without checking its alleged proof.  It would contribute
to scientific progress by not deterring further work on the claim
(as eloquently stated in the above statement of ethics of the American
Mathematical Society).

    If the policy of PRL is that a large fraction of readers must be able
to follow all "Comment"s in full technical detail, then almost no "Comment"s
should be published.  Such an insistence seems close to a double standard,
a pretext for summarily settling an inconvenient matter which no editor fully
understands.

(2)  The DAE does not address the important issue that SD's attempted
arXiv proof of SAD's claim requires important, non-trivial hypotheses
not stated in SAD.  One of these is the pseudo-inverse prescription,
which is at least mentioned though not clearly stated as a hypothesis for
anything, but there are others as well.  Even if SD's arXiv proof were correct
(and I don't think it is), it would not prove the claim as stated in SAD.

Can it be that the policy of PRL is to allow misleading claims which
can be proved, if at all, only under strong and complicated additional
hypotheses?  Would the "premier" journal PRL have initially published SAD
had it realized this?  If not, should it not feel an obligation to correct
misleading claims which were mistakenly published?
The gist of the  Editor's final rejection of the Comment was:
"Dear Dr. Parrott,
...

The DAE delves into the details in an effort to understand the
dispute, but the second paragraph of the report presents a sufficient
case for final rejection of your Comment. The advice is in agreement
with what we have been trying to communicate to you--that a Comment is
not the right vehicle for your results.

Thus, the scientific review of your Comment has come to an end.

Yours sincerely,

[Signed by the Editor who has handled all my submitted "Comments" on SAD]

Editor

Physical Review Letters

The premier APS journal for current research"
    This seems hypocritical, perhaps deliberately so.
I am sure that the Editor is fully aware that it would be impossible
to publish the counterexample anywhere other than PRL.
The counterexample has no mathematical interest.  Its only interest is
to prevent SAD from misleading further readers, and that is the responsibility
of "the premier APS journal" PRL which published SAD, not some other journal.

Summary and Conclusions

    The above is so long and convoluted that possibly no one will read it in detail,
 but I felt that I should put forward all the evidence. Following is a summary. 
Recall that "S", "A", and "D" are pseudoinitials referring to the three authors of
a 2010 paper in Physical Review Letters. The paper is called SAD..

What happened
(1)    SAD made a claim (to be called the "CLAIM" below) which turned out
 to be  demonstrably false as stated in SAD.  (Two of the authors later
maintained that the claim could be proved under strong hypotheses not
mentioned in SAD.)  It was not initially known that the claim was false,
but it was known that the authors had no proof.
There was also a serious, demonstrable mathematical
error in SAD (to be called the ERROR below), among lesser errors.

(2)  The authors refused private (and civil) requests for a valid proof
of the CLAIM, and even refused to furnish necessary definitions. 
(This was after an initial response of SAD to a proof request had furnished
an incorrect proof.)

(3)  PRL rejected a "Comment" paper  pointing out the ERROR,
for stated reasons other than the ERROR, and without mentioning the ERROR.
The impression given is that PRL considers the ERROR unimportant.

(4)  PRL rejected a "Comment" paper which both pointed out the ERROR and
gave a reference to a counterexample to the CLAIM.  The reference was to
an arXiv paper which fully worked out the counterexample.  The mathematics
of this counterexample has never been questioned, even by the authors of SAD.
The stated reason for the rejection was that the original counterexample was
too long to include in the 1-page "Comment" itself.  The ERROR was again
ignored.

(5)  A revised "Comment" which did include a counterexample was then rejected.
The stated reason was that general readers (presumably, ones not familiar
with the original SAD paper) would not be able to follow the technical details
of the Comment.

(6)  Although the paper was sent to a referee, it was a referee who
had initially approved publication of SAD and thus had a vested interest
in justifying his initial recommendation.  (I had specifically requested
that it be sent to an independent referee as well.)  The referee ignored
the issue of the validity of the counterexample, which one would think
would be the main issue!

Conclusions
   
(1) If the above is typical of the practices of PRL, then it cannot be
considered a reliable journal because it makes it almost impossible
to correct errors in what it publishes. Had I realized the ultimate effort
involved, I might never have started.  Were I not retired, I could never
have afforded the time.  According to a Divisional Associate Editor,
the PRL file (most of which was probably written by me) is about
70 pages.

(2)   There is an outside possibility that the handling of these "Comment"
submissions may not reflect the policy of PRL, but might be an artifact of
practices of a particular editor. 
    Evidence for this possibility includes intemperate
remarks of this editor, including insulting personal comments unrelated
to the "Comment" or PRL.  (The Editor later gave a half-hearted apology,
"Sorry".)  Also, there is evidence that the Editor may have
(intentionally or not) "led" the referee by indications of his personal views
of the matter.  Further, the Editor chose a referee who had a vested interest
in justifying his previous positive recommendation for publication of SAD,
despite my request that the "Comment" also be sent to an independent referee.
    Evidence against this possibility includes the concurrence of
a Divisional Associate Editor to whom the matter was appealed.  If he saw
anything wrong with the Editor's conduct, he did not mention it.

(3)  I find it very disturbing that PRL seemed to be actively enabling the
unprofessional behavior of the SAD group.  They had refused to furnish
a proof of the major CLAIM of the PRL publication SAD, and had even refused
to state a definition relevant to the CLAIM.  They had ignored all
questions about the SAD paper since February (when I had sent them a
counterexample).

    The Editor seemed to find these "stonewall" tactics perfectly acceptable. 
Had he requested the SAD group to provide minimal cooperation toward coming
to agreement on what SAD claimed and what it actually proved, the whole
matter might have been resolved informally.  At a minimum, the issues would
have been more clearly delineated and much time would have been saved.  
The Editor acted more as an advocate for SAD than as an impartial judge using
all means at his disposal to determine the facts of the matter.
 (4)  It is easy to lose sight of the forest for the trees, as the saying goes. 
The welter of detail above is enough to confuse anybody, as the DAE's report
indicated.  Let us step back and take a commonsense view.

    PRL has never disputed the credibility of my objections to the SAD
paper.  The referee did not dispute it.  What can be a possible justification
for "Comment" policies which preclude alerting potential readers that SAD
is credibly disputed?

    I know of no other publication which forbids expressions of opinion
in "Comments" (more usually called Letters to the Editor) even though space
may preclude definitive proof of what the letter-writer opines.  For example,
if some article draws some conclusion about global warming based on a computer
model, a letter expressing doubts about the assumptions of the model
would normally be acceptable.  If the letter-writer is a known expert on
global warming, the knowledge that the model is disputed can be useful to
readers even if a reader cannot definitively judge the details of the dispute.  

    The narrow and rigid PRL policy that a "Comment" must
"make a convincing case in one PRL page that the original Letter was
fundamentally flawed" will preclude most expressions of doubt and is
a disservice to potential readers of the Letter.  A journal which touts
itself as "the world's foremost physics letters journal" and "the premier
APS journal for current research" ought to enable, rather than suppress,
genuine peer review.