This entry relates experiences which give
insight into the
standards of
Physical Review Letters (PRL),
which is the self-proclaimed
"world's foremost physics letters journal"
(a quote from the
American Physical Society (APS) website).
To state the conclusion
in advance, I have found the standards of PRL
to be much lower
than they should be. The journal seems to have
no
interest in correcting
erroneous work, nor in injecting any genuine peer
review.
The story which I shall relate is perhaps the most
bizarre of my
forty-odd years of
professional experience. It is long because
the evidence
is complicated.
Perhaps few will care to read all of it in detail.
Those who do not
may prefer to skip to a summary at the end.
The story starts
with three authors whom I shall call "S", "A", and
"D",
who published a paper
in the 2010 Physical Review Letters. This paper,
together with its
authors, will be called "SAD".
The initials "S",
"A", and "D" are not in 1-1 correspondence with the
actual initials of
the authors. During all of the incidents to
be
described, "D" was
an Assistant Professor at a good university,
subsequently promoted to Associate
Professor with tenure. The lead
author "S" is a graduate
student of "D". "A" is another graduate student
with a different
advisor. The main characters will be "S" and
"D";
"A" appears only
as an author of SAD. Subsequent papers about
the same
topic were written
only by "S" and "D" and will be called SD1, SD2, etc.
I decided to substitute these "pseudoinitials"
for the
authors' real initials
to decouple the personal story from my main aim
of commenting on
the standards of PRL. Apart from this
substitution, nothing
in the story has been altered.
Readers who look into the matter will have no difficulty
guessing
the true identities
of "S", "A", and "D", but I hope that maintaining
this minimal level
of anonymity may help the reader focus on the broader
picture.
It does seem sad that individuals can act as disgracefully
as
the "SAD" (or later,
"SD") group has, but they are only individuals.
What is truly sad
is that such behavior seems to be accepted, and
in some ways actively
enabled, by various prominent representatives of
the physics community
such as PRL.
In January, 2011,
I became interested in the SAD paper published in
Physics Review Letters.
The paper starts with various vaguely described
but tantalizing claims such as:
"WV [the theory of weak values] can be subsumed as
a special case
of the CV ["contextual values"] formalism".
Its abstract gives
a more specific claim:
"We introduce contextual
values as a generalization of the eigenvalues of
an observable that
takes into account both the system observable
and a general measurement
procedure. This technique leads to
a natural definition
of a general conditioned average that
converges uniquely
to the quantum weak value in the minimal disturbance
limit
[emphasis mine]."
When I speak of the
"main claim", or "claim" of SAD, the italized quote
is what I shall mean.
Since no definition
of "minimal disturbance limit" was given in
the paper,
I initially interpreted
this term according to the normal meaning
of its words.
I assumed that the authors meant what would normally
be called a "weak
limit"---a measurement depending on a "weakness" parameter
g
which for sufficiently
small g would result in an arbitrarily small disturbance
to the system being
measured. This turned out to be an incorrect
interpretation,
but there was no
way that the authors' actual meaning could be
deduced
from the paper, and
I did not learn this for some time.
Under my original
interpretation, the quoted claim looked as if it might
be
in contradiction
to a recent paper of mine. Of course, I had
to look into
the possible contradiction.
Still under the impression
that "minimal disturbance limit" was synonymous
with "weak limit",
I found a counterexample to the claim quoted above
from SAD's abstract.
I also found that the "contexual values" concept
had been implicit
in the literature for a long time.
I wrote a paper relating
contextual values to existing literature
and presenting the
counterexample. Before placing it in the arXiv,
I sent it to SAD
asking for their comments.
I was anxious to
correct any errors before making it public.
In February, 2011,
they sent me a reply which in retrospect I see
that I did not fully
understand. (This was not entirely my fault
because the reply
was not clearly written.) In particular,
I did not pick up
on the fact that they were using "minimal disturbance
limit"
in a technical sense
which has little to do with the ordinary meanings
of its words.
A recent book of Wiseman and Milburn
(Quantum Measurement
and Control, Cambridge U. Press, 2009,
of which I had never
heard) defines "minimally disturbing measurements"
as ones whose measurement
operators are positive. This is the only instance
of anything similar
to this usage of the term which I have
seen anywhere
in the literature
apart from SAD and subsequent papers of its authors.
It turns out that
SAD was using "minimal disturbance limit" in
a technical sense
similar to that of the Wiseman/Milburn book,
but not identical.
No real definition is given in SAD,
and the Wiseman/Milburn
book is not even in their reference list.
This should never
have gotten past the referee.
I don't see how a
typical reader of the paper could possibly be expected
to understand the
above claim. And if the claim is not clearly
stated,
what is the point
of publishing it? And what justification can
there be
for refusing to clarify
it?
SAD's reply also
included an attempted proof of the quoted claim.
After I questioned
parts of it, they sent me a second proof.
The second proof
was definitely incorrect because I found a counterexample
to one of its steps.
However, at that time, I did not have a counterexample
to the claim itself.
I sent the counterexample to SAD,
but they never acknowledged
it.
Over the next few months I sent them perhaps four
or five queries
about various aspects
of the SAD paper, but all were ignored. It later
turned out that the
authors were deliberately ignoring all my correspondence.
(Up to the point
where the SAD began ignoring correspondence,
all correspondence
on both sides had been civil as, I believe, has been
all correspondence
on my side.)
After they had ignored the counterexamples and also
hints that they
should retract the
main claim of SAD (italicized in the quote above),
I submitted a
"Comment" paper to Physical Review
Letters (PRL).
It pointed out a
major error in their mathematics, namely the assumption
that *any* collection
of unitary operators U(g) indexed by a real parameter
g
can be written as
U(g) = exp(iHg), with H a Hermitian operator.
(This is true only
for 1-parameter
group U, i.e., satisfying
U(g+h) = U(g)U(h),
a condition which would rarely be satisfied
in their context.)
As is their policy, PRL sent it to the authors for
their comments.
The authors
essentially ignored the error, instead seeming to
pretend
that they had intended
all along to assume that g -> U(g) was a
1-parameter group.
They apparently didn't think it worthwhile to
correct this
important error or
misstatement.
They also pointed out that their undefined "minimal
disturbance limit"
was not identical
to a "weak limit" as I had thought. Although
they certainly
should have defined
this unusual usage in SAD (or at least given a reference
to
their intended definition),
this was a valid objection to the Comment.
In light of this new information, I resubmitted
a
revised comment. I think that
the last two paragraphs are worth quoting
separately:
"The assumption
U_j (g) = \exp (ig G_j) is so strong
that the essence of the hypothesis for [SAD's] claim
to have established (7) [the main claim of SAD] is
probably that
the measurement operators be positive (i.e., that
U_j be trivial).
However, this is still an interesting hypothesis.
My
attempts to prove (7) or find a counterexample
under the assumption that the measurement operators
are positive
have been unsuccessful, nor have I been able to obtain
a correct proof
from the authors. ...
Those thinking
of building on the work of
[SAD] or citing it should be aware
that the status of (7)
may be uncertain. The authors
would do a service to other workers
in the field by removing this
uncertainty, either by a retraction or
making public a precise definition
of their "minimal uncertainty limit"
and a peer-reviewable proof
that (7) does follow in this
minimal uncertainty limit."
The main points are that
(1) There is at least one serious
mathematical error in the statement of
the main
claim of SAD, namely
that it applies to an *arbitrary* unitary U(g)
(without assuming that g --> U(g) is a 1-parameter unitary
group).
(2)
The main claim of SAD is unproved. The authors have
been unable
or unwilling to furnish a proof. It seems to
me that this is
of paramount importance! A result for which
there is no proof
has been claimed and published in the self-proclaimed
"world's foremost physics letters journal".
To allow an unproved claim to remain in the literature
is unethical,
according to standards
of the American Mathematical Society (which I found
via a link from an
APS web page, so one assumes that it should apply
to
the APS as well):
[From the ethical standards of the American
Mathematical Society]:
"... mathematicians have certain responsibilities,
which include
the following:
... To publish full details of results which are
announced without
unreasonable delay, because claiming a result in advance
of its
having been achieved with reasonable certainty injures
the
community by restraining those working toward the
same goal.
... To correct in a timely way or withdraw work that
is erroneous."
The rejection letter for the comment ignored point
(1), and the Editor
was not at all bothered
by point (2):
Dear
Dr. Parrott,
We have considered your response to my letter and your
resubmission.
It seems that you have tried to understand the assumptions
and claims of [SAD], but "lacking knowledge of [SAD]'s
precise assumptions,
it would be impossible to say for sure what they thought
they had proved,"
so you do not know if Eq. (7) is correct "because
the answer would have
to depend on their unknown definition of 'minimal
disturbance limit'."
It may well be that [SAD] should have been clearer
in stating their
assumptions and in outlining the steps to arrive
at (7). It may well
be that had they done so, you would be able to prove
their analysis
was wrong. But these are hypotheticals. The bottom
line is that we
cannot publish a Comment which does not demonstrate
that the original
Letter was fundamentally wrong, even if the reason
for that is that
the Letter was too vague to pin down.
[... The rest is essentially irrelevant.]
[signed by an Editor]
PRL seems to maintain
that its authors have no obligation
to justify their
published claims. I had formally requested
from the authors
a definition of the
"minimal disturbance limit" hypothesis of SAD's claim
and also a proof
of the claim. (The SAD paper intimated that
the claim followed
from routine power series manipulation, which I had been
unable to duplicate.) The authors had ignored
the request.
If PRL is at all concerned with
the integrity of what it publishes,
it should provide some means of notifying potential readers that what
has been published has been credibly questioned, questions which the authors
have essentially refused to answer.
According to the above PRL rejection letter, a "Comment"
can
appear in PRL only
if it proves that a claim is "fundamentally wrong",
and this has to be
done within PRL's strict 1-page limit for "Comment"
papers! The
authors have no obligation to prove that the claim
was correct!
Moreover, PRL apparently considers it entirely proper
for the authors
to withhold a critical
definition necessary to disprove the claim.
I was
95% certain that
I could guess the authors' definition of "minimal disturbance
limit"
from their reply to my first comment, but in view of my
initial misinterpretation of
this definition, I wanted
to be sure before
making any definite statements about
the correctness
of SAD's main conclusion
(7).
I proceeded to look for a counterexample based on
my best guess
at the meaning of
"minimal disturbance limit" (which turned out to be
essentially correct).
Eventually, I found one.
Before continuing, I would like to interject a personal
note.
I run considerable
risk to my reputation by raising these issues publicly.
It is so unusual
that people are bound to wonder if my account can be
trusted,
or even if I might
be some kind of nut. I hope that recounting the
following
may allay some doubts.
I said above that I thought that the SAD group has
acted not just
unprofessionally,
but disgracefully, and that PRL was enabling this behavior.
By refusing to answer
civil questions about their published work,
they have wasted
much time of others. However, not all of
"S", "A", and "D"
may be equally culpable. For example, "S" is
a graduate student
who may well be financially dependent on "D"
(who has a large
NSF grant).
As I submitted what I thought at the time was a definitive
counterexample, I
was uneasy about its possible effect on the career
of "S".
I wondered if he
could be under some explicit or implicit pressure to
act
unprofessionally.
Whatever the fallout from the counterexample, "D"
would
amply deserve whatever
he got, but "S" might not. I did not want
to
take the chance that
"S" might not be a willing accomplice to "D".
So, I decided to give "S" a chance to bail out, along
with the
rest of the SAD group.
Before placing the counterexample in the arXiv
and submitting a
new "Comment" to PRL, I wrote him that I had
found
a counterexample
and offering the chance for either the SAD group or
for
"S" alone (in case
the group refused) to submit an Erratum before
the
counterexample was
made public. Since they would not have seen
the
counterexample, they
would have no need to mention it. To all appearances,
they would be ethical
authors correcting a mistake.
Of course, that would probably preclude publication
of the counteraxample
on which I had worked
so hard, but as a retired person,
I have no need to
publish. After a few days without a response,
I appended the counterexample
to an existing 30-odd page arXiv paper
analyzing SAD and
submitted a
third Comment to PRL.
[ I should emphasize that
when this Comment was submitted,
my only information about possible proofs of the main
claim of SAD
were the two incorrect proofs which SAD had sent
me in February,
after which they ignored all correspondence, including
counterexamples
to those incorrect proofs. Based on this
sole information, I had to
assume that SAD had no proof of their claim and that
they knew this.
Subsequently the "SD" group (seemingly, "A" has dropped
out after SAD was published) has muddied the waters
by adding strong
additional hypotheses to the main claim of SAD and
claiming that they
could prove it under these additional hypotheses.
They further seem
to claim, or intimate, that these additional hypotheses
were assumed
from the start and somehow omitted or downplayed
in SAD due to
lack of space. (I know that is false because
the incorrect
`
proofs that they sent me in Februrary did not assume
these
additional hypotheses.) ]
This is important for judging the integrity
of PRL. At this point,
to the best of my knowledge (and to the best knowledge
of PRL,
unless they had information which they did not share
with me)
PRL knew (or could and should have known) that PRL
had published claims
in SAD for which the authors had no proof. ]
The cover letter for the submission informed the
Editor that
the Comment might
prove superfluous if SAD first submitted an Erratum:
[Exerpts from my cover
letter for the third "Comment" submission]
"...
I don't know if you are aware that [SAD]'s lead
author
["S"] is a graduate
student whose advisor is one of the other autors [sic],
["D"]. It seems
to me that ["D"] must shoulder a large share of
the responsibility
for the excessive hype and poor exposition
of the PRL article.
It is easy to see how a graduate student might
get carried away
by an approach which on careful examination proved
questionable, but
one would expect his advisor to bring him down to earth
and to insist on
a sober evaluation of results, claims, and hopes.
And it bothers me that the more senior members of
[SAD] may be
jeopardizing ["S"]'s
career by insisting on counterproductive behavior
such as refusing
to respond to civil requests for basic information
like definitions.
For this reason, I wrote to ["S"] last week
(April 26) letting
him know that I had found a counterexample
which I was writing
up for the arXiv and that if [SAD] or himself
submitted an erratum
before it appeared, it would not be necessary
to mention the counterexample
(because they would not have seen it).
So, if they did submit
an erratum before the counterexample appeared
in arXiv:1102.4407v4
(May 5), they will have fulfilled a professional
responsibility, albeit
belatedly. (Of course, they should have submitted
an erratum months
ago when they realized that they could not prove
their claim that
(6) implies (7) in the "minimal disturbance limit.)
I assume they will
eventually feel obligated to submit an erratum
after seeing the
counterexample. If so, I assume that my Comment
will probably never
be published. At least that is the impression
I get
from a document explaining
the policies of PRL concerning Errata and
Comments. That
will be fine with me as long as they
correct the errors
which have cost so much time (for me, and to a lesser
extent for PRL),
and which if uncorrected have the potential of causing
others much more
time. ...
Stephen Parrott "
I was astounded when the Editor's
reply contained the following:
" ...
I should tell you that Dr. ["D"] wrote me a concerned
note about your
letter to ["S"].
I have no plans to intervene between you and them.
But you should know
that a paragraph beginning "You have a small
window of time. .
." might be viewed by anyone as a threat, no matter
how well meaning
you intended to be. If you wish more free advice, I
would suggest you
refrain from any further such 'advice' to ["S"],
and avoid any inferences
about behavior."
The sneering tone seems unprofessional, to say the
least.
The judgment
that I had somehow made an improper threat is
beyond my comprehension.
Had I said "If you do not submit an erratum,
I will place a counterexample
in the arXiv", that could possibly be considered
a threat, though not
an improper one. But I did not even say that.
I simply informed
"S" that I would place the counterexample in the arXiv
within a few days.
I would have done so no matter what they did,
and never did I intimate
otherwise. They were free to do what they wished
with this information.
Were I in their position, I would be grateful
for this additional
information.
For the editor to issue
any value judgment on
my letter to "S"
was unprofessional.
I had not sent him a copy; his response was based
entirely on whatever
"D" had sent him, and he had no way to verify
its accuracy or completeness.
That letter was private correspondence
which was entirely
outside the purview of PRL. It was this Editor's
response
that made me realize
that I might be dealing with editorial bias,
apart from the Editor's
previously evident lack of concern about the integrity
of what PRL publishes.
For reference, I reprint the relevant parts of
my letter to "S", including
the paragraph which the Editor unaccountably characterized as a "threat":
Dear Mr. "S":
I have found a counterexample to [SAD]'s assertion that (6) implies
(7) for positive measurement operators, with the ``weak limit'' defined
by my (105). I am fine-tuning the writeup and expect to submit it
to the
arXiv in a few days.
...
You have a small window of time, almost certainly no more than
a week, and perhaps as little as a day or two, to submit an erratum
to PRL
before my counterexample goes public in the arXiv, and perhaps a
few days
more before my Comment paper is resubmitted.
You may not be able to persuade [SAD] to do this because the other
members of the group have far less to lose than you. In that case,
you may
want to consider the possiblity of doing it yourself. Since you
have not
seen my counterexample and have no way to know if it is correct,
you would need only say that since publication of [SAD], a gap in its
proof
that (6) implies (7) in the "minimal disturbance limit" has been
discovered,
and you no longer make that claim. After the counterexample appears
in the arXiv, you would almost certainly have to mention it.
...
Sincerely yours,
Stephen Parrott
Returning to the main narrative,
recall that I have just posted
my counterexample
in the arXiv and submitted a third "Comment" to PRL including
a reference to this
counterexample which is worked out in detail in the
arXiv.
This "Comment" was
rejected because the counterexample was not included
in the Comment itself
(which would have been impossible because
the counterexample
was too long for the 1-page limit for a "Comment").
I then refined the counterexample so that its essence
could
be described in one
page and verified mentally, and rewrote and submitted
a
fourth attempt at a "Comment" to
PRL. `
The authors were asked to reply, and a letter from
"S" and "D"
(but perhaps significantly,
not signed by "A")
said that they could
not adequately do so in the 1-page limit
for "Reply" papers,
but that they had posted a more lengthy reply in
a new arXiv paper.
The Editor responded that unless I could refute
the
criticisms in that
arXiv paper, the Comment would be rejected.
[From SD's reply letter to
PRL ]
"... we
have now posted a paper on the arXiv [citation] devoted
entirely
to this issue [the
counterexample submitted in the Comment]. Just
as
Dr. Parrott cannot
put his objection into a one page summary, we cannot
put our reply into
one page. ...
In our new paper
that can be taken as a reply to Dr. Parrott's latest
comment we do the
following:
1) Review the basics
of the contextual value formalism.
2) Analyze Dr. Parrott's
counterexample and demonstrate that it does
not satisfy our pseudo-inverse
prescription for constructing the contextual values.
...
4) Give a discussion
of the pseudo-inverse prescription and discuss why
it is both mathematically
and physically important."
...
[signed "S" and "D"]
SD's arXiv reply
will henceforth be abbreviated as SD1, to distinguish
it from later papers
of SD.
The essence of the
Editor's response was:
" ... We would be willing to reconsider your Comment further only
if you can
make a convincing
case in one PRL page that the original Letter was
fundamentally
flawed in light of all the information available. Given
the extensive
postings by you and ["S"] et al, it is hard to see how
that could
be accomplished. ...
In other words, we
can reconsider your Comment only if you can rebut
all of the criticism of ["S"] et al reflected in their
response via
minor modifications to your current Comment.
[signed by the Editor] "
Recall that the stated reason for rejecting
my third "Comment"
submission was that
it referred to a counterexample fully worked out
in the arXiv, but
too long for a 1-page Comment. However, the Editor
is perfectly willing
to accept at face value SD's claim to have refuted
the counterexample
in an arXiv paper. This reinforced a growing
suspicion
that editorial bias
might be playing a role in my efforts to inject
some genuine peer
review into PRL.
Item 2) of SD's reply injects a new element into the
controversy.
They now claim that
SAD specifies a new hypothesis, their so-called
"pseudo-inverse prescription"
(which the counterexample of the Comment
does not satisfy,
though a later counterexample does). Before examining
this new claim, I
must say a bit more about the logical structure
of the SAD paper.
First, it defines "contextual values", which it notes
are
not necessarily unique.
Then SAD states:
"...we propose that the physically
sensible choice of
CV [contexual values] is the least redundant set uniquely
related
to the eigenvalues through the Moore-Penrose pseudoinverse."
[The strange
language (e.g. "least redundant
set") probably carries meaning only to the authors.
Subsequent
developments made clear what the authors actually
meant. Such
logically meaningless language should never have gotten
past the
referee.]
This is what the above item 2) calls the "pseudo-inverse"
prescription,
and the quote just
given is the only substantive reference
to it in SAD.
(A following paragraph
describes a method of computing the
Moore-Penrose pseudoinverse,
but this is standard mathematical material which
has nothing to do
with the pseudo-inverse prescription.)
No explanation is
given as to why this should be the "physically sensible
choice", and no indication
is given that this should be a hypothesis for
further statements
in the paper.
Then SAD obtains what it characterizes as its "main
result",
a so-called "conditioned
average" given by its equation (6). The
pseudo-inverse prescription
is not necessary for the derivation
of (6).
Next follows a section entitled "Weak values" which
attempts
to justify the only
nontrivial claim of its abstract, namely that its
"general conditioned
average" "converges uniquely to the quantum weak
value
[SAD's equation (7)]
in the minimal disturbance limit". This section
does
not mention the pseudo-inverse
prescription.
How can a reader possibly guess that the "pseudo-inverse
prescription",
introduced in such
a vague way above, is to be taken as a hypothesis
for
(7) but not for (6)?
This should never have gotten past a referee who
read
the paper with understanding.
Even if one accepts the questionable claim that
the pseudo-inverse
prescription should be taken as a hypothesis for (7),
it seems unquestionable
that the SAD paper does not clearly state this.
Is this not worth
a "Comment", to prevent future readers from being misled?
When I posted the counterexample to (7), I did consider
whether
the pseudo-inverse
prescription might possibly be a hypothesis, but
rejected
the possibility because
the incorrect proofs which SAD had sent me in February
did not require the pseudo-inverse prescription
as a hypothesis. So,
I knew to a certainty
that when SAD was submitted, the authors did not
assume the pseudo-inverse
prescription as a hypothesis for (7). They
added
it later only after
they realized that their initial proof was wrong.
Morever, when SD finally explained in their arXiv
reply SD1
why they consider
the pseudo-inverse prescription "mathematically and
physically important",
the explanation (which happens to be questionable,
but that's another
issue) has nothing to do with weak values.
It has
nothing to do with
the reasoning of SAD leading to (7).
Their arXiv reply adds
other nontrivial
hypotheses for (7)
which are unequivocally not even mentioned in
SAD.
I hesitate to subject
the reader to these technical details,
but I had to include
them in order to understand what happened next.
The
bottom line is that
their reply to the submitted "Comment" claims that
(7) can be proved
under additional hypotheses not given in SAD, but
seems to pretend
that these additional hypotheses were already clearly
stated in SAD.
Finally, after months of exchanges, PRL sent the Comment
to a referee.
My cover letter for
the first "Comment" submitted back in February,
specifically requested
that if the Comment were to be refereed,
the referee not
be one who had approved the SAD paper.
That was because
it was clear to me that the referee could not
possibly have read
SAD with understanding; if he had, he could never
have
approved it in anything
close to its published form. At a minimum,
he should have insisted
on clear definitions and statements of hypotheses.
It seems obvious
that if a referee were to be consulted, for confidence in
his report,
it should be one who did not have a
vested interest
in justifying his
initial positive recommendation.
Nevertheless, the Editor did send the Comment to
a referee
who had originally
approved SAD. I know this because his report
said so. I
would like to include the report verbatim, but I will
not
because the Editor
specifically asked me not to, which raises copyright
issues.
However, I will paraphrase
parts of it and include a quote which can
be considered "fair
use" under the copyright law.
[Paraphrase
of parts of the referee's report on the "Comment"]
The referee expresses reservations about the SAD
paper and
related work of other
authors. He encourages me to publish my
" findings about the
concept of weak values and contextual values as an ordinary article.
In this respect
I agree with the Editor [emphasis
mine], [name of Editor],
that this would be the more appropriate solution."
The emphasized
phrase, "I agree with the Editor", is troubling. It suggests that the Editor
may have (intentionally or
not) improperly influenced the referee, perhaps subtly
indicating the kind
of report which he hopes for. These editors
have a lot
of power, and should
scrupulously avoid any appearance of "leading" referees.
They should seek
a referee's opinion without interjecting their own.
The report continues:
"It is basically because of the lack of self-containedness
of the
comment, which make
it practically impossible for a reader to gain any
useful insights
concerning the problem. She/He would have to read
additional
manuscripts published as E-prints, as well as Author
replies
and Referee comments, which are obviously inaccessible
to the
readers,
in order to understand the discussion. It is for those
reasons,
that I cannot recommend the publication of the comment
in
Physical
Review Letters."
This is simply wrong.
It is true that a reader unfamiliar with SAD cannot expect
to follow the technical details of the counterexample
given in the Comment.
But it would be completely unreasonable
to demand
that a Comment limited to
only one page should summarize the original
paper sufficiently that a reader unfamiliar
with it could follow the technical details.
Contrary to what the referee says,
a reader who was already familiar with SAD
should be able to grasp the essence
of the counterexample. More importantly, readers
not already familiar with SAD
(which would include almost all readers) might find
it useful to know that
a nontrivial claim of SAD had been credibly questioned.
They would find it even more useful to know that
an independent
referee had checked
the counterexample, if he had. (The counterexample
was
extremely simple,
involving only 2x2 diagonal matrices.) Unfortunately,
this referee ignored
the issue of the correctness of the counterexample.
But that is the essential
issue! If the counterexample is incorrect,
then obviously PRL
should not publish it in a Comment. If it is
correct,
then PRL should have
a professional obligation to inform readers that
a published claim has turned out to be false.
Because the referee's report ignored the main issue,
it is obviously
deficient.
It should never have been accepted by a conscientious
editor.
He should have consulted
another referee at that point.
Recall that the Editor had previously written:
"We would be willing to reconsider your Comment
further only if you can
make a convincing
case in one PRL page that the original Letter was
fundamentally
flawed in light of all the information available."
The mathematics of the Comment's counterexample
has never been
questioned, even
by the authors. The only issue is whether SAD
clearly states
the pseudo-inverse
prescription (along with the other strong hypotheses
added later in SD1)
as a hypothesis for its nontrivial claim (7).
The referee either
did not recognize or ignored this crucial issue.
If a referee ruled
that SAD did not clearly state these additional
assumptions as hypotheses
for (7), then would not this constitute
clear evidence that
the SAD paper was "fundamentally flawed".
Why did the Editor
not insist that the referee address this crucial
point?
Or can it be that
PRL's standards do not preclude publishing claims
that require very
strong, unstated hypotheses to prove?
I was dissatisfied not only with this outcome, but
more importantly,
the way that it was obtained. In view of the
intemperate remarks
of the Editor noted above, I wondered if editorial
bias might be playing
a role. So, I decided to appeal, mainly to
determine the true standards of PRL. The result
truly surprised me,
based on my previous experiences with PRL,which
had involved only this Editor.
The appeal went to a Divisional Associate Editor (DAE),
who had the
integrity to sign
his name (which of course I won't reveal). It
is my
impression that Divisional
Associate Editors are not APS employees like
the Editor who handled
the Comment, but volunteers somewhat analogous
to referees for papers.
Based on the previous superficial treatment
by one Editor, I was very surprised
when this DAE submitted
a report which made clear that he had spent
a lot of time and
effort trying to understand the substance of the matter.
It also made me realize
how difficult the mathematical issues (which seem
almost trivial to
me) could appear to a competent person not already
intimately familiar
with the problem.
His report is lengthy and technical,
and I would not reproduce
it in full even if
there were not copyright issues. But I can
see no
harm in quoting the
first few paragraphs, which only show the integrity
of this individual:
"Under normal
circumstances when dealing with an appeal, I would read
the manuscript first so as to form an independent
opinion of it. Doing
so on this
occasion (and also reading the original manuscript, of
course)
did not give me a clear understanding of the issues and
in no
way prepared
me for the 70 pages of correspondence, which was enough
to make
my head spin.
In all my
time as a board member and editor for four journals,
this
case has
been the most stamina-sapping. I have tried very hard
to
understand
the problem here. I have read the original paper and
the
comment
(both several times) and also referred to the arXiv papers.
I
have not
published research explicitly on weak measurements, but
I
have followed
the field since its inception and one thing must,
I
think, be
quite clear: If I, as an informed even specialist reader,
have had
this much difficulty then there is no way that our general
readership
will be able to follow the subtleties by reading a one
page
comment
and reply.
If the above
paragraph presents what must be my recommendation, it
does not
suffice as a report and (understandably) would not satisfy
the author.
I felt that it was necessary to try to understand the
roots of
the problem that seems to have been the source of so much
acrimony.
Most of my time, and the remaining parts of this long
report
are devoted
to this task. I do hope that this effort has not been
wasted.
[The report
goes on for about 2 pages to examine technical details.
The DAE
makes some mistakes, but with admirable honesty admits
that his
understanding is limited. He concludes:]
"My suspicion
(I dare state it no more strongly than that) is that
it is here
[referring
to a previous statement] that the problem lies."
Before commenting
on this, I want to make clear that
I greatly appreciate
the time and effort that the DAE devoted to this.
Up to now, my dealings
with PRL (i.e., with one particular Editor) had
given no hint that
PRL had the least interest in correcting errors
in what it had published.
The DAE's report shows integrity, even where
it may falter on
technical details. However the following criticisms
should not go unnoted.
(1)
The DAE is right that readers not already familiar with
the SAD paper (which
is probably almost all readers) will not be able
to follow the technical
details of the counterexample.
This is normal.
I do not recall ever understanding a PRL "Comment"
for a paper which
I had not previously read in detail.
But that does not imply that such a Comment
is useless
and unfit for publication.
Its utility lies in alerting the reader that
a nontrivial claim
of SAD is credibly questioned. It would
contribute
to the reliability
of the literature by deterring those who might be
tempted
to build on the claim
without checking its alleged proof. It would
contribute
to scientific progress
by not deterring further work on the claim
(as eloquently stated
in the above statement of ethics of the American
Mathematical Society).
If the policy of PRL is that a large fraction of readers
must be able
to follow all "Comment"s
in full technical detail, then almost no "Comment"s
should be published.
Such an insistence seems close to a double standard,
a pretext for summarily
settling an inconvenient matter which no editor fully
understands.
(2) The DAE
does not address the important issue that SD's attempted
arXiv proof of SAD's
claim requires important, non-trivial hypotheses
not stated in SAD. One of these is the
pseudo-inverse prescription,
which is at least
mentioned though not clearly stated as a hypothesis
for
anything, but there
are others as well. Even if SD's arXiv proof
were correct
(and I don't think
it is), it would not prove the claim as stated in
SAD.
Can it be that the
policy of PRL is to allow misleading claims which
can be proved, if
at all, only under strong and complicated additional
hypotheses?
Would the "premier" journal PRL have initially published
SAD
had it realized this?
If not, should it not feel an obligation to correct
misleading claims
which were mistakenly published?
The gist of the
Editor's final rejection of the Comment was:
"Dear Dr. Parrott,
...
The DAE
delves into the details in an effort to understand the
dispute,
but the second paragraph of the report presents a sufficient
case for
final rejection of your Comment. The advice is in agreement
with what
we have been trying to communicate to you--that a Comment
is
not the
right vehicle for your results.
Thus, the
scientific review of your Comment has come to an end.
Yours sincerely,
[Signed
by the Editor who has handled all my submitted "Comments"
on SAD]
Editor
Physical
Review Letters
The premier
APS journal for current research"
This seems hypocritical,
perhaps deliberately so.
I am sure that the
Editor is fully aware that it would be impossible
to publish the counterexample
anywhere other than PRL.
The counterexample
has no mathematical interest. Its only interest
is
to prevent SAD from
misleading further readers, and that is the responsibility
of "the premier APS
journal" PRL which published SAD, not some other journal.
Summary and Conclusions
The above is so long and convoluted that possibly no one
will read it in detail,
but I felt that I should put forward all
the evidence. Following is a summary.
Recall that "S",
"A", and "D" are pseudoinitials referring to the three authors of
a 2010 paper in Physical Review Letters. The paper
is called SAD..
What happened
(1) SAD made a claim (to be called the "CLAIM"
below) which turned out
to be demonstrably false as stated in SAD.
(Two of the authors later
maintained that the claim could be proved under strong
hypotheses not
mentioned in SAD.)
It was not initially known that the claim was false,
but it was
known that the authors had no proof.
There was also a serious, demonstrable mathematical
error in SAD (to
be called the ERROR below), among lesser errors.
(2) The authors refused private (and civil) requests
for a valid proof
of the CLAIM, and even refused to furnish necessary definitions.
(This was after an initial response of SAD to a proof
request had furnished
an incorrect proof.)
(3) PRL rejected a "Comment" paper pointing out
the ERROR,
for stated reasons
other than the ERROR, and without mentioning the
ERROR.
The impression given
is that PRL considers the ERROR unimportant.
(4) PRL rejected a "Comment" paper which both pointed
out the ERROR and
gave a reference
to a counterexample to the CLAIM. The reference
was to
an arXiv paper which
fully worked out the counterexample. The mathematics
of this counterexample
has never been questioned, even by the authors
of SAD.
The stated reason
for the rejection was that the original counterexample
was
too long to include
in the 1-page "Comment" itself. The ERROR was
again
ignored.
(5) A revised "Comment" which did include a counterexample
was then rejected.
The stated reason
was that general readers (presumably, ones not familiar
with the original
SAD paper) would not be able to follow the technical
details
of the Comment.
(6) Although the paper was sent to a referee, it was
a referee who
had initially approved
publication of SAD and thus had a vested interest
in justifying his
initial recommendation. (I had specifically
requested
that it be sent to
an independent referee as well.) The referee
ignored
the issue of the
validity of the counterexample, which one would think
would be the main
issue!
Conclusions
(1) If the above is typical of the practices of PRL, then
it cannot be
considered a reliable
journal because it makes it almost impossible
to correct errors
in what it publishes. Had I realized the ultimate effort
involved, I might
never have started. Were I not retired, I could
never
have afforded the
time. According to a Divisional Associate Editor,
the PRL file (most
of which was probably written
by me) is about
70 pages.
(2) There is an outside possibility that the
handling of these "Comment"
submissions may not
reflect the policy of PRL, but might be an artifact
of
practices of a particular
editor.
Evidence for this possibility includes
intemperate
remarks of this editor,
including insulting personal comments unrelated
to the "Comment"
or PRL. (The Editor later gave a half-hearted
apology,
"Sorry".) Also,
there is evidence that the Editor may have
(intentionally or
not) "led" the referee by indications of his personal
views
of the matter.
Further, the Editor chose a referee who had a vested
interest
in justifying his
previous positive recommendation for publication of
SAD,
despite my request
that the "Comment" also be sent to an independent
referee.
Evidence against this possibility includes
the concurrence of
a Divisional Associate
Editor to whom the matter was appealed. If
he saw
anything wrong with
the Editor's conduct, he did not mention it.
(3) I find it very disturbing that PRL seemed to be
actively enabling the
unprofessional behavior
of the SAD group. They had refused to furnish
a proof of the major
CLAIM of the PRL publication SAD, and had even refused
to state a definition
relevant to the CLAIM. They had ignored all
questions about the
SAD paper since February (when I had sent them a
counterexample).
The Editor seemed to find these "stonewall" tactics
perfectly acceptable.
Had he requested
the SAD group to provide minimal cooperation toward
coming
to agreement on what
SAD claimed and what it actually proved, the whole
matter might have
been resolved informally. At a minimum, the issues
would
have been more clearly
delineated and much time would have been saved.
The Editor acted
more as an advocate for SAD than as an impartial judge
using
all means at his
disposal to determine the facts of the matter.
(4) It is easy to lose sight of the forest for the trees,
as the saying goes.
The welter of detail
above is enough to confuse anybody, as the DAE's
report
indicated.
Let us step back and take a commonsense view.
PRL has never disputed the credibility of my objections
to the SAD
paper. The
referee did not dispute it. What can be a possible
justification
for "Comment" policies
which preclude alerting potential readers that SAD
is credibly disputed?
I know of no other publication which forbids expressions
of opinion
in "Comments" (more
usually called Letters to the Editor) even though
space
may preclude definitive
proof of what the letter-writer opines. For
example,
if some article draws
some conclusion about global warming based on a computer
model, a letter expressing
doubts about the assumptions of the model
would normally be
acceptable. If the letter-writer is a known
expert on
global warming, the
knowledge that the model is disputed can be useful
to
readers even if a
reader cannot definitively judge the details of the
dispute.
The narrow and rigid PRL policy that a "Comment"
must
"make a convincing
case in one PRL page that the original Letter was
fundamentally flawed"
will preclude most expressions of doubt and is
a disservice to potential
readers of the Letter. A journal which touts
itself as "the world's
foremost physics letters journal" and "the premier
APS journal for current
research" ought to enable, rather than suppress,
genuine peer review.