Meine Studie wird publiziert – oder warte – nein, doch nicht

Ich bin im Moment echt sprachlos, daher stelle ich einfach mal den eMail-Verlauf hier rein, der mit dem letzten Journal ablief. Ich kürze da, wo es uninteressant ist. Ein früher Preprint meiner Studie ist hier zu finden, wenn sie dort etwas runter scrollen finden sie eine deutsche Zusammenfassung.

---

Bitte um Revision

16-Sep-2021

Dear Mr. Fögen,

Thank you for submitting your article to [XXX].

The manuscript has now been reviewed by me and expert referees. The reviewers’ comments are listed below.

While we all found the work to be quite interesting, we think that it would benefit from a revision. Therefore I’m inviting you to respond to the reviewer comments and submit a revised version of your article.

[…]
_____________________________

Reviewer Comments to Author:

Reviewer: 1

Comments to the Author
This is a retrospective study (secondary data analysis) aims at assessing the influence of mask mandates on case fatality rate (CFR) by comparing the CFR between two groups, one with and the other without mask mandates.
The subject is very important to clinical practice. The paper is well-written and provides useful information for the readers.
There are however some additional information needed meaning editing and revision of the manuscript:
[…]

Reviewer: 2

Comments to the Author
Dear author,

This was a very interesting study with a highly controversial finding. Although we could not find any major flaws from a methodological point-of-view, since it is so controversial, we have some suggestions to strengthen the analysis in your manuscript. We have also recommended it be sent out for additional statistical review.
[…]
Thank you for the opportunity to review your manuscript. We found it to be an interesting study, but due to the highly controversial finding, we feel additional analysis is needed, and have recommended a statistician to the editors. But, we congratulate you on investigating a potentially serious public health finding.

Editor Comments to Author:
[…]

---

Als Publikation angenommen

29-Sep-2021

Dear Mr. Fögen,

Thank you for submitting your revised manuscript to [XXX]. I very much enjoyed reading the revision.
I’ve reviewed your revised manuscript and your response to my comments and those of the reviewers, and I think all of the concerns raised have been satisfactorily addressed. I am thereby pleased to inform you that your manuscript has now been accepted for publication in [XXX].
While your submission has been accepted, the files will now be checked to ensure that we have everything in place for publication. The editorial office will return the manuscript to your Author’s dashboard within 2 working days for you to make the final adjustments before proceeding with production. Once you have completed uploading the final files, we will proceed with production.
[…]

Congratulations! I hope your experience with us was a positive one, and the reviews and editorial comments were useful and fair. I also hope you will consider us for future articles. Any final comments made by the referees are included below.

Thank you again for choosing [XXX] for publishing your work.

[…]

---

---

Warum die Verzögerung?

Dear Mr. Fögen,

I’m sorry to the delaying of your manuscript. According to the suggestion of the reviewers, we need to wait the opinion of our statistics advisory Board to make the final decision.

Thanks for submitting your work to [XXX].

---

Als Publikation abgelehnt

21-Oct-2021

Dear Mr. Fögen,

Thank you for submitting your article to [XXX]. The reviewer comments are attached below.

Unfortunately, on the basis of the reviewer comments, we are not able to accept it for publication. We therefore recommend that you submit your article to another journal and we hope that the reviewer comments will be useful in revising your article.
[…]
_____________________________

Reviewer: 3

Comments:
This is an interesting study and well written, however, the „foegen effect“ as described is very controversial during the time of pandemic when all countries are going after mandatory use of masks in many countries. This paper gives controversial findings just opposite to the public health policy to use masks to prevent respiratory-related infections. This paper is inviting further studies on a larger scale to duplicate or to validate the findings “ The Foegen Effect“.
Another thing is; I could not understand the calculations and the analysis part. So I recommend authors crosscheck the analysis part with the statistician once before resubmitting it to the journal.
Finally, I would like to congratulate the authors for coming up with the new findings in the field of public health intervention to the pandemics.
Thank you

Reviewer: 4

[…]
I am fond of out-of-the-box argumentation

I do not think that the proposal in this paper is impossible.
I think this idea would deserve a serious consideration.

However, I am not at all convinced with the presentation in the manuscript.

Proposing title “The Foegen effect” is narsistic

Why was Kansas selected?
Did the author go through all 50 states and Kansas was the only one that gave a positive result?
I would expect some justification.


In many stages we do not have any understanding what we are actually comparing

In RCTs we know that one group is subjected to intervention A, and another to intervention B (eg control), but here the comparison is ambiguous.
The outcome is defined so that it is uniform over all the trial, here we do not know what is the potential variation in the classification of COVID deaths in different counties


#################
105 counties were categorized into counties with mask mandate (MMC) and counties without mask mandate (noMMC). Further, the noMMC group was evaluated to identify cities with mask mandates5 to assess the percentage of the county population6 that was represented by these cities7.
If the city’s population was within +/-20% of half of the county’s population (that is, between 30% and 70%), the county was excluded. If the city’s population represented more than 70% of the county, it was moved to the MMC group. If the city’s population represented less than 30% of the county, the counties remained in the noMMC group
#################

My question about the above description is that what do we end with?
If the city has 29% or 71% it is not excluded, but confuses the comparison.

The author does not consider at all what are the possible/probable factors that lead a county to set a mask mandate or not. Setting or not setting a mask mandate is not at all a random process.
If there are more old and sick people that is one justification for politicians to require a mask.
If there are old and sick people, but the population is less educated and believe in conspiracies the politicians may consider that it is better for them not to set a mask requirement.
Because the political process that leads or not to a mask mandate is very complex with lots of issues that are taken into account (not just risk for severe COVID), there is potential for serious systematic biases in this comparison.
I do not consider that the potential biases are discussed properly in Discussion, rather I consider that they are not discussed at all.



#################
the CDR of each county for 2019 was modified by subtracting deaths from causes that are
clearly not a risk factor for COVID-19 to prevent statistical anomalies when comparing CDR, like deaths from other causes that are related to neither old age nor pre-existing illness. These included pregnancy complications, birth defects, conditions of the perinatal period (early infancy), sudden infant death syndrome, motor vehicle accidents, all other accidents and adverse effects, suicide, homicide, and other external causes8
#################

I dont think this is reasonable.
It gives to me the same question as the selection of Kansas. Why Kansas?
Are those death causes excluded, because if they are included, the difference would disappear?
There are numerous causes of death that are “clearly not a risk factor for COVID” but such exclusions is a problem. There is much room to game in order to twist the data to fit the hypothesis.


“Parallelization”

#################
There were two ways in order to get almost the same mCDR in both groups:
A) Removing counties with the hightest mCDR in the group with a higher mCDR until both groups had the same mCDR: Configuration A.
B) Removing counties with the lowest mCDR in the group with a lower mCDR until both groups had the same mCDR: Configuration B.
Therefore, cut-off limits of mCDR were used in an attempt to reduce the mCDR difference while trying to include the largest percentage of the eligible Kansas population
#################


The author justifies this as if this takes into account “age and pre-existing illness in the underlying population”.

In our city, there is a less-well-to-do neighborhood where life expectancy of men is some 10 years less than in the best-well-to-do neighborhood.
Age does influence mortality, but there are many other factors also. Smoking, alcohol, lack of any exercise, obesity, less education, etc etc. A physician who smokes, lives on average longer than a factory worker who does not smoke, etc

In my view it is an unjustified assumption that cutting out the counties with highest or lowest CDR levels would lead to comparable groups. There are very many kinds of differences between counties. The author does not give any arguments why we should believe that the resulting sets are comparable.

#######
Table 1
#######

Table 1 shows number of deaths, but does not explicitly state that they are deaths for COVID

241 in MMC and 95 in noMMC

As far as I have been reading literature, there is much inconsistency in the classifications. Does a patient die because of coronavirus, or does the patient die with the coronavirus.
Such decisions are sometimes subjective decisions by physicians, and there can be variations between counties in such views.
I recollect that Trump administration tried to order that the COVID diagnoses should be avoided – in order to hide the magnitude of the COVID problem. In analogy, there can be variations between counties because of variations in the political views in the community. Such political views in the community not just influences whether masks are mandated, but may also influence, where is the limit between „because of“ and „with“ in the above sentence.
The author does not discuss at all the problem of the outcome.


#################
To correct for the CFR outlier of Gove County, the number of deaths in Gove County was reduced
from 13 to 3, as marked by the lowercase ‘G’.
Furthermore, a sensitivity analysis was performed by excluding counties without a mask mandate
that had counties with a mask mandate, as shown in Table S4, which did confirm the prior results
#################

This is very odd. What is the justification to reduce from 13 to 3.

“ excluding counties without a mask mandate that had counties with a mask mandate”

What does that mean?


In summary,

I dont think that the MMC and noMMC groups are defined well enough so that we would know what we are actually comparing

I dont think that “parallelization” is any solution to the great potential biases between the MMC and noMMC counties

I am not convinced that the 241 and 95 are based on procedures that are the same diagnostically over all the counties.

In the Discussion section the author instructs what some other researchers should do, instead of considering the validity of the parallelization, COVID death diagnoses, etc


%%%%%%%%%%%%%%

As noted above, I like novel ideas. However, when the data are seriously flawed and the hypothesis – if true – would have great political consequences, one needs to be very cautious.

Why does the author work alone?
I would think that collaboration would have given him many of my comments as feedback from colleagues.

---

Mein Fazit

überlasse ich heute mal den Lesern, hier oder auf Twitter @ZachariasFoegen. Wenn ihr auf dem laufenden bleiben wollte, folgt mir auf Twitter oder abonniert meinen Blog.