A couple of weeks ago I started seeing a paper circulating on social media, “Policy reforms for antibiotic use claims in livestock”, about antibiotic use in cattle with final products being sold as “raised without antibiotics” as a claim.

In brief, the study collected urine samples from the bladder of cattle after harvest and evaluated the urine for the presence of antibiotics. The beef was packaged under the label of “Raised without Antibiotics” (RWA) and in the paper, the authors indicate that consumers do not have a means to verify this claim. Generally, the study reports that antibiotic residues were found in the urine from cattle that were produced under RWA claims.

I found this finding intriguing because the paper resulted in a request to FSIS to recall meat having these claims produced during this time frame in this facility. As a scientist, I found it odd that another group of scientists would make that request to the agency such as this, as hypothesis-driven research should not be underpinned with an agenda. A quick look at several elements of the study highlights the fact that while packaged as a “hypothesis-driven” research paper, it is actually an “agenda” or “policy” driven paper with many flaws that I will highlight:

Authors. Two of the authors have an affiliation with the Antibiotic Resistance Action Center at George Washington University (which I might note, is not an agricultural research university), and the third author is with “Food In-Depth,” the testing company that developed the test for antibiotic resistance. This conflict of interest isn’t disclosed in the document. This company has an agenda of having devices for consumers to test food at home to verify label claims on foods. Additionally, much of the data are redacted or withheld to protect this company. Food In Depth funded the study and directed the testing protocols.

Sample type. Urine is an unusual sample type as antibiotics clear from urine relatively quickly. Blood and/or tissue is the typical sample type and there is no justification of the discrepancy. Clearly a consumer consumes tissue, not urine. While the presence of antibiotic residues could be relevant, there should be more information on the disconnect between the standard sample types.

Sample collection. There is no mention of blinding on the sample types nor are there any internal positive or negative controls to ensure the accuracy of findings. Bias can be imposed in baseline studies without these control elements.

Testing methodology. The presence of antibiotics was determined using the “Food In-Depth” immunoassay test in a lateral flow. Typically, antibiotic testing is done by Mass Spectrophotometry, so I searched for the validation and verification studies for this type of analysis. They even claim that they can get a “semi-quantitative” result by uploading the data into the cloud. There is some information on the “Food In-Depth” website about comparing data to mass spec and in testing in various matrices. However, the methods are not listed as being AOAC or third-party approved or validated using acceptable standards. After reaching out to testing labs as well as other testing companies, it was verified that this methodology has not been fully vetted and is certainly not valid to make policy recommendations as the data could be fully inaccurate.

I acknowledge that there could be valuable information presented in this publication. However, one is unable to determine if the data are legitimate nor should any action be taken based on these findings because the methods used are not validated.

Additionally, the paper was agenda-based, not hypothesis-based from the beginning. Time, money and resources were spent on this study and it is unfortunate the authors did not follow sound scientific principles. What is especially concerning is the fact that the “data” are presented to be legitimate. It looks legitimate, but there are numbers masquerading as legitimate science.

In order to make data-driven and science-based decisions, we must hold high scientific standards and let unbiased, hypothesis-driven research drive our policies. This approach is how we impact public health and ensure a safe food supply.

When numbers masquerade as science

Join for free* to continue reading.