100% of academic studies were dismissed on unscientific grounds

The chemicals expert Tony Tweedale of RISK Consultancy has commented on our article reporting the European Food Safety Authority’s conclusion that glyphosate is not an endocrine disruptor. Tweedale has analyzed the industry report on which the EFSA conclusion was based. He tells us that all peer-reviewed studies were excluded from the EFSA evaluation. Yet a significant proportion of these studies point to glyphosate and its commercial formulations having endocrine disruptive effects.

The EFSA evaluation was prompted by the EU Commission asking for an updated literature review of the endocrine disrupting properties of glyphosate. The review was performed by the Knoell consultancy firm for the Glyphosate Task Force (GTF), the coalition of pesticide companies that is trying to get glyphosate re-approved in Europe.

The review found 104 published studies.

Tweedale told GMWatch, “Two were excluded by a full-text evaluation of relevance to the study question. A further five were excluded by a full-text evaluation of reliability. But the remaining 97 (93%) were excluded just by screening the title and/or abstract – in other words, before any evaluation of study quality was done. The number of published studies deemed reliable enough to be a key study, i.e. to set a safe dose or make another risk assessment decision, was precisely zero.”

EFSA failed to challenge this mass disposal of scientific data.

Relevance and reliability defined by industry

Surely “relevance” and “reliability” are desirable qualities in any study used for risk assessment? Well, it depends on who is defining and using the criteria.

The criteria for judging the relevance and reliability of toxicity tests for inclusion in risk assessment were defined by three employees of the chemical company BASF in a paper published in 1997. This paper is known as the Klimisch paper, after the first author.

The Klimisch criteria are recommended to be used in risk assessment by the European Chemicals Agency (ECHA) as well as by EFSA.

Where the Klimisch criteria fail, according to Tweedale, is that they define “reliability” simply as adherence to the test guidelines set by the Organisation for Economic Cooperation and Development (OECD). These guidelines set down protocols that industry must follow in carrying out toxicity studies performed in lab animals for regulatory purposes. They include a requirement that the experiments be performed in accordance with Good Laboratory Practice (GLP), a set of laboratory management rules that were introduced by regulators in the 1970s in response to serious industry fraud in chemicals safety testing.
 
Scientists working outside of the industry generally do not use the OECD’s test guidelines, Tweedale says, because they test high, unrealistic doses and have other insensitivities. So the only studies that adhere to these guidelines are industry studies performed for regulatory purposes. Thus the Klimisch criteria dismiss all non-industry studies as unreliable or reliable only “with restrictions” – less reliable than an OECD-compliant study.

In the case of the glyphosate endocrine disruption review, 99 studies were dismissed using a handful of criteria that are attributes of OECD studies.  The remaining five studies were ruled “not reliable” or reliable only “with restrictions”, by explicitly using a “Klimisch Evaluation”.

Tweedale says, “Overall, Knoell dismissed a few dozen serious findings of glyphosate or formulation endocrine disruption, based largely on the Klimisch criteria – adherence to OECD guidelines and/or guidelines established by the US EPA.”

Tweedale says the problem is not confined to glyphosate but applies to virtually all pesticide re-authorizations. “Some industry reviews fail to find dozens to hundreds of published toxicity findings. To the extent that they do find them, they dismiss almost all before evaluating them, typically on the Klimisch reliability criteria.”

Registrations for chemicals under the EU’s REACH (Registration, Evaluation, Authorisation and Restriction of Chemicals) regulation are just as bad, Tweedale says, with large numbers of peer-reviewed studies being ignored or dismissed on the basis of the Klimisch criteria.

What the regulations say

The EU pesticide regulation requires that peer-reviewed studies from the open scientific literature are taken into account during the pesticide approvals process. It even allows them to be relied upon to ban a pesticide. REACH has the same  requirement to evaluate all available data. Tweedale says, “Both laws have failed, with 75-80% of published findings not even being identified, and every one that is found being dismissed on unscientific grounds.”

While EFSA Guidance states that industry applicants should not dismiss studies as irrelevant just because they are not conducted in accordance with OECD guidelines or GLP, it also recommends Klimisch as a prime tool to assess the quality of studies.

In allowing use of the Klimisch criteria, EFSA and ECHA allow the practice of dismissing studies out-of-hand without full evaluation. We should care, Tweedale says, because “Low dose exposure to chemicals is the norm. But that’s precisely what industry tests fail to examine.”

Less than 1% of studies considered in Canada

Tweedale’s comments come as two academics have accused the Canadian government of ignoring the vast majority of published peer-reviewed studies in its re-approval of glyphosate.

The Canadian government’s Pest Management Regulatory Agency (PMRA) stated, “An evaluation of available scientific information found that products containing glyphosate do not present risks of concern to human health or the environment when used according to the revised label directions.”

Louise Vandelac, professor of environmental sciences and sociology and Marie-Hélène Bacon, research officer at the interdisciplinary research group GRETESS, both at the University of Quebec at Montreal, conducted their own analysis of the published scientific literature to find out how many of the studies were analyzed by the PMRA to reach its decision. They found that the agency took account of “less than 1% of the abundant scientific literature on the health and environmental effects of glyphosate and glyphosate-based herbicides (GBH), published since the early 1970s” – a situation that they describe as “highly problematic”.

Vandelac and Bacon argue that the scientific studies that were considered are old and outdated and that the decision document “essentially consists of confidential documents from 1970 to 1990 originating from the industry, which is the main economic beneficiary of the decision”, all factors that “call into question the scientific rigour and independence of the approach”. They add that the decision to approve glyphosate was therefore made “on the basis of 95 references submitted by companies dating from over 20, 30 or 40 years ago… which, given the rapid evolution of the research on the health impacts of certain chemical substances and chemical cocktails, does not in any way correspond to scientific requirements.”

Moreover, Vandelac and Bacon add, the regulatory process is inadequate as it primarily considers glyphosate alone, even though glyphosate herbicide formulations as sold and used can be up to 1000 times more toxic.

Nonetheless Tweedale says he glimpses light at the end of the tunnel. “Fortunately, systematic review, a concept borrowed from medicine, is challenging the chemicals regulatory system. It consists of a thorough and strict evaluation of all available data, based on objective scientific criteria.

“If this system were applied to pesticides, it would be much harder for industry to exclude the findings of peer-reviewed studies. And public health and the environment would be the winners.”