More Data, Less Understanding: An XPS Case Study

In various debates, Aristotle, Leonardo da Vinci, and Leibniz are often described as the last men who mastered all the knowledge of their time. Today, however, we are confronted with such an overwhelming amount of information that it is no longer within the power of a single person to comprehend it all. As society develops, we naturally lose comprehensive expertise within individual disciplines.

There is simply too much information for everyone to know everything. Likewise, we have inevitably reached a point where it is impossible for every scientist to fully understand and master all scientific approaches and methods used in a given field. And this is precisely where some of the errors in scientific literature originate.

X-ray Photoelectron Spectroscopy (XPS) as a Case Study

To illustrate this phenomenon, let us focus on a method commonly used in materials science: X-ray Photoelectron Spectroscopy (XPS). XPS is a technique that provides information about the chemical composition of samples, including the chemical states in which individual atoms are found. Simply put, it tells us which elements are present in a sample and how they are chemically bonded to one another.

The method is based on the internal photoelectric effect, in which electrons are emitted from atoms as a result of absorbing X-ray photons. From the measured kinetic energy of the emitted electrons and the known energy of the photons, we can determine the binding energy of the electrons in the atoms. This binding energy is characteristic of each element and shifts slightly depending on how the atom is bonded to its surroundings. In the measured spectrum, each chemical state corresponds to a peak at a specific energy. For example, if we analyse PET polymer chains using XPS, we discover not only that carbon and oxygen are present in the sample, but also that there are two types of oxygen atoms in a 1:1 ratio and three types of carbon atoms in a 1:1:3 ratio. Based on known information about the energies of individual bonds, we then assign the observed peaks to the bonds present in PET.

CEITEC/Jan Čechal: More Data, Less Understanding: An XPS Case Study: X-Ray Photoelectron Spectroscopy (XPS) at CEITEC BUT.

The Majority of Spectra Is Misinterpreted

XPS is a widely used method that, especially when combined with other surface analysis techniques, provides highly valuable data about sample composition. Today, XPS spectrometers are standard equipment wherever materials research is conducted. After brief training, most users can operate the spectrometers and measure their own spectra. So, where is the problem?

In 2020, a study was published showing that approximately 60% of XPS spectra in scientific journals were interpreted incorrectly, or at least, questionably. The conclusions were based on the examination and evaluation of XPS analyses published over a six-month period in 2019 across three different journals. The first had a narrow disciplinary focus (IF 25), the second focused on materials surface analysis, and the third was a general scientific journal containing a large number of publications in materials science. The results were alarming: on average, 60% of the spectra had been interpreted incorrectly.

Modeling the Spectra, i.e. Fitting, Increased the Error Rate

When only the measured spectra were published and conclusions were drawn directly from them, the situation was not quite so severe. However, this changed dramatically once spectral modelling (so-called fitting) was used. Through fitting, we attempt to obtain more detailed information when spectral peaks have complex shapes caused by the presence of multiple chemical states for each element. Here, the error rate reached 80%. Roughly 40% of these were very serious errors that essentially invalidated the conclusions of the published XPS analyses. And we are talking about respectable scientific journals. Sixty percent incorrectly interpreted XPS measurements is not a negligible problem.

In response to this study, my colleagues and I prepared a publication examining the future of science in light of these findings, the causes of the current situation, and possible ways to remedy it. 

A deeper examination of incorrectly interpreted spectra quite clearly shows that most of the mistakes could have been easily avoided. In essence, only a basic understanding of XPS’s fundamental principles was missing. Spectral modelling reflects the process of photoemission and thus must follow its physical principles. Many spectral interpretations contradict quantum mechanics (and some authors even acknowledge this themselves in their papers), while others are simply inconsistent. In many cases, the process resembles drawing curves like those seen in the literature, rather than a well-founded interpretation of measured data. 

The Invention of New Errors

So is the problem simply that mistakes spread from one publication to another? Initially, we thought so. However, a subsequent study showed otherwise. In incorrectly interpreted analyses that cited a single prominent example, the problem was usually not the reproduction of specific mistakes, but the invention of entirely new ones. Essentially, the goal was merely to include a citation from a high-impact journal that appeared to support the claims made in the article. Authors were therefore not copying particular errors, but citing superficially similar work, without understanding the context.

Is this a manifestation of the often-discussed crisis of science? Not yet. We are still in a pre-crisis stage. We could speak of a true crisis, were the entire body of scientific literature to become worthless because we were no longer able to determine what is true and what is not. We are not there yet. However, the results of the above-mentioned studies 1suggest that we are heading in that direction.

Everything begins with scientists themselves. To avoid such a crisis, efforts must be made on many fronts. In the past, XPS analysis was the domain of specialists. Today, however, it is used by scientists and students from many different backgrounds, who are often trained only to operate the spectrometer, not to interpret the measured data. For them, it is just one of many methods used to assemble the mosaic of a new publication. But do they have enough time to deeply understand every method they use, when they are expected to publish, and above all, to publish quickly?

Am I Sure that what I write Is True?

We must pay close attention to education: scientists need to be taught how to properly work with XPS data. To this end, we have written a summary of the most common mistakes and how to avoid them. But papers alone are not enough. Last year, therefore, I began offering a two-day course at CEITEC BUT, in which I primarily teach the physics and chemistry underlying the method, followed only afterward by mathematical modelling, while also highlighting the most common sources of error. Since we live in a fast-paced world, I also prepared a short text summarizing the basics of XPS in 20 minutes. Because even 20 minutes can prevent most of the fundamental mistakes in spectral interpretation.

Unfortunately, this is not just an XPS story. Similar situations may exist in other analytical methods and in other scientific disciplines as well. So what can we do about it? I believe the most important step is a change in mindset. Every (co-)author should honestly ask themselves before submitting a publication – and answer honestly: “Am I sure that what we are writing is true?”

CEITEC/Jakub Rozboud: More Data, Less Understanding: An XPS Case Study: Jan Čechal, CEITEC Brno University of Technology.