Useful or nonsense – can ChatGPT get to the bottom of scientific publications?
von Réka Enz
Recently, there has been much buzz surrounding ChatGPT and its capabilities in both positive and negative contexts. To put its abilities to the test, we decided to compare ChatGPT's performance against our in-house product specialist Jonas Schäfer in summarizing a scientific paper. The paper, by Petteri Parkkila et al.1, focuses on MP-SPR technology.
We provided ChatGPT with the abstract of the paper and tasked it with summarizing the key highlights and the utilization of MP-SPR technology. However, the process was not without its challenges, as the AI required precise instructions to produce the desired results. As ChatGPT lacks web access, it cannot independently read entire research papers.
Despite these limitations, ChatGPT was able to impress us with its ability to succinctly summarize the paper based solely on its abstract and title. We had our product specialist proof-read the summary to make sure no invalid information was provided.
This scientific paper presents a protocol for the characterization of nanoparticles (NPs) and their subpopulations in heterogeneous samples using a Protein A/G-functionalized surface plasmon resonance biosensor. The protocol uses anti-streptavidin antibodies to regenerate the capture of biotin-containing NPs and antibodies directed against glycophorin A and B to capture red blood cell-derived extracellular vesicles (RBC EVs) with specific and efficient surface capture. The study demonstrates the efficacy of the biosensor in determining the size and bulk concentration of NP subpopulations, with comparable results to well-established NP sizing techniques. The Protein A/G-functionalized biosensor is presented as a generic alternative to existing NP-capturing sensors.
Next, we challenged ChatGPT to produce a more comprehensive and extended synopsis of the research presented in the paper. To our surprise, it performed quite well, presenting the information accurately and with only a minor deviations. For full transparency we indicated these deviations in the text:
This scientific paper presents a new protocol for the characterization of nanoparticles (NPs) and their subpopulations in from* complex biological samples using a Protein A/G-functionalized surface plasmon resonance biosensor. The precise characterization of NPs and their subpopulations is crucial in various applications, such as the initial design of targeted NP therapies and the different stages of their production cycle. In recent years, biological NPs, such as extracellular vesicles (EVs), have gained attention as they show potential in improving the drug delivery capabilities compared to traditional NP-based therapies, especially in the treatment of cancer and neurodegenerative diseases.
The protocol described in the paper utilizes anti-streptavidin antibodies to regenerate the capture of biotin-containing NPs, such as large unilamellar vesicles commonly used as drug delivery vehicles. Additionally, the use of antibodies directed against glycophorin A and B (CD235a and b) enables diffusion-limited specific surface capture of RBC EVs. The efficacy of the biosensor in determining the size and bulk concentration of NP subpopulations isolated from a complex biological matrix was demonstrated by capturing RBC EVs. The results showed that the mean size of the surface-captured RBC EVs was comparable to the corresponding sizes derived from well-established NP sizing techniques, such as nanoparticle tracking analysis and dynamic light scattering.
Overall, the Protein A/G-functionalized biosensor described in this paper provides a generic and alternative solution to existing NP-capturing sensors that rely on covalent antibody attachment, hydrophobic surfaces, or biotin-capped self-assembled monolayers. This new protocol offers a more efficient and versatile way to accurately characterize NPs and their subpopulations in from complex biological samples.
*The RBC EVs were isolated from complex biological samples, namely leukocyte-reduced erythrocyte concentrates. However, the experiments with the EVs were performed in DPBS after isolation, which is a crucial piece of information.
But now for the interesting part of the story: What is so special about MP-SPR that the authors used this technology for their research? ChatGPT states the following:
The authors used MP-SPR (surface plasmon resonance) technology to study the characterization of nanoparticles (NPs) and their subpopulations in heterogeneous samples. MP-SPR technology allows real-time, label-free and non-invasive monitoring of biomolecular interactions at the surface of a biosensor. By using a Protein A/G-functionalized surface plasmon resonance biosensor, the authors were able to perform the antibody-mediated surface capture and analysis of NPs. The MP-SPR technology provided a way to determine the size and bulk concentration of NP subpopulations in complex biological matrices and compare it to well-established NP sizing techniques.
Well, none of this is really wrong, but what does our product specialist have to say to this?
In their study Parkkila et al. showed that MP-SPR can be used to characterize artificial and biological nanoparticles (LUVs and RBC EVs) with regard to relative abundance of subpopulations and their sizes. For the determination of NP diameter, they utilized the multiple wavelengths detection of MP-SPR.
Their newly developed biosensor system based on Protein A/G coated gold sensor slides is presented as a generic alternative to other immobilization strategies and leads to site-directed immobilization of the specific antibodies.
The most important feature of the Protein A/G-functionalized capturing system is its regenerability in combination with low non-specific binding. These features make size-determination with multi-parametric SPR an efficient alternative to well-established sizing techniques like nanoparticle tracking analysis and dynamic light scattering.
What do we learn from this little experiment?
While AI chatbots like ChatGPT can be a valuable resource and its performance with this example was surprisingly good, it is crucial to always validate its output with human oversight, particularly regarding research. We cannot generalize from a few well written summaries to the overall reliability of the AI. The potential for biases and misinformation, especially when it comes to very specific topics where the AI simply lacks sufficient relevant resources is undeniable. To make sure all the information is correct we cannot do without human specialist validation. Furthermore, it is important to note that ChatGPT does not provide any original sources. So while we know, that most of these topics have probably been addressed in the paper, we will never know who said it first and what was an original thought. Putting this into context with research in general, this surely increases the risk for plagiarism. Despite these limitations, it remains a practical tool that can aid in writing - it surely did with writing this (entire) blogpost. ;)
And for all of you who are now hooked on MP-SPR and its application for nanoparticle capture and analysis can read the entire article here: