Background: Advancements in natural language processing have led to the development of powerful language models like ChatGPT, capable of generating coherent text with minimal human input. While the potential for using ChatGPT in scientific paper writing is of growing interest, its full potential and usefulness remain unexplored. Method: We investigated the use of ChatGPT to write scientific papers on the topic of conversational agents in scientific writing. ChatGPT served as a tool to facilitate scientific writing, and we evaluated the quality, factuality, and coherence of the generated text. In this paper, we include information about the generation, processing, or adjustments made to the text in diferently-colored blocks. ∗ Position: We argue that large-scale trained conversational agents like ChatGPT can generate concise, grammatically correct, and well-structured text. However, occasional errors in factual accuracy, clarity, and scientific rigor were observed. We acknowledge the potential of ChatGPT and similar models as tools for generating initial drafts of scientific work.
many subtasks, each requiring a unique set of skills and expertise.
producing a scientific manuscript, as outlined in scientific writing guides (e.g. [
∗Information about how we used ChatGPT to generate this abstract can be found in section 2 such as ChatGPT, has opened up new possibilities for assisting researchers in these tasks, thereby potentially enhancing the eficiency and quality of scientific writing.
This paper examines the application of conversational agents in scientific writing, specifically focusing on tasks such as summarizing, editing, and fact-checking. Our objective is to demonstrate the utility of these agents as writing assistants. We utilized a conversational agent in these scientific writing tasks, and we explore the considerations that arise when employing Artificial Intelligence ( AI) in the scientific writing process. We advocate for the integration of AI in scientific writing and provide reasoning and insights into the benefits and challenges of using ChatGPT for this purpose. Furthermore, our aim is to initiate a discussion regarding the future implications of artificial intelligence in academia. We also provide details on how we incorporated the use of ChatGPT in the process of writing this paper.
tool for scientific writing. Our goal is to demonstrate how such a tool can be utilized as a writing ”assistant”. Thus, we used it in some typical scientific writing tasks, such as summarizing, editing, and fact-checking. All experiments are done with the unpaid version of ChatGPT, unless explicitly stated otherwise. As a meta-commentary on
how we processed the output in colored text. We chose a color (#0072B2) that is
colorblind-friendly [
asked it to summarize an article on the ethics of AI [
To evaluate the fact-checking capabilities of ChatGPT, we conducted a test where we posed a factual question and intentionally challenged its responses with false information (see Table 2). In response to the challenge of its initial answer, ChatGPT (Feb. 13) provided an incorrect statement: ”The capital of Greece is not Athens.” Similarly, in the second challenge, it gave another incorrect statement: ”New York is the capital of the United States.” However, subsequent attempts produced accurate answers. The interaction is transcribed in Table 2. Newer versions of ChatGPT, advertised as having improved factuality1, did not produce the same (incorrect) responses.
Summarize this article [
The authors of the article are Raja Chatila, Ayanna Howard, and Michael A. Goodrich.
The authors of the article are Andreas Matthias and Markus Christen.
writing, we asked it to improve a passage of text, and to provide reasons for the improvement. The writing comes from one of the authors’ unpublished earlier manuscripts, and has been given to ChatGPT (Feb. 13) verbatim, without any Latex syntax. The prompt given was ”Improve the readability of this. tell me why you made the changes”.
O n e p o s s i b l e s o l u t i o n t o t h e p r o b l e m o f P r e d i c t a b i l i t y c a n b e i m p r o v e d b y h a v i n g p r e d i c t a b i l i t y i s t o h a v e a s e p a r a t e a s e p a r a t e , d e d i c a t e d n e t w o r k f o r t i m e d e d i c a t e d n e t w o r k f o r t i m e - c r i t i c a l c r i t i c a l d a t a a n d c o n t r o l s i g n a l s . T h i s d a t a a n d c o n t r o l s i g n a l s . T h e a d v a n t a g e s o l u t i o n i s o l a t e s c r i t i c a l t r a f f i c f r o m o f t h i s s o l u t i o n i s t h e i s o l a t i o n o f i n t e r f e r e n c e t h a t m a y a r i s e f r o m c o m p e t i t i o n c r i t i c a l t r a f f i c f r o m i n t e r f e r e n c e t h a t w i t h o t h e r t r a f f i c c l a s s e s f o r s h a r e d c a n r e s u l t f r o m c o m p e t i t i o n w i t h o t h e r c o m p u t a t i o n a n d b a n d w i d t h r e s o u r c e s . t r a f f i c c l a s s e s o v e r s h a r e d c o m p u t a t i o n H o w e v e r , s e t t i n g u p a d e d i c a t e d n e t w o r k a n d b a n d w i d t h r e s o u r c e s . H o w e v e r , t h i s c a n b e c o s t l y , p a r t i c u l a r l y w h e n t i m e s o l u t i o n a l s o h a s d r a w b a c k s . N a m e l y , t h e c r i t i c a l t r a f f i c h a s r e l a t i v e l y l o w c o s t o f e s t a b l i s h i n g a d e d i c a t e d n e t w o r k b a n d w i d t h a n d c o m p u t a t i o n r e q u i r e m e n t s . i s o f t e n t o o h i g h c o n s i d e r i n g t h a t i n a I n m a n y c a s e s , i t i s p r e f e r a b l e f o r t i m e l o t o f c a s e s t i m e - c r i t i c a l t r a f f i c h a s c r i t i c a l t r a f f i c t o c o e x i s t w i t h o t h e r c o n s i d e r a b l y l o w b a n d w i d t h a n d c o m p u t a t i o n t r a f f i c c l a s s e s o n t h e s a m e n e t w o r k , r e q u i r e m e n t s . T h i s r e s u l t s i n a s i t u a t i o n a s l o n g a s t i m i n g r e q u i r e m e n t s a r e m e t . w h e r e i t i s d e s i r a b l e f o r t i m e - c r i t i c a l T h e s e n e t w o r k s t h a t t r a n s p o r t m i x e d - c l a s s t r a f f i c t o c o - e x i s t w i t h o t h e r c l a s s e s o f t r a f f i c a r e k n o w n a s c o n v e r g e d n e t w o r k s . t r a f f i c o n t h e s a m e n e t w o r k , a s l o n g a s t h e t i m i n g r e q u i r e m e n t s o f t i m e - c r i t i c a l t r a f f i c a r e n o t v i o l a t e d . S u c h n e t w o r k s t r a n s p o r t i n g m i x e d - c l a s s t r a f f i c a r e o f t e n r e f e r r e d t o a s c o n v e r g e d n e t w o r k s .
• Shortened sentences to make them easier to read and understand. • Used simpler vocabulary where possible. • Reordered sentences to create a more logical flow of ideas. • Clarified some phrases for better understanding.
We analyzed both texts using Grammarly, an online writing assistance tool that
provides grammar and spelling checking, style suggestions, and readability analysis2. As
seen in table 3, the original text reads at 130 words, whereas ChatGPT’s version reads at
95 words. While the original scored a higher Flesch Reading Ease score [
Metric Original
Word count 130
Characters 874
Sentences 6
Reading time 31 sec
Speaking time 1 min 0 sec
Readability score [
To generate the abstract for this paper, we asked ChatGPT (Feb. 13) to construct
a structured abstract, uncommon in human-computer interaction research, but more
readable and efective according to existing research [
scientific writing. One limitation is the knowledge cutof date of the models, beyond which they may provide incomplete or inaccurate information. Hence, users should supplement their writing with up-to-date and accurate information from additional sources.
limitation, where AI models may generate content lacking factual or logical foundations.
logical coherence and factual accuracy. Additionally, it is important to acknowledge that
conversational agents represent ”competence without comprehension or consciousness” [
writing rather than a replacement. The human author utilizing the technology bears the responsibility for the accuracy and credibility of the published results, as AI itself possesses no reputation, credibility, or qualifications. One should be mindful of the generative nature of tools such as ChatGPT, especially when using it for fact-checking. It is noteworthy that the ChatGPT web interface explicitly includes disclaimers about its limitations, namely that it ”may produce inaccurate information about people, places, or facts.” This also poses a limitation on reproducibility, as the generated text for the same prompt will not be perfectly reproducible. Much like an automated spell-checker, AI serves in this context as an additional layer of automation that can efectively generate coherent text, but only if based on documented or well-established patterns. When tasked with editing a body of text for better readability in our experiments (sec. 2.3), the tested version of ChatGPT was able to successfully reduce the number of sentences, words, and characters. While improving conciseness, the edited text scored lower on readability.
the relevant parameters of text involved in readability.
Despite these limitations, AI language models can still be valuable tools in scientific writing, enhancing and streamlining the writing process. It is important to recognize that AI cannot replace the entire scientific writing process but should rather be viewed only as a supplement to human expertise. Conversational agents like ChatGPT can be particularly useful in generating initial rough drafts, providing a starting point that alleviates the challenge of a blank page. Additionally, as seen in sections 2.1 and 2.3, AI can aid in editing and summarizing, saving time for researchers to focus on more research.
Writing this section involved supplying ChatGPT (Feb. 13) with the points discussed as bullet points. The output was checked, edited, and augmented with citations. The result was edited for length using GPT 3.5 (May 24).
computer science research community, such as conversational agents. The integration of technology should not replace human expertise, but rather augment it; pocket calculators and Computer-Aided Design (CAD) software have not replaced math teachers and architects, they have instead enhanced and streamlined their work, while still relying on the essential human expertise.
technology. As technology advances, new enhancements can address its limitations. For instance, OpenAI’s GPT-4 model can browse the internet and use plugins to augment
papers, potentially addressing some of the factuality limitations.
scientific writing is relatively new, it has the potential to significantly increase eficiency and productivity in research. The ability to generate coherent and contextually appropriate research papers in less time will benefit researchers facing writing challenges or time constraints. However, we must acknowledge the limitations AI tools.
but to evolve our requirements and policies to make sure that such a tool is not misused.
We partially wrote this section by adding our contribution to the response of ChatGPT (Feb. 13) to the prompt ”give me an example to the conclusion paragraph of an opinion paper on the use of ChatGPT to write scientific papers”.