{"id":83289,"date":"2023-01-31T14:56:21","date_gmt":"2023-01-31T19:56:21","guid":{"rendered":"https:\/\/brocku.ca\/brock-news\/?p=83289"},"modified":"2024-09-01T09:54:35","modified_gmt":"2024-09-01T13:54:35","slug":"opinion-blayne-haggart-discusses-concerns-about-new-chatgpt-chatbot","status":"publish","type":"post","link":"https:\/\/brocku.ca\/brock-news\/2023\/01\/opinion-blayne-haggart-discusses-concerns-about-new-chatgpt-chatbot\/","title":{"rendered":"OPINION: Blayne Haggart discusses concerns about new ChatGPT chatbot"},"content":{"rendered":"

This article written by Blayne Haggart, Associate Professor of Political Science at Brock University, was originally published in <\/em>The Conversation<\/a>.<\/em><\/p>\n

Of all the reactions elicited by ChatGPT, the chatbot from the American for-profit company OpenAI that produces grammatically correct responses to natural-language queries, few have matched those of educators and academics.<\/p>\n

Academic publishers have moved\u00a0to ban ChatGPT from being listed as a co-author and issue strict guidelines outlining the conditions under which it may be used. Leading universities and schools around the world, from France\u2019s renowned\u00a0Sciences Po\u00a0to\u00a0many Australian universities, have banned its use.<\/p>\n

These bans are not merely the actions of academics who are worried they won\u2019t be able to catch cheaters. This is not just about catching students who copied a source without attribution. Rather, the severity of these actions reflects a question, one that is not getting enough attention in the endless coverage of OpenAI\u2019s ChatGPT chatbot: Why should we trust anything that it outputs?<\/p>\n

This is a vitally important question, as ChatGPT and programs like it can easily be used, with or without acknowledgement, in the information sources that comprise the foundation of our society, especially academia and the news media.<\/p>\n

Based on my work on the\u00a0political<\/u>\u00a0economy<\/u>\u00a0of\u00a0knowledge governance<\/u>, academic bans on ChatGPT\u2019s use are a proportionate reaction to the threat ChatGPT poses to our entire information ecosystem. Journalists and academics should be wary of using ChatGPT.<\/p>\n

Based on its output, ChatGPT might seem like just another information source or tool. However, in reality, ChatGPT \u2014 or, rather the means by which ChatGPT produces its output \u2014 is\u00a0a dagger aimed directly at their very credibility as authoritative sources of knowledge<\/u>. It should not be taken lightly.<\/p>\n

Trust and information<\/h2>\n

Think about why we see some information sources or types of knowledge as more trusted than others. Since\u00a0the European Enlightenment<\/u>, we\u2019ve tended to equate scientific knowledge with knowledge in general.<\/p>\n

Science is more than laboratory research: it\u2019s a way of thinking that prioritizes empirically based evidence and the pursuit of transparent methods regarding evidence collection and evaluation. And it tends to be the gold standard by which all knowledge is judged.<\/p>\n

For example, journalists have credibility because they investigate information, cite sources and provide evidence. Even though sometimes the reporting may contain errors or omissions, that doesn\u2019t change the profession\u2019s authority.<\/p>\n

The same goes for opinion editorial writers, especially academics and other experts because they \u2014 we \u2014 draw our authority from our status as experts in a subject. Expertise involves a command of the sources that are recognized as comprising legitimate knowledge in our fields.<\/p>\n

Most op-eds aren\u2019t citation-heavy, but responsible academics will be able to point you to the\u00a0thinkers\u00a0and\u00a0the work\u00a0they\u2019re\u00a0drawing\u00a0on. And those sources themselves are built on verifiable sources that a reader should be able to verify for themselves.<\/p>\n

Truth and outputs<\/h2>\n

Because human writers and ChatGPT seem to be producing the same output \u2014 sentences and paragraphs \u2014 it\u2019s understandable that some people may mistakenly confer this scientifically sourced authority onto ChatGPT\u2019s output.<\/p>\n

That both ChatGPT and reporters produce sentences is where the similarity ends. What\u2019s most important \u2014 the source of authority \u2014 is not\u00a0what<\/em>\u00a0they produce, but\u00a0how<\/em>\u00a0they produce it.<\/p>\n

ChatGPT doesn\u2019t produce sentences in the same way a reporter does. ChatGPT, and other machine-learning, large language models, may seem sophisticated, but they\u2019re basically just complex autocomplete machines. Only instead of suggesting the next word in an email, they produce the most statistically likely words in much longer packages.<\/p>\n

These programs repackage others\u2019 work as if it were something new. It does not \u201cunderstand\u201d what it produces.<\/p>\n

The justification for these outputs can never be truth. Its truth is the truth of the correlation, that the word \u201csentence\u201d should always complete the phrase \u201cWe finish each other\u2019s \u2026\u201d because it is the most common occurrence, not because it is expressing anything that has been observed.<\/p>\n

Because ChatGPT\u2019s truth is only a statistical truth, output produced by this program cannot ever be trusted in the same way that we can trust a reporter or an academic\u2019s output. It cannot be verified because it has been constructed to create output in a different way than what we usually think of as being \u201cscientific.\u201d<\/p>\n

You can\u2019t check ChatGPT\u2019s sources because the source is the statistical fact that most of the time, a set of words tend to follow each other.<\/p>\n

No matter how coherent ChatGPT\u2019s output may seem, simply publishing what it produces is still the equivalent of letting autocomplete run wild. It\u2019s an irresponsible practice because it pretends that these statistical tricks are equivalent to well-sourced and verified knowledge.<\/p>\n

Similarly, academics and others who incorporate ChatGPT into their workflow run the existential risk of kicking the entire edifice of scientific knowledge out from underneath themselves.<\/p>\n

Because ChatGPT\u2019s output is correlation-based, how does the writer know that it is accurate? Did they verify it against actual sources, or does the output simply conform to their personal prejudices? And if they\u2019re experts in their field, why are they using ChatGPT in the first place?<\/p>\n

Knowledge production and verification<\/h2>\n

The point is that ChatGPT\u2019s processes give us no way to verify its truthfulness. In contrast, that reporters and academics have a scientific, evidence-based method of producing knowledge serves to validate their work, even if the results might go against our preconceived notions.<\/p>\n

The problem is especially acute for academics, given our central role in creating knowledge. Relying on ChatGPT to write even part of a column means they\u2019re no longer relying on the scientific authority embedded in verified sources.<\/p>\n

Instead, by resorting to statistically generated text, they are effectively making an argument from authority. Such actions also mislead the reader, because the reader can\u2019t distinguish between text by an author and an AI.<\/p>\n

ChatGPT may produce seemingly legible knowledge, as if by magic. But we would be well advised not to mistake its output for actual, scientific knowledge. One should never confuse coherence with understanding.<\/p>\n

ChatGPT promises easy access to new and existing knowledge, but it is a poisoned chalice. Readers, academics and reporters beware.\u00a0<\/em><\/p>\n

\"The<\/p>\n","protected":false},"excerpt":{"rendered":"

Blayne Haggart, Associate Professor of Political Science at Brock University, recently published a piece in The Conversation about the ChatGPT chatbot.<\/p>\n","protected":false},"author":1,"featured_media":83291,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[7,6,1,38],"tags":[3445,12384,42,5512],"_links":{"self":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/83289"}],"collection":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/comments?post=83289"}],"version-history":[{"count":7,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/83289\/revisions"}],"predecessor-version":[{"id":83294,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/83289\/revisions\/83294"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/media\/83291"}],"wp:attachment":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/media?parent=83289"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/categories?post=83289"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/tags?post=83289"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}