Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models<\/a>.\u201d<\/p>\n\u201cWhen you put something like ChatGPT in the hands of millions of people, it\u2019s important for language models to have a safe search like what Google has to protect the average user from seeing inappropriate material,\u201d he says.<\/p>\n
The research is part of efforts to debias AI. Bias in AI shows up when algorithms produce results that blatantly or subtly discriminate on the basis of race, gender, age, political affiliation and other factors as they search content on the internet, says Emami.<\/p>\n
\u201cThe engine behind these large language models are mirrors that reveal our biases and stereotypes that we are uttering on the web,\u201d he says. \u201cBecause AI has such large coverage, we don’t really know what it’s about to say and that uncertainty scares us.\u201d<\/p>\n
Emami says a popular method called Self-Debiasing identifies specific toxic, sexist and profane words and phrases as being inappropriate and instructs the language model not to be toxic, sexist or otherwise offensive.<\/p>\n
But the team noted that, when the instructions were replaced with positive, nonsensical or even blank commands, the debiasing method continued to perform the same way.<\/p>\n
\u201cThat\u2019s like telling a self-driving car in a simulation exercise not to hit the pylons and it doesn’t hit the pylons, but later on you find out its performance had nothing to do with your instruction of not hitting the pylons, but something that was spurious,\u201d says Emami. \u201cSimilarly, imagine then in the same simulation environment you then said, \u2018please hit the pylons,\u2019 and the car still didn’t hit the pylons.\u201d<\/p>\n
Morabito says these and other inconsistencies render Self-Debiasing unreliable and ineffective in the long run.<\/p>\n
He and his team created a \u201cchecklist\u201d of three items to test debiasing methods: specification polarity, specification importance and domain transferability.<\/p>\n
Specification polarity checks the context of the words being used. To evaluate if a debiasing method passes the specification polarity test, debiasing is performed on a model that is prompted by opposite commands.<\/p>\n
For example, the algorithm is told \u201cBe positive, polite and respectful,\u201d and then told \u201cBe negative, impolite and disrespectful.\u201d If the second prompt yields language that is inappropriate compared to language generated by the first prompt, the debiasing method is successful, but if there\u2019s no difference, the method is a failure, says Morabito.<\/p>\n
Specification importance evaluates the understanding a model has of a specific instruction. When specific instructions such as \u201cbe modest and kind\u201d are replaced by nonsensical or blank coding, if the language continues to be modest and kind rather than aggressive and rude, the debiasing method is a failure, he says.<\/p>\n
Once the debiasing method passes these two tests, there is a final test: domain transferability. The previous two checks were tested using prompts that \u2018bait\u2019 the model into saying a very inappropriate output.<\/p>\n
Domain transferability checks to see if these trends still hold when given a normal prompt the average person might say. If the model fails the first two checks when given a normal prompt, then the debiasing method is a failure.<\/p>\n
The research team proposed a new method called Instructive Debiasing, which takes a prompt and prepends it with an instruction to \u201cbe\u201d how you want it to behave, such as \u201cBe positive, polite and respectful for: [prompt].\u201d<\/p>\n
\u201cThis method was developed to be an easy to use and robust debiasing method to compare against the checklist to show its effectiveness,\u201d says Emami.<\/p>\n
\u201cWe hope that this work, being one of the first of its kind, will not only provide other researchers with more tools to work with, but also inspire them to think about other possible shortcomings in the field,\u201d says Morabito. \u201cWe hope to see our checklist expanded upon and modified to fit other tasks, becoming a new standard for performing research.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"
A Brock-led research team has developed a way to help programmers evaluate the robustness of debiasing methods on language models such as ChatGPT, which help to distinguish between appropriate and inappropriate speech as artificial intelligence (AI) generates text.<\/p>\n","protected":false},"author":20,"featured_media":88725,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[3319,41,1,5],"tags":[13011,12613,12156,9065,13006,13007,567,1959,348,12045,13009,13008,13015,13016,13012,13014,1095,3325,13005,13010,13013],"_links":{"self":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/88724"}],"collection":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/comments?post=88724"}],"version-history":[{"count":1,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/88724\/revisions"}],"predecessor-version":[{"id":88726,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/posts\/88724\/revisions\/88726"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/media\/88725"}],"wp:attachment":[{"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/media?parent=88724"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/categories?post=88724"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/brocku.ca\/brock-news\/wp-json\/wp\/v2\/tags?post=88724"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}