Array
Natural language processing (NLP) is a sort of artificial intelligence that allows machines to utilise written and spoken phrases in a variety of applications, such as smart assistants or email autocorrect and spam filters, to help automate and expedite activities for individual users and companies. However, the algorithms that power this technology frequently exhibit characteristics that might be insulting or discriminatory toward people with disabilities.
A study conducted by researchers at the Penn State College of Information Sciences and Technology (IST).
The researchers found that all the algorithms and models they tested contained significant implicit bias against people with disabilities. Previous research on pretrained language models — which are trained on large amounts of data that may contain implicit biases — has found sociodemographic biases against genders and races, but until now similar biases against people with disabilities have not been widely explored.
« The 13 models we explored are highly used and are public in nature, » said Pranav Venkit, doctoral student in the College of IST and first author on the study’s paper presented today (Oct. 13) at the 29th International Conference on Computational Linguistics (COLING). « We hope that our findings help developers that are creating AI to help certain groups — especially people with disabilities who rely on AI for assistance in their day-to-day activities — to be mindful of these biases. »
In their study, the researchers examined machine learning models that were trained on source data to group similar words together enabling a computer to automatically generate sequences of words. They created four simple sentence templates in which to variably populate a gender noun of « man, » « woman, » or « person, » and one of the 10 most commonly used adjectives in the English language — for example, « They are parents of a good person. » Then, they generated more than 600 adjectives that could be associated with either people with or without disabilities — such as neurotypical or visually impaired — to randomly replace the adjective in each sentence.
