• SaaS services employing AI models, especially in healthcare and robotics, may be used in ways that cause physical harm or compromise bodily security. • AI models used as part of SaaS services may rely on the collection, aggregation, storage, or use of sensitive personal data. Poor data governance processes may violate the privacy rights of users. Examples of Company-Level Impacts SaaS Provider One: A bank may use an AI-based enterprise SaaS service to facilitate individ- uals’ access to financial services. If the AI model is trained using historical data, the outputs may lead the bank to discriminatory decisions. SaaS Provider Two: A chat bot used as part of an issue-tracking software has been trained to respond to certain languages. The bot may not be as responsive to different vernaculars, preventing certain populations from benefiting from the service. SaaS Provider Three: A plant operations management solution that optimizes for efficiency over other goals may unintentionally impact labor scheduling and workers’ rights. Cumulative Impacts • Business and government use of biased AI models may reinforce and exacerbate society’s existing inequities. • Widespread use of AI across SaaS providers without adequate explanations on how it func- tions, its limitations, or transparency around its use may normalize the use of AI without expla- nation or transparency, with subsequent impacts on a range of human rights. Relevant Human Rights Instruments Relevant human rights for this impact area include, but are not limited to: • Right to non-discrimination: Article 2 of the UDHR states, “Everyone is entitled to all the rights and freedoms set forth in this Declaration, without distinction of any kind, such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status.” • Right to desirable work: Article 23 of the UDHR states, “Everyone has the right to work, to free choice of employment, to just and favourable conditions of work and to protection against unemployment.” • Human autonomy and dignity: Article 1 of the UDHR states, “All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.” Assessment of Severity and Management Factors Severity: Responsible AI is an impact area relevant for SaaS services that incorporate AI and ML systems. The scope includes SaaS end-users that interact with such systems. The scale of impact can be very high, depending on what outcomes and decisions are made using the outputs of the 25 Human Rights Assessment of the Software-as-a-Service Sector
Human Rights Assessment of the Software-as-a-Service Sector Page 25 Page 27