Inference & Profiling · Restriction or exception
Fully automated high-risk decisions affecting individual rights need human supervision
The Customer may not use the services to make automated decisions that have a material detrimental impact on individual rights without human supervision in high-risk domains. The policy gives employment, healthcare, finance, legal, housing, insurance and social welfare as examples of such domains. The restriction is tied to three conditions together: the decision is automated, it has a material detrimental impact on individual rights, and it is made without human supervision in a high-risk domain.
Supporting quotations
to make automated decisions that have a material detrimental impact on individual rights without human supervision in high-risk domains, such as in employment, healthcare, finance, legal, housing, insurance, or social welfare;
Qualifications & exceptions
- Applies to automated decisions without human supervision, not to every use in these fields.
- Domains are introduced by "such as", so the list is not shown as complete.
Not stated here
- The policy does not say what level of human supervision is enough.
Inference & Profiling · Restriction or exception
AI outputs may be inaccurate and consequential uses need risk evaluation and human oversight
The policy says the AI Model Services use artificial intelligence and machine learning models that generate outputs, which may be inaccurate, and that outputs should be evaluated for accuracy and appropriateness. If the Customer uses the AI Model Services to make consequential decisions, it must evaluate the potential risks of the use case and implement appropriate human oversight, testing and other use-case-specific safeguards to mitigate them.
Supporting quotations
AI Model Services use artificial intelligence and machine learning models that generate outputs, which may be inaccurate. Outputs should be evaluated for accuracy and appropriateness.
If you use the AI Model Services to make consequential decisions, you must evaluate the potential risks of your use case and implement appropriate human oversight, testing, and other use case-specific safeguards to mitigate such risks.
Qualifications & exceptions
- The safeguard duty applies when the AI Model Services are used for consequential decisions.
Not stated here
- The policy does not define consequential decisions or say what oversight or testing is appropriate.
Also annotated in the same passage (2)