Responsible AI: The Human Impact of Automated Decisions

Artificial intelligence (AI) is no longer a promise for the future: it is part of everyday decisions that directly influence access to social services, employment, credit, health, education, and security. In Latin America and the Caribbean, the public and private sectors are rapidly advancing digital transformation and AI agendas, often driven by the urgency to modernize and improve efficiency. However, this raises a key question: what human impact do these automated decisions have, and on which individuals and groups do their risks and consequences fall?

A study by the Inter-American Development Bank (IDB), the OECD, and UNESCO has documented that AI systems used to pre-screen job candidates can replicate biases against women when they are trained or designed without explicit equity criteria. In practice, these systems learn from historical decisions, such as who was hired most frequently, and tend to reproduce the dominant patterns of each sector. 

Thus, in masculinized occupations they can privilege men and exclude or relegate women. In turn, in feminized sectors, such as domestic or care work, they can reinforce the concentration of women, limiting opportunities and reproducing pre-existing labor segmentations, even when the qualifications are similar.

In Chile, for example, the MIRAI Project applies AI to the public health system to anticipate breast cancer risk, validating predictive models with data from tens of thousands of patients in the country's public health services, representing a pioneering use of AI to improve early detection and clinical follow-up in an essential social service. 

When AI is designed and deployed without explicit equity criteria, it can reproduce and even deepen pre-existing inequalities. A landmark study in the field of AI and bias shows, for example, that certain facial recognition systems have significantly higher error rates in darker-skinned women than in lighter-skinned men, due to unrepresentative training data. 

More than isolated "technical failures," these results reflect human decisions throughout the development process: what data is selected, who is considered an "average user," what variables are included or excluded, and who is involved in the design, validation, and deployment of the systems. 

In Latin America and the Caribbean, the incorporation of these perspectives into digital and AI agendas or strategies is still in the process of consolidation. An IDB study that analyzed 27 countries in the region showed that, although 59.2% of countries had a digital agenda: 

• 18.5% did not consider the dimensions differentiated by populations, nor the associated digital divide in their plans and/or policies
• 48.1% made a general mention without defining specific actions
• Only 33.3% included specific actions or programs with a differentiated approach

Likewise, a comparative analysis of six national AI strategies reveals that only four explicitly mentioned the differentiated perspective and none incorporated indicators or budgets to monitor progress. 

In practical terms, incorporating this perspective into AI implies considering or anticipating the effects of these technologies on different people and groups, considering variables such as sex, ethnicity, race, disability, and other relevant dimensions. It also means asking not only who benefits from these systems, but also how they participate and make decisions in their management, who could be excluded, made invisible or face greater risks, based on these dimensions. 

This approach translates into deliberate decisions throughout the entire AI lifecycle, not only to avoid negative effects on these populations, but also to actively contribute to reducing pre-existing inequalities. 

From a governance perspective, integrating differentiated approaches in AI is not limited to recognizing these gaps, but involves embedding them in the rules, processes, and institutional arrangements that guide how the use of these technologies is prioritized, designed, deployed, and monitored. 

This includes, for example, defining clear institutional responsibilities, establishing minimum standards for social impact assessment, ensuring coordination between digital agendas and equality and rights policies, and strengthening state capacities to monitor and correct unintended effects of AI systems.

Against this backdrop, moving towards responsible AI requires moving from ethical discourse to operational tools. A key approach is to incorporate accurate criteria throughout the entire lifecycle of AI projects, from design to post-deployment monitoring and evaluation.

Among the fundamental criteria are:

• Equipment and capabilities. It means, first of all, recognizing that decisions about AI are not made by algorithms, but by the people who design and train them. When teams developing or implementing AI systems are homogeneous, they are more likely to overlook differentiated impacts on women and other groups. Having diverse teams makes it possible to identify early risks, question implicit assumptions and prevent certain groups from being invisible in the design of systems.
• Quality data and ethical use. The reality is that artificial intelligence has the potential to make better decisions to the extent that it is fed with quality data. When data reflects the diversity of the population, including differences between women and men, as well as the reality of people with disabilities, indigenous peoples or Afro-descendants, AI systems can produce more accurate and equitable results. Therefore, having disaggregated and complete data is key. At the same time, using data responsibly implies recognizing and correcting possible structural biases present in administrative records or historical databases, as well as respecting basic principles of informed consent, privacy, and protection of personal data. Beyond regulatory compliance, this is also a matter of public trust: people must know that their data will be used in accordance with the purposes for which it was collected. 
• Transparent and fair processes. Transparency is key to detecting errors, correcting biases, and strengthening trust in the use of AI.  Decisions made with the support of AI should not be a "black box", neither for institutions nor for citizens. It is essential to document and explain the logic of the models in understandable terms, especially when they affect rights or access to services. In addition, defining in advance what is considered an acceptable outcome in terms of equity allows us to guide the design and use of these systems. This involves setting clear limits on how much difference is tolerable between the results received by different groups. For example, it can be defined that the approval rate of a benefit should not differ by more than 5% between groups (80% versus 75%); if that threshold is exceeded, a space opens to review and adjust the model. Likewise, if a facial recognition system achieves an accuracy of 95% in urban men, but only 70% in rural women, it can establish that the model should not be used if it does not reach a minimum acceptable level (For example, at least 85%) in the latter group and in other historically underrepresented groups.
• Accessible design.  An accessible design is one that works adequately for as many populations as possible, without excluding those who face physical, sensory, cognitive or technological barriers. This implies applying principles of accessibility and universal design from the beginning, incorporating reasonable adjustments when necessary, and considering diverse realities such as connectivity gaps, different levels of digital literacy, and differences between urban and rural contexts.
• Governance and accountability. It implies that there are clear rules about who is responsible for automated decisions and what happens when a system produces undesired or inequitable results. This includes enabling accessible channels of feedback and complaints so that people and communities can question decisions, request explanations or ask for corrections. It also involves establishing oversight mechanisms, such as independent audits to identify areas for improvement and potential biases. Having clear procedures in place to adjust or suspend the use of AI systems when negative impacts are detected helps to strengthen public trust and the protection of rights.
• Digital security. AI should contribute to safer digital environments if it is designed and used with a preventive and inclusive approach. This implies anticipating risks that may affect women and other populations in a differentiated way. It is important to prevent harmful uses such as deepfakes (digital content created with artificial intelligence that imitates a person in a very realistic way to make them say or do something that never happened), impersonation or algorithmic harassment, the latter understood as a form of digital violence in which algorithms, instead of protecting, end up facilitating or amplifying harassment or discrimination practices, either by design or as an unintended effect.

Incorporating these criteria is not a bureaucratic exercise or a simple "compliance checklist". It is a strategic decision. In contexts of high inequality, AI can be a powerful tool to expand rights and improve the efficiency of the State, if it is designed and deployed with a deep understanding of the social realities that women and other populations go through.

The question, then, is not whether the region should adopt AI, but how to do so in a way that strengthens equity, rights, and public trust. Committing to responsible and inclusive AI does not imply slowing down innovation: on the contrary, it is a condition to ensure its positive human impact, its social legitimacy and its sustainability over time.