Women, Work, and Algorithms: How AI Hiring Tools Are Failing Caribbean Women

In 2018, Amazon shut down an artificial intelligence recruitment tool it had spent four years building. The system had been trained on a decade's worth of successful hires, which were predominantly male, and it had learnt to penalise CVs containing the word "women" and to downgrade graduates of all-women's colleges. The story was reported and cited as a cautionary tale. Six years later, the lesson has not been learnt, and in the Caribbean the problem is arriving without even the benefit of the warning.

My hypothesis is straightforward: AI hiring tools currently deployed or being adopted by Caribbean businesses are encoding historical gender and racial biases into automated employment decisions, and no legal or regulatory accountability framework exists in the region to address this. The consequences fall disproportionately on women, particularly Black Caribbean women, who already face intersecting barriers of gender, race, class, and geography in reaching economic opportunity.

What follows sets out what we know about algorithmic hiring bias, why Caribbean labour markets are especially exposed, and the policy changes needed before this problem becomes entrenched.

The Algorithm Problem Is Not Hypothetical

The Amazon case is the most famous documented instance of AI hiring bias, but it is far from unique. Academic literature has accumulated a body of evidence that automated screening tools replicate and sometimes amplify patterns of historical discrimination. A 2019 study by Raghavan, Barocas, Kleinberg, and Levy, published in the Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, found that predictive hiring tools commonly use features that serve as proxies for protected characteristics, including postcode, educational institution, and name.

The mechanism is consistent: tools trained on historical hiring data will optimise for candidates who resemble previously successful employees. In organisations where women, and particularly women of colour, were historically underrepresented in senior and technical roles, the algorithm learns to deprioritise candidates who share their characteristics. The tool does not intend to discriminate. It simply repeats the past with greater speed and scale.

The WEF Global Gender Gap Report 2024 documents that women remain underrepresented across leadership, STEM, and technical occupations globally, including in the Caribbean. When those occupational structures become the training data for hiring AI, the algorithm absorbs the exclusion and treats it as the rule to follow.

Why the Caribbean Context Is Distinct

Caribbean labour markets have specific characteristics that intensify the risk. Women in the Caribbean outperform men at tertiary level by a wide margin: in Jamaica, women consistently account for approximately 60 per cent of university graduates. Yet this educational attainment does not translate linearly into labour market outcomes at senior levels. The gap between female educational achievement and female representation in leadership positions is well-documented in ILO reports on gender equality in Caribbean labour markets.

This mismatch matters for AI training data in a specific way. An AI system examining who holds senior roles in Caribbean organisations will find relatively few women, regardless of the talent pool available. It will then treat this distribution as normal and replicate it. The algorithm does not ask why women are underrepresented; it treats underrepresentation as the correct baseline.

The BPO sector illustrates this with particular clarity. Jamaica's business process outsourcing industry is a large employer, with women making up an estimated 55 to 60 per cent of the workforce. Many of these employers rely on automated screening tools supplied by their international clients, particularly large North American corporations that outsource call centre and back-office operations. The workers applying through these systems have no visibility into the criteria being applied. They receive no explanation when rejected. They have no regulator to approach with a complaint.

The intersecting challenges of gender, race, class, and geography in Caribbean digital access compound this. GSMA Connected Women Report 2024 data confirms that the gender mobile internet gap persists across developing regions, with women less likely to own smartphones and less likely to have reliable internet access. This affects not only whether women can access digital job applications but also how their digital footprints appear to algorithmic screening tools that scan professional networking profiles and online activity.

The Legislative Gap

Jamaica's Data Protection Act 2020, administered by the Office of the Information Commissioner, represents a genuine step forward for data rights in the Caribbean. The Act establishes principles of transparency, accountability, and data subject rights including the right to object to certain automated decisions. These provisions have potential relevance to algorithmic hiring.

But the Act was drafted in a different moment, before algorithmic decision-making in employment became the mainstream practice it is today. It does not require employers to conduct bias audits of AI systems used in recruitment. It does not mandate transparency notices specifying when an automated tool has influenced a hiring decision. It does not give candidates a right to an explanation of why an algorithm rejected their application. And its enforcement provisions have not been tested against algorithmic discrimination.

The CARICOM Regional Digital Economy Policy Framework provides a broader architecture for digital governance across member states, but it does not yet address AI hiring tools with the specificity required. The Framework articulates principles of inclusion and equity, but principles without enforcement mechanisms do not protect individual women from algorithmic rejection.

Compare this to the EU AI Act, adopted by the EU Council on 21 May 2024. Under Annex III, AI systems used in recruitment and employment are classified as high-risk. This classification triggers requirements for conformity assessments, human oversight, technical documentation, and transparency measures before a system can be deployed. Caribbean countries are importing tools that would be heavily regulated in their countries of origin, and deploying them in a regulatory vacuum.

Labour Law's Blind Spot

Employment discrimination law across CARICOM jurisdictions generally prohibits discrimination on grounds of sex in hiring and employment. Jamaica's Employment (Equal Opportunities) Act provides this protection in principle. The difficulty is that these laws were designed for human decision-makers, not algorithmic ones. A woman who is rejected by an AI screening tool before her CV reaches a human being has been discriminated against, potentially, at a stage that is legally invisible.

She cannot demonstrate that a hiring manager saw her application and chose a less qualified male candidate. The algorithm made the decision before any human reviewed her file. Under current law, she has no standing to demand the algorithmic criteria that produced her rejection, no right to a human review of that decision, and no clear path to a discrimination complaint.

The ILO's decent work agenda explicitly addresses gender equality in labour markets, and the ILO has published guidance on non-discriminatory recruitment practices. But ILO guidance does not have the force of domestic law, and Caribbean labour ministries have not yet adapted their enforcement frameworks to address algorithmic hiring.

What the Seoul Summit and UN Resolution Signal

The international policy environment is moving. The Seoul AI Safety Summit of 21 to 22 May 2024 produced commitments from participating governments to address AI harms across a range of domains, including workplace applications. The UN General Assembly's first AI resolution, adopted on 21 March 2024, calls on member states to govern AI in a manner consistent with human rights and fundamental freedoms. Both instruments signal that the era of voluntary, unaccountable AI deployment is ending, at least in the most advanced governance jurisdictions.

Caribbean states participated in these multilateral conversations but have not yet translated the commitments into domestic policy action. There is a real risk that the gap between international AI governance standards and Caribbean domestic regulation will widen precisely at the moment when AI hiring tools are becoming standard practice in the region's largest employment sectors.

ISO/IEC 42001, published in December 2023, provides an international standard for AI management systems, including provisions on risk assessment and responsible deployment. Caribbean organisations adopting AI hiring tools could apply this standard voluntarily, and Caribbean regulators could require conformity as a condition of use. Neither is currently happening at scale.

What Bias Looks Like in Practice

I want to be concrete about what algorithmic bias in hiring means for an individual woman in the Caribbean. Consider a woman in Kingston who applies for a supervisory role at a BPO operation. She holds a degree from the University of the West Indies and has seven years of relevant experience. The employer uses an applicant tracking system purchased from a US-based vendor. The system scores candidates using a model trained on the employer's previous hires, drawn from a period when the supervisory grades were predominantly male.

The system assigns her a lower score than a male candidate with equivalent qualifications. The score is not labelled as a gender score. It is labelled as a "fit score" or a "culture match" or a "potential indicator". No human examines why she scored lower. Her CV never reaches a recruiter. She receives an automated rejection. She has no way of knowing that an algorithm, not a person, made this decision, or that the decision reflected historical hiring patterns rather than her actual capabilities.

This is not speculation. It is the documented mechanism of every major case of AI hiring bias that researchers have been able to examine. And it is happening in Caribbean employment markets today, in the absence of any legal framework to prevent or remedy it.

Recommendations

1. Amend the Jamaica Data Protection Act 2020 to require mandatory bias disclosures for AI hiring tools. Any organisation using an automated or AI-assisted system in a hiring process should be required to disclose this to applicants, specify the criteria used, and provide a right to human review of automated rejections. The Office of the Information Commissioner should be given the authority and resources to investigate complaints of algorithmic discrimination in employment.
2. Introduce a sector-specific AI hiring standard for BPO operators. Given that the BPO sector employs a majority-female workforce and routinely uses AI screening tools supplied by foreign clients, the Jamaica Promotions Corporation and the Business Process Industry Association should jointly require that all AI recruitment tools undergo an independent bias audit before deployment, with results disclosed to the regulator.
3. Classify AI hiring tools as high-risk in any forthcoming CARICOM or national AI policy framework. The EU AI Act's classification of recruitment AI as high-risk provides a clear model. Caribbean governments should adopt equivalent classifications in their own policy frameworks, attaching audit, transparency, and human oversight obligations to this classification from the outset.
4. Require employers to report disaggregated hiring data by gender. An algorithmic bias audit is only meaningful if employers collect and report data on who applied, who was shortlisted, and who was hired, disaggregated by gender. This data requirement should be built into the Statistical Institute of Jamaica's labour market data collection and into the reporting obligations of companies above a defined size threshold.
5. Commission a Caribbean-specific study of AI hiring tool deployment and its gender impact. No rigorous regional study of AI tool adoption in Caribbean hiring exists. The University of the West Indies, in collaboration with UN Women Caribbean and the ILO Caribbean office, should conduct this research within 18 months to establish a factual baseline for policy intervention.
6. Establish a women's economic rights unit within the Office of the Information Commissioner. Algorithmic discrimination in hiring is fundamentally a data rights issue. The Office should have dedicated capacity to receive, investigate, and adjudicate complaints from women whose employment opportunities have been affected by automated systems, with the power to order audits and impose sanctions.

Conclusion

The Amazon case in 2018 was a warning. The EU AI Act in 2024 is a response to that warning, built on years of documented harm and sustained advocacy from researchers, civil society, and affected communities. The Caribbean is arriving at this conversation later, and that lateness carries a specific risk: the tools are already here, and the governance is not.

Caribbean women are not passive recipients of technology. They are workers, entrepreneurs, graduates, and professionals who deserve to compete for employment on the basis of their qualifications and capabilities, not on the basis of what an algorithm has learnt from a past in which they were excluded. The legal and regulatory frameworks that should protect them are incomplete. Completing them is an urgent task, not a future priority.

Every month that Caribbean governments delay action on algorithmic hiring bias is another month in which women's employment opportunities are shaped by tools that were not designed for fairness and are not held accountable for harm. The EU has adopted a framework and CARICOM can build one. Whether the political will exists to act before the harm becomes irreversible is the open question.

About the Author

Dr. Shirley Budall is a Caribbean expert in gender, inclusion, and AI governance with demonstrated experience in the ethical, legal, social and governance dimensions of artificial intelligence and digital technologies. She conducts legal and regulatory framework reviews and develops policy recommendations for legal reform in AI governance, data protection, human rights, and gender equality. Dr. Budall has knowledge of international and regional AI governance standards and has advised Caribbean government institutions and regional organisations on inclusive AI policy. She is a researcher and consultant working across the CARICOM region on digital economy governance, women's rights in the digital age, and equitable technology development. Contact: insights@starapple.ai