Biden admin guidance lays out how AI in schools can be racist, anti-trans
The Department of Education’s Office for Civil Rights (OCR) recently released guidance on the discriminatory use of artificial intelligence (AI) in schools, following a mandate from President Biden. The guidance aims to address how AI systems can disproportionately impact minority and transgender students, potentially leading to federal investigations.
President Biden’s Executive Order 14110, signed last year, directed the Education Department to develop resources and policies to ensure responsible and non-discriminatory use of AI in schools. The guidance emphasizes the need to consider the impact of AI on vulnerable and underserved communities, highlighting the potential for AI systems to contribute to discrimination.
According to the Education Department’s guidance, the increasing use of AI in schools for instructional and safety purposes can create or perpetuate discrimination. The document provides information on federal civil rights laws within OCR’s jurisdiction and outlines scenarios that could prompt federal investigations into discriminatory AI practices.
One example cited in the guidance involves a plagiarism checker using generative AI that exhibits a high error rate when evaluating essays written by non-native English speakers. If a school continues to use this biased plagiarism checker despite complaints from students and parents, it could trigger a federal civil rights investigation.
Another scenario mentioned in the guidance pertains to AI systems determining disciplinary procedures for students, potentially leading to racial bias in school discipline practices. The guidance warns that schools relying on historic disciplinary data that reflects discriminatory practices could face scrutiny from civil rights officers.
Additionally, the guidance addresses the potential for AI to discriminate based on sex, with one example illustrating how facial recognition software could improperly flag students who do not conform to traditional gender norms. Students falsely flagged by such AI systems may experience embarrassment and missed class time, potentially resulting in Title IX violations.
The Biden administration has prioritized addressing discrimination within AI systems, with various federal agencies committing to crack down on discriminatory AI tools. President Biden’s executive order in October 2023 included initiatives to combat discrimination in AI, such as the recent guidance from the Department of Education.
As the government takes steps to address discrimination in AI, the nomination of Linda McMahon as the next leader of the Department of Education signals a commitment to promoting equity and fairness in education. The Department of Education’s guidance underscores the importance of ensuring that AI systems in schools do not perpetuate discrimination and harm vulnerable student populations.
Overall, the guidance from the Department of Education serves as a critical resource for schools to navigate the complex intersection of AI and civil rights, highlighting the need for responsible and equitable use of AI technologies in educational settings.
