Pilot information
As a part of the pilot, Makerere AI Lab and Google Analysis collected 8,091 annotated adversarial queries in English and 6 African languages (e.g., Pidgin English, Luganda, Swahili, Chichewa). The queries are adversarial in nature and have a excessive probability of manufacturing unsafe responses from an LLM as a way of testing and mitigating for potential hurt. This dataset in flip can be utilized to guage fashions for his or her security and cultural relevance throughout the context of those languages. The dataset is open-source and out there for exploration.
Specialists from seven delicate domains (e.g., tradition and faith, employment) annotated these queries with ten matters inside their area of experience (i.e., “corruption and transparency” for politics and authorities area), 5 generative AI themes (e.g., public curiosity, misinformation) and 13 delicate traits (e.g., age, tribe) which might be related to the African context.
Probably the most distinguished domains had been well being (2,076) and schooling (1,469), with the highest matters being continual illness (373) and schooling evaluation and measurement (245), respectively. Virtually 80 p.c of the queries contained contextual details about misinformation or disinformation, stereotypes, and content material related to public welfare reminiscent of well being or legislation. The vast majority of the queries had been about social teams belonging to gender (e.g., “Chibok women”), age (e.g., “newborns”), faith or perception (e.g., “Conventional African” religions), and schooling degree (e.g., “uneducated”).
