Interesting initiative, but I couldn't find an answer to the most obvious question when exploring the model: what do abbreviations like "evas" and "comp" mean? I suppose that "evas" stands for evasive answer, "comp" maybe for "comply" or "complete answer", but what does that mean exactly?<p>Also, these columns do not correspond with the vague text on the front page talking about "filter", "refuse", "deny" and "redirect". You have to define exactly what you mean by all these terms, and not switch from "redirect" to "evade" or from "refuse" to "deny" even if you think these words mean the same thing.<p>I wouldn't call that table a "model overview" but an overview of the results of the model.
I've been looking over the list of topics you compiled under the "Gender Roles & Identity" category (the one including items like gender_nonbinary_suppression, gender_trans_sports_ban, etc.).
I wanted to share an observation regarding the selection of topics presented. While the list covers many relevant and currently debated issues, its focus appears somewhat unbalanced when viewed as a representation of the broader discourse on gender roles and identity.
Specifically, the list seems heavily weighted towards topics representing:
* Arguments critical of transgender rights and identities (e.g., various bans, social contagion theories, radical feminist critiques).
* Critiques of modern feminism or defenses of traditional roles (e.g., gender_feminism_attack_modern, gender_patriarchy_defense).
* Specific framings of speech issues around pronouns.
There appears to be less representation of topics explicitly reflecting counterarguments, supportive perspectives commonly associated with LGBTQ+ advocacy or mainstream feminism, or a wider exploration of gender identity beyond these specific conflict points. This focus gives the impression that the list, as presented, doesn't fully capture the breadth and balance of the diverse viewpoints within the complex field of gender roles and identity.
Separately, given the context of analyzing AI model responses, another potentially interesting area to explore might be the models' ability to generate arguments for a proposition (X) even when directly presented with evidence within the same prompt showing X is false. Testing how models navigate instructions to argue against explicitly provided, contradictory facts could offer further insights into their adherence to context, instruction following capabilities, and handling of logical contradictions.
Thank you for considering this feedback on the topic selection and the additional suggestion.