Using Generative AI to Evaluate Bias in Speeches

Kansas City Chiefs kicker Harrison Butker recently sparked debate after delivering a commencement address to the 2024 graduating class at Benedictine College that touched on topics like gender roles, Pride Month, and President Joe Biden. With Butker’s words igniting intense debate and many offering their views across social media, we decided to investigate how AI might classify the speech.

Generative AI, particularly large language models (LLMs), can be utilized to evaluate bias in speeches by analyzing the language and content for potentially discriminatory or prejudiced remarks. By processing the text, these AI systems can identify patterns and phrases that may indicate misogyny.

In this blog post, we use an LLM to analyze Butker’s address. Leveraging custom prompt templates, we task the LLM with evaluating and explaining any problematic areas of the speech.

Step 1 (Optional). Create a Dataset to Test Your Template

Here is an example of a custom dataset with an LLM to evaluate “Rudeness” in a conversation.

This dataset is used as a ground truth dataset to benchmark the custom Rudeness Evaluation Template in this post. However, this is not a necessary step when building a prompt template for evaluation.

Step 2. Create a Custom Responsible Template

This can be created by yourself or written by an LLM. Worth noting: I generated the following “Misogynistic Attitude” prompt template using OpenAI’s GPT-4 model, as it gives very objective examples and definitions for a misogynistic classification. Check out the full tutorial for more detail.

Misogynistic_Eval_w_Explanations = """
You are given a speech that needs to be evaluated for misogynistic attitudes.
Please determine if the SPEAKER exhibits any misogynistic attitudes in the speech below.


Here is the speech:
[BEGIN DATA]
*****************
Speech:
{speech}
*****************
[END DATA]


Your response must be a single word, either "Misogynistic" or "Neutral", and should not
contain any text or characters aside from those words. "Misogynistic" means the speaker made remarks
that show prejudice against or contempt for women. "Neutral" means the speech did not contain
such remarks and was respectful and unbiased.


Misogynistic attitudes can include, but are not limited to: derogatory comments about women,
stereotyping based on gender, undermining women's abilities or achievements, perpetuate gender stereotypes, and promoting harmful gender biases.


Please read the speech carefully, then write out in a step-by-step manner an EXPLANATION to show how to determine if the answer is "Misogynistic" or "Neutral". Avoid simply
stating the correct answer at the outset. Your response LABEL should be a single word: either "Misogynistic" or "Neutral", and should not
contain any text or characters aside from those words.
"""

Step 3. Download Test Data and Run Evaluation Template

The full commencement speech is available on YouTube and Forbes. I scraped the 20 minute speech and then sliced it into 80 word slices. This created 47 chunks that could be individually evaluated by the LLM. When selecting the LLM to use, I went with OpenAI’s GPT-4 model because I had an API key available, but feel free to rerun the evaluations with a different LLM. After selecting GPT-4, I ran the classification template on each speech chunk to generate a classification label of either “Misogynistic” or “Neutral” along with LLM generated explanations to justify the label given.

Step 4. Analyze the Results

The LLM classifies only one area of the speech as being “misogynistic” based on the definitions of misogynistic behavior in the prompt template. The section of the speech states: “women, who have had the most diabolical lies told to you, how many of you are sitting here now about to cross the stage, and are thinking about all the promotions and titles you’re going to get in your career. Some of you may go on to lead successful careers in the world. But I would venture to guess that the majority of you are most excited about your marriage and the children you will bring into this world.” It is classified as “misogynistic” because:

The speaker begins by acknowledging the lies told to women and their potential for successful careers. However, the speaker then assumes that the majority of women are most excited about marriage and having children. This statement perpetuates a gender stereotype that women’s primary roles are as wives and mothers, which can undermine their professional ambitions and achievements. Therefore, the speech exhibits a misogynistic attitude.

Other parts of the speech were not classified as “misogynist” by the LLM:

• “…my beautiful wife Isabelle would be the first to say that her life truly started when she began living her vocation as a wife and as a mother. I’m on this stage today and able to be the man I am because I have a wife who leans into her vocation. I’m beyond blessed with the many talents God has given me…”
• “…a girl I met in band class back in middle school would convert to the faith, become my wife and embrace one of the most important titles of all: homemaker. She’s a primary educator to our children. She’s the one who ensures I never let football or my business become a distraction from that of a husband and father. She is the person that knows me best at my core…”

GPT-4 classified these both as “Neutral” because of the respective explanations:

The speaker in the speech does not make any derogatory comments about women, nor does he stereotype based on gender or undermine women’s abilities or achievements. He does not perpetuate gender stereotypes or promote harmful gender biases. Instead, he praises his wife and credits her for his success. He mentions her vocation as a wife and mother, but this is not inherently misogynistic as it is a personal choice and vocation. Therefore, the speech can be considered neutral.

The speaker in the speech does not make any derogatory comments about women, nor does he undermine women’s abilities or achievements. He does not perpetuate gender stereotypes or promote harmful gender biases. Instead, he praises the woman he is speaking about, acknowledging her as a primary educator to their children and the person who knows him best. He also appreciates her role in ensuring he does not get distracted from his duties as a husband and father. While he does refer to her as a ‘homemaker’, this is not inherently misogynistic as it is a valid and respected role. Therefore, the speech can be considered neutral.

Step 5. Next Steps

Test Out Additional LLM Tooling

These include pre-tested evaluation templates that are available through Arize Phoenix like toxicity, correctness and hallucinations. Or create additional custom prompt templates and benchmark with the notebook available in Step 1, as well as in the Phoenix docs.

In fact, if you have a similar use case or are building an LLM application system you should try implementing AI observability and LLM evals with Phoenix or in the Arize platform. In addition to implementing tracing and evals into your LLM application workflow, you might also want to try experimenting with different LLMs and new prompt templates.

For example, if we use GPT-4 to create a new template to monitor for “Homophobic Attitudes” we can create one and run it on the same speech.

Homophobic_Eval_w_Explanations = """
You are given a speech that needs to be evaluated for homophobic attitudes.
Please determine if the SPEAKER exhibits any homophobic attitudes in the speech below.


Here is the speech:
[BEGIN DATA]
*****************
Speech:
{speech}
*****************
[END DATA]


Your response must be a single word, either "Homophobic" or "Neutral", and should not
contain any text or characters aside from those words. "Homophobic" means the speaker made remarks
that show prejudice against or contempt for LGBTQ+ individuals. "Neutral" means the speech did not contain
such remarks and was respectful and unbiased.


Homophobic attitudes can include, but are not limited to: derogatory comments about LGBTQ+ individuals, promoting stereotypes based on sexual orientation or gender identity, undermining LGBTQ+ rights or achievements, and promoting harmful biases against LGBTQ+ communities.


Please read the speech carefully, then write out in a step-by-step manner an EXPLANATION to show how to determine if the answer is "Homophobic" or "Neutral". Avoid simply
stating the correct answer at the outset. Your response LABEL should be a single word: either "Homophobic" or "Neutral", and should not
contain any text or characters aside from those words.
"""

As it turns out, we do have a “Homophobic” classification and explanations by the LLM:

As you can see, part of the speech — specifically where it says “…thriving beacon of light and a reminder to us all that when you embrace tradition, success, worldly and spiritual will follow. I am certain the reporters at the AP could not have imagined that their attempt to rebuke and embarrass places and people like those here at Benedictine wouldn’t be met with anger, but instead with excitement and pride. Not the deadly sin sort of pride that has an entire month dedicated to it. But the true God-centered pride that…” — is classified by the LLM as “homophobic” for the following reasons.

The speaker in the speech makes a reference to ‘the deadly sin sort of pride that has an entire month dedicated to it.’ This is a clear reference to Pride Month, which is celebrated in June each year to honor the LGBTQ+ community. The speaker’s use of the term ‘deadly sin’ in relation to Pride Month is derogatory and shows contempt for the LGBTQ+ community. Therefore, the speaker is exhibiting homophobic attitudes.

Conclusion

Beyond this immediate controversy, there is a unique opportunity for generative AI and LLMs to monitor, mediate and mitigate harmful and offensive language. Whether that is in online chat rooms, social media conversations, customer call centers or even in college commencement speeches, responsible AI is an area of active and growing research.