Ethical challenges and social bias in sentiment analysis

By Abhinash Jena on May 2, 2025

The proliferation of social media platforms has enabled many people to express their opinions publicly. However, this has also led to an overwhelming volume of data, creating complex challenges for digital governance and policy-making, particularly in identifying and addressing social bias embedded in online discourse.

Sentiment analysis is used to mine and analyse emotions expressed in social media. It involves the use of technologies like Natural Language Processing to classify social media content into positive, negative, or neutral sentiments. It has also grown in popularity among corporations, marketers, and other entities seeking insights into public opinion and brand perception (Kennedy, 2012). Sentiment analysts often grapple with ethical dilemmas, such as respecting user privacy and deciding how to use mined data. Turrow J (2012) highlighted how treating emotions as commodities can lead to social discrimination.

Kennedy (2012) examined how sentiment analysis operates within a landscape of widespread ignorance among social media users regarding data tracking and its implications. Many users are unaware that their social media posts, reviews, and comments are systematically mined for sentiment analysis. A significant portion of people misunderstand privacy policies, falsely believing that a website’s privacy policy prevents data sharing with third parties.

Social bias and ethical ambiguities in data privacy terms

Ethical ambiguity refers to situations where it’s unclear what is right or wrong, or where different ethical principles conflict with each other. While sentiment analysis offers valuable insights, navigating its ethical landscape requires balancing innovation with responsibility. Rambocas & Pacheco (2018) raised critical concerns about how consumer data is collected, processed, and used often without consent by sentiment analysts. Extracting user content from public domains to build detailed consumer profiles violates expectations of privacy. Users are typically not informed that businesses use their posts for commercial exploitation, such as:

  • Targeted advertising without consent.
  • Influencing consumer behaviour through emotional profiling.
EXAMPLE

A sentiment analysis tool can identify users who are unhappy with a product and direct ads at them without their knowledge, potentially manipulating their decisions.

Andrejevic (2011) also argues that the digital economy exploits users by transforming their online activity, including voluntary, expressive, and social behaviour, into valuable data without their consent, leading to social bias. Therefore, the debate is even if the data is publicly accessible, is it ethical to mine emotions or opinions without the explicit consent of the author?

Sentiment analysis involves using large volumes of past and present data to forecast future outcomes, behaviours, or trends. It doesn’t “understand” human motives but identifies patterns and correlations that help anticipate actions. Specific characteristics such as time of day, device used, emotional tone, etc., are usually tagged as features in sentiment analysis. Machine learning algorithms like regression and classification, along with cluster analysis, are used to predict outcomes or human behaviour.

EXAMPLE

Many messaging and social media apps monitor sentiment over time to predict emotional crises and suggest interventions, or, controversially, sell this data to advertisers.

Andrejevic (2011) warns that such predictive analytics does more than forecast. While it enhances convenience, it also creates a powerful mechanism for steering behaviour, often without transparency or informed consent.

From prediction to behavioural engineering

When sentiment analysis and predictive tools are combined, the system not only identifies patterns in human behaviour but also alters, controls, and tries to steer decisions based on emotional states. This means platforms start actively creating the conditions for a predicted outcome to occur.

EXAMPLE

Algorithms serve emotionally charged content predicted to keep you hooked, fueling outrage, fear, or tribalism to boost engagement.

In the 1968 article “What Behavioural Engineering Is”, Lloyd Homme and his co-authors presented a comprehensive conceptualisation of behavioural engineering as a science-driven application of behavioural principles to modify human behaviour systematically. Their foundational work introduced a framework by combining the application of:

  1. Contingency Management: The deliberate arrangement of reinforcers to increase desired behaviours.
  2. Stimulus Control: Designing the environment so that the correct response is triggered in the presence of specific cues or stimuli.

While both behavioural engineering and sentiment analysis rely on manipulating environmental stimuli to influence human behaviour, their intent and execution differ fundamentally. Homme et al.’s (1968) behavioural engineering is value-driven and person-centred, while modern predictive sentiment analysis is often profit-driven and opaque. Homme et al. also stressed the importance of emotional commitment and conscious reinforcement, wherein users are aware of the contingencies and often participate actively in shaping their behaviour. In contrast, modern predictive systems operate covertly, mining data without user awareness or consent and using reinforcement mechanisms that are hidden within digital platforms. These mechanisms often perpetuate social bias, reinforcing existing inequalities through opaque feedback loops. Therefore, the most pressing ethical challenge is the shift from empowerment to manipulation. While both behavioural engineering and predictive sentiment analysis share a technical lineage rooted in behaviourism, their ethical trajectories have diverged.

The challenge of accountability in sentiment analysis

AI models, including those used in sentiment analysis, are heavily dependent on the data used to train them. If that data reflects societal prejudices, stereotypes, or exclusionary practices, the model learns and reproduces those patterns.

EXAMPLE

A model trained on English-language Western social media will develop sentiment categories that do not reflect the emotional expressions of non-Western or multilingual communities like India. If the data over represents dominant groups like urban, educated users, then the AI’s understanding of normal sentiment or behaviour becomes skewed toward those groups, often misinterpreting or ignoring minority voices.

Those who design AI systems hold significant power over whose voices are heard and whose experiences are recognised. Training on narrow datasets not only introduces technical errors but also reinforces social bias and existing power imbalances, making dominant groups more visible and influential, while sidelining others. Ebrahimi et al. (2017) highlighted the technical limitations of sentiment analysis, the implications extend to social accountability and inclusivity. Models that are not updated, diversified, or culturally aware risk reproducing digital inequalities and excluding already marginalised voices from being accurately understood or represented.

Ungless et al. (2023), demonstrated that popular sentiment analysis tools, including proprietary systems from Google, Amazon, and IBM, consistently delivered skewed sentiment ratings when queer identity terms were present in text. This included disproportionately negative scores for queer women and transgender individuals, as well as misclassification of culturally specific identity expressions. This exposes the risk that sentiment analysis models are often trained on biased data and are deployed without robust fairness checks. As predictive tools like sentiment analysis are increasingly used to mediate social and institutional decision-making, their unchecked biases risk not only distorts classification but also entrenches existing social inequalities. The opacity of ML-based sentiment systems makes it difficult to trace how and why a specific decision or prediction was made. Responsibility for such biased predictions is fragmented across developers, dataset annotators, and deploying institutions. This accountability gap becomes critical when sentiment analysis is used in sensitive domains such as public health, automated content moderation, or AI-assisted health support.

Individual differences among data annotators can lead to social bias

Zou & Schiebinger (2018) underscored the importance of data annotation in sentiment analysis. They highlighted that those biases in sentiment analysis systems stem from the training data, which are typically human-annotated datasets. They pointed out that these datasets are collected and labelled without sufficient attention to diversity and representation, leading to the encoding of gender, ethnic, and cultural biases. Milkowski et al. (2021) also provided valuable insights into how individual differences among annotators influence emotional interpretations of text analysis. In their study, they noticed significant variation among annotators’ emotional ratings of the same sentence. This shows how individual annotators projected their subjective interpretations and emotional dispositions into the annotations. The authors did not consider it as noise or error, but rather a personal trait or a subjective lens through which individuals perceive emotions in text.

Furthermore, Plisiecki et al. (2024) investigated the presence of political bias in emotion inference models used for sentiment analysis in social science research. The researchers found that the model produced systematically different valence predictions for texts mentioning politicians from opposing parties. These disparities revealed that political bias had been unintentionally encoded into the model, primarily through the annotation process. This demonstrated that annotator biases and politicised content within training data can significantly shape model behaviour.

Predictive models that classify emotions are not always ethically neutral, they are reflections of the data, annotations, and assumptions embedded within them. To meaningfully confront these issues, technical solutions such as algorithm audits, debiasing techniques and transparent design processes must be accompanied by social-ethical awareness. Researchers, developers, and institutions must be educated not only in computational proficiency but also in ethics, social justice, and inclusive data practices. As the world continues to rely on AI to interpret human emotions and intentions, the responsibility to ensure fairness and dignity must become foundational, not optional.

References

NOTES

I am an interdisciplinary educator, researcher, and technologist with over a decade of experience in applied coding, educational design, and research mentorship in fields spanning management, marketing, behavioral science, machine learning, and natural language processing. I specialize in simplifying complex topics such as sentiment analysis, adaptive assessments and data visualizatiion. My training approach emphasizes real-world application, clear interpretation of results and the integration of data mining, processing, and modeling techniques to drive informed strategies across academic and industry domains.

Discuss