CrystalFeel - Multidimensional Emotion Intensity Analysis from Natural Language



Welcome to the CrystalFeel demo site.

While humans experience and express feelings and emotions every day, the degree or intensity of our emotions varies from one experience to another. For example, feeling annoyed by a noisy neighbour and feeling livid about a terrorist attack are very different experiences of anger; missing a bus and losing a loved one are very different experiences of sadness.

To date, a vast majority of language analytic services and theoretical models on emotion understanding are limited in that they predominantly consider a discrete, categorical sense of sentiments (i.e., positive, negative, neutral) and emotions (i.e., happy vs. not happy, sad vs. not sad, angry vs. not angry). Although such discrete emotion and sentiment classifiers can also report real-valued scores, these scores indicate the confidence or probability of the classification but not the intensity of the emotional experience. Scoring emotion intensity along a continuous scale is a relatively less explored feature.

CrystalFeel is a collection of machine learning-based emotion analysis algorithms for analyzing the emotional-level content from natural language. CrystalFeel produces multiple psychologically meaningful analytic outputs based on a multi-theoretic conceptual ground in emotion type, emotion dimension, and emotion intensity. It is developed by researchers studying affective and social intelligence from A*STAR's Institute of High Performance Computing.

This site shows an interactive demo, explains how the results can be interpreted, introduces the science behind the development and describes its applications to date.

Given a text, CrysalFeel analyses the emotional information by running five independently trained algorithms simultaneously and reports five dimensions of analysis results: fear intensity, anger intensity, joy intensity, sadness intensity, and valence intensity.

Based on the intensity scores, CrystalFeel generates two classification outputs: emotion and sentiment.


Quantitative Outputs – Emotion Intensity Scores

The following table describes the output codebook for the five emotion intensity scores.


Output Dimension
Conceptual Meaning
Value Range
FearFear refers to an unpleasant emotion arising from a perceived threat, danger, pain, or harm. Fear often leads to a confrontation with or escape from the threat, and sometimes freezing or paralysis in extreme events.

In CrystalFeel, the conceptual meaning of fear is consistent with Plutchik’s psychoevolutional theory of emotions (Plutchik, 1980; Plutchik, 1991, pp. 72-78). Fear may range from low-intensity worry to high-intensity terror.

Fear is perhaps the most frequently discussed emotion type in the literature, plausibly due to its functions from the psychoevolutional perspective. Fear’s synonyms may include apprehension, anxiety, worry, scared, dread, horror, and terror (Ortony et al., 1998).		tFear is a continuous variable ranging from 0 to 1, where 0 indicates that this text does not express the fear emotion at all, and 1 indicates that this text expresses an extremely high intensity of the fear emotion.
AngerAnger is an unpleasant emotion involving a strong, uncomfortable, and hostile response to a perceived provocation, hurt, or threat. Anger usually has many physical and mental consequences.

In CrystalFeel, the conceptual meaning of anger is consistent with Plutchik’s psychoevolutional theory of emotions (Plutchik, 1980; Plutchik, 1991, pp. 78-84). Anger may range from low-intensity annoyance to high-intensity rage.

The anger emotions in CrystalFeel may correspond to a group of four fine-grained emotions discussed in the OCC’s appraisal theory of emotions, including resentment (or envy, jealousy), reproach (or contempt), anger, and disliking (or disgust, hate) (Ortony et al., 1998).		tAnger is a continuous variable ranging from 0 to 1, where 0 indicates that this text does not express the anger emotion at all, and 1 indicates that this text expresses an extremely high intensity of the anger emotion.
Joy (or Happiness)Joy refers to a positive emotion of gladness, delight, or exultation of the spirit arising from a sense of well-belling or satisfaction. In English, joy may refer to a particular feeling of great happiness.

In CrystalFeel, the conceptual meaning of joy is consistent with Plutchik’s psychoevolutional theory of emotions (Plutchik, 1980; Plutchik, 1991, pp. 85-91). Joy may range from low-intensity contentment to high-intensity elation.

Note that the joy emotions in CrystalFeel may correspond to a very comprehensive group of eleven fine-grained positively valenced emotions discussed in the OCC’s appraisal theory of emotions, including joy (happiness, elation), happy-for, gloating, hope (anticipation, optimistic), satisfaction, relief, pride, appreciation, admiration, gratitude (thankfulness, appreciation), gratification (self-satisfaction, smug, pleased-with-oneself), and liking (affection, attracted-to, love) (Ortony et al., 1998).		tJoy is a continuous variable ranging from 0 to 1, where 0 indicates that this text does not express the joy emotion at all, and 1 indicates that this text expresses an extremely high intensity of the joy emotion.
SadnessSadness is an unpleasant emotion characterized by feelings of loss, disadvantage, helplessness, disappointment, sorrow, and despair. Sadness may often lead to silence, inaction, withdrawal from others, and in extreme cases, depression.

In CrystalFeel, the conceptual meaning of sadness is consistent with Plutchik’s psychoevolutional theory of emotions (Plutchik, 1980; Plutchik, 1991, pp. 91-95 on grief). Sadness may range from low-intensity distress to high-intensity grief.

The sadness emotions in CrystalFeel may correspond to a very comprehensive group of six fine-grained emotions discussed in the OCC’s appraisal theory of emotions, including distress (unhappy, sad, grief), sorry-for (pity), fears-confirmed, disappointment, shame (self-reproach, guilt, embarrassment), and remorse (self-anger) (Ortony et al., 1998).		tSadness is a continuous variable ranging from 0 to 1, where 0 indicates that this text does not express the sadness emotion at all, and 1 indicates that this text expresses an extremely high intensity of the sadness emotion.
Valence In psychology, emotional valence refers to the overall unpleasantness (negative feelings) or pleasantness (positive feelings) of an emotional experience or expression.

In CrystalFeel, the conceptual meaning of valence is aligned with the dimensional view of emotions, such as the circumplex model of emotions (Russel, 1980).

Note that the term valence is loosely related to and often used interchangeably with the term sentiment, though they are different.		tValence is a continuous variable ranging from 0 to 1, where 0 indicates that this text expresses extremely negative or unpleasant feelings, and 1 indicates that this text expresses extremely positive or pleasant feelings.

Qualitative Outputs - Sentiment and Emotion Labels

The CrystalFeel algorithms also generate classification-based labels (i.e., qualitative, categorical values) based on the intensity scores. Such classification outcomes can often be used for more straightforward descriptive analyses.

Output Dimension
Conceptual Meaning
Value Range
EmotionIn psychology, emotion refers to a a complex reaction pattern, involving experiential, behavioural, and physiological elements, by which an individual attempts to deal with a personally significant matter or event.

CrystalFeel differentiates five emotion classes: fear, anger, joy, sadness, and no specific emotion. The logic that is used to convert the emotion intensity scores to this emotion category can be found here.		tEmotion is a categorical variable that indicates the text mainly expresses one of the five emotion classes: fear, anger, joy, sadness, and no specific emotion.
SentimentAccording to Cambridge Dictionary, sentiment refers to a a general feeling, attitude, or opinion about something, or a way of thinking about something.

Sentiment is commonly classified into three classes as negative, positive, or neutral.

CrystalFeel differentiates five fine-grained sentiment classes: very negative, negative, neutral or mixed, positive and very positive. The logic that is used to convert the emotion intensity scores to this sentiment category can be found here.		tSentiment is a categorical variable that indicates the text mainly expresses one of the five sentiment classes: very negative, negative, neutral or mixed, positive, and very positive.

Notes on general use

The conversion logic is based on application assumptions where CrystalFeel is used for processing short informal text (e.g., tweets, Facebook posts, and comments) for a general, open-domain purpose.

For specific applications, users may adjust their conversion logic as appropriate or suitable for different analytical goals. For example, for measuring emotions from more formal text inputs (e.g., news headlines), the conversion logic shall be adjusted accordingly based on domain-specific ground truth data. For critical applications, users may consider defining a purpose-specific annotation scheme, create an annotated dataset, re-train, and calibrate the CrystalFeel outputs to achieve optimal predictive accuracy.

Why is it valuable to understand and measure emotions?

From the scientific literature of emotions, we know that emotions are valenced (positively or negatively) reactions to their eliciting situations, such as an event, an agent, or an object. Therefore, there are at least three reasons for us to understand and measure emotions.

First, emotions reflect how people respond to real-world stimuli, such as events, agents, and objects. For example, emotion measures can be used as sensitive descriptive indicators to track the evolution of events, especially during a crisis (e.g., how people respond to an epidemic). Emotions can be analyzed to surface potential causes of concerns towards agents (e.g., those who are blamed for undesirable situations and for what reasons). Emotions can also be analyzed to help evaluate objects (e.g., finding out which products or attributes bring delight to consumers).

Second, some emotions such as fear and anger, especially in high intensity, have behavioral implications. Fear can cause flight and anger can cause fight. On the other hand, prolonged sadness harms mental and physical health or even causes suicidal ideation. Timely and accurate sensing of such emotions could provide actionable insights for the stakeholders to prioritize communication, make informed decision strategies, and plan necessary resources.

Third and more profoundly, emotions arise as a situation touches at least one of our concerns. Because emotions are rooted in goals, standards, principles, and attitudes, a good understanding and ability to measure emotions can illuminate a person‘s underlying concerns.

Concepts related to emotions in the context of human experiences


Emotion concepts vary not only in distinct categories and dimensions but also in the intensity of emotional experiences

Emotion is one of the most central, complex, and natural aspects of human experience. There are at least three essential theoretical concepts that pertain to the very structure of emotions.

Three structural concepts of emotions

One stream of work focuses on categorizing emotions as discrete and distinct types, such as fear, anger, joy (or happiness), and sadness. For example, Ekman (1993), the probably most widely cited emotion model in emotion recognition technologies, drew the evidence from studies of facial expressions and focuses on six basic emotions: fear, anger, happiness, sadness, disgust, and surprise. Robert Plutchik focuses on relations to adaptation in evolution and proposes a psychoevolutional theory of emotions. Plutchik (1980, 199) proposes eight primary emotions (fear, anger, joy, grief, acceptance, disgust, anticipation, surprise) and various emotion mixes (e.g., love, optimism, pessimism). On the other hand, Andrew Ortony, Gerald Clore, and Allan Collins analyze the cognitive origins of emotions and propose an appraisal theory covering twenty-two fine-grained emotions (Ortony et al., 1988).

Second, there is a dimensional view of emotions which receives increasing popularity in the computational research communities. The dimensions are like characteristics of emotions, and examples include valence (degree of hedonic quality/ pleasantness), arousal (physical activation), dominance (control/power), and expectancy (degree of anticipation/expectation). Using the dimensional variables such as valence and arousal, different emotional states like happy, angry, sad, and afraid can be mapped to a two-dimensional scale, known as the circumplex model of affect (Russel, 1980).

Third, we consider the intensity of emotions—the degree or depth of the emotion being experienced or expressed—a very salient aspect of human emotional experiences. This concept of emotion intensity is highlighted in the cognitive theories of emotions (Frijda,1986; Frijda et al., 1992; Ortony et al., 1988). However, emotion intensity still represents a relatively new, less explored aspect of emotional information that can benefit computational methods and their applications.

The following examples show various linguistic expressions of emotions with different degree of fear (see low-intensity vs. high-intensity examples [1, 2]), anger ([3, 4]), joy ([7, 8]), and sadness ([5, 6]).

Fear
[1]   They are building a shell command on a server, combining that with user input, and then executing that in a shell on the client. #shudder
[2]   Let‘s hope the ct scan gives us some answers on this lump today #nervous

Anger
[3]   No words Sir... Thank you for the concern..
[4]   Everything I order online just comes looking like a piece of shit

Joy
[5]   You‘re here to feed me. I won‘t die of starvation, he said, slightly smiling. I frowned. Panira. Kainis.
[6]   Omgsh Alexis is sooooo freaking funny on #BachelorInParadise That pizza segment! Plus I love her and Jazzy’s friendship!

Sadness
[7]   I will not fall to the dark side
[8]   That moment when you look back and realise you‘ve been a #selfish #horrible #judgemental person. #FeelingAshamed


Primary or basic emotions

There is substantial debate about the basicality of emotions (Ortony and Turner, 1991).

Our priority is to start with emotions that are valuable for analytical applications. Hence, CrystalFeel currently focuses on fear, anger, joy, and sadness: the four emotions are presented in the order of their adaptive relations to evolutions consistent with Plutchik‘s psychoevoluational theory of emotions (1991).

The four primary emotions and their adaptive relations to evolutions

Emotion measurement from natural language: Word-level vs. message-level analysis

We learn from the science of emotions that emotions are mental experiences and are not directly observable. We can directly observe the facial, tonal, verbal/written expressions, movements, and behaviors, using which we can only indirectly observe or infer emotions. Here, of particular interest in our research is to infer emotions from natural language.

Psychologists have conventionally employed a computer program that counts words and word frequency as the sole features to infer the likely emotional states expressed in a text. While words indeed carry semantic meanings, simple word-level analysis using word count and frequency are limited in their effectiveness. In particular, a word-level analysis may work for sufficiently long text (e.g., essays, news articles) but not for short text messages (e.g., tweets, comments, headlines). It requires sentence-level or message-level analysis to infer the latent meaning when the linguistic context is in place (see examples [11 - 14]). Moreover, people may not use affective words to express their emotions [15 - 18].

There are emotional expressions with emotional words, but the word-level and sentence-level meanings are very different:

[11]   Don’t be scared of trying something new. Fear nothing. If you believe in it, go for it!
[12]   Parliament: Hate speech may be handled differently elsewhere, but Singapore must be strict on it, says Shanmugam
[13]   Arrrhhh I hardly feel happy any more these day...
[14]   He cried when he heard that his son had been found alive and well.

On the other hand, there are emotional expressions without the use of specific emotional words used:

[15]   No not going out...may get contacted with the virus
[16]   Get your manager out! I don’t want to talk to you!
[17]   Woohoo all tasks done!!!
[18]   What to do with my life... I have no more choices...


Our innovation and approach

We experimented with features derived from multiple affective lexicons (including our in-house developed Emotion Intensity lexicons) in conjunction with parts-of-speech, n-grams, word embedding, and pre-trained language models to predict the degree of the intensity associated with fear, anger, sadness, and joy in a tweet at the sentence or message level. We found that including the affective lexicons-based features allowed the system to obtain strong prediction performance while revealing interesting emotion word-level and message-level associations. One key design feature is our in-house curated enhanced emotion intensity (E2I) lexicon. The E2I lexicon helps the CrystalFeel machines not only able to differentiate positive and negative words but also the intensity, the emotion type, the psychological condition, and the polarity property of the emotion-bearing words. Our experiments showed that hybrid features performed much better than affective lexicons-based features alone or word embedding / pre-trained language models alone.

This science behind the earlier development of CrystalFeel is described in a paper published and presented at the 12th International Workshop on Semantic Evaluation (SemEval 2018).

To date, CrystalFeel engine‘s predicted accuracy, based on a Pearson correlation coefficient (r) value evaluating against out-of-training test sample of human annotations as the ground truth, had 0.765, 0.818, 0.788, 0.765, and 0.856 on predicting fear intensity, anger intensity, joy intensity, sadness intensity and valence intensity, respectively.


References and advanced readings

The following provides some helpful references related to CrystalFeel’s psychological and conceptual foundations.

  • Clore, G. L., Ortony, A., & Foss, M. A. (1987). The psychological foundations of the affective lexicon. Journal of Personality and Social Psychology, 53(4), pp. 751–766.
  • Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384–392.
  • Frijda, N.H. (1986). The emotions: Studies in emotion and social interaction. London, England: Cambridge University Press.
  • Frijda, N. H., Ortony, A., Sonnemans, J., & Clore, G. L. (1992). The complexity of intensity: Issues concerning the structure of emotion intensity. In M. S. Clark (Ed.), Review of Personality and Social Psychology, No. 13. Emotion (pp. 60–89). Sage Publications, Inc.
  • Plutchik, R. (1980). A general psychoevolutionary theory of emotion. In R. Plutchik & H. Kellerman (Eds.), Emotion: Theory, research, and experience: Vol. 1. Theories of emotion (pp. 3-33). New York: Academic.
  • Plutchik, R. (1991). The Emotions: Revised Edition, Random House, Inc.: University Press of America.
  • Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge University Press.
  • Ortony, A. & Turner, T. J. (1990). What's basic about basic emotions? Psychological Review, 97, pp. 315-331.
  • Russell, J. A. (1980). A circumplex model of affect, Journal of Personality and Social Psychology, 39(6), pp. 1161–1178
  • Shaver, P., Schwartz, J., Kirson, D. & O’Connor, C. (1987). Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52(6), pp. 1061–1086.
  • Sonnemans, J., & Frijda, N. H. (1994). The structure of subjective emotion intensity. Cognition and Emotion, 8(4), 329–350.
  • Sonnemans, J., & Frijda, N. H. (1995). The determinants of subjective emotion intensity. Cognition and Emotion, 9(5), 483–506.


The following provides references related to CrystalFeel’s initial development.

  • Gupta, R.K. and Yang, Y. (2017). CrystalNest at SemEval-2017 Task 4: Using sarcasm detection for enhancing sentiment classification and quantification, In Proceedings of The 11th International Workshop on Semantic Evaluation (ACL-SemEval 2017), 3-4 August 2017, Vancouver, Canada https://www.aclweb.org/anthology/S17-2103.pdf
  • Gupta, R.K. and Yang, Y. (2018). CrystalFeel at SemEval-2018 Task 1: Understanding and detecting intensity of emotions using affective lexicons, In Proceedings of The 12th International Workshop on Semantic Evaluation (SemEval @ NAACL-HLT 2018), Jun 5-6, 2018, New Orleans, USA. https://www.aclweb.org/anthology/S18-1038.pdf
  • CrystalFeel – Multidimensional Emotion Intensity Analysis from Natural Language. 2021. Institute of High Performance Computing, A*STAR. Available online: https://socialanalyticsplus.net/crystalfeel



The references help to show the underlying science behind and the experimental processes that illustrate the approach. However, it is helpful to note that technologies could have evolved with more advanced components, and the latest engine does not necessarily follow the descriptions as the originally published papers.

Theoretically speaking, accurately measuring the degree or intensity of emotions is beneficial to many applications. For example, a virtual service assistant would be able to employ a more appropriate response strategy when a wave of high-intensity anger or frustration is sensed from its customer compared to respond monotonically in usual dialogues. Customer relationship management systems can be more targeted by engaging customers who express high degrees of joy or excitement with their products and services. Homecare robots, empowered with the ability to recognize high-intensity grief or distress, would be less likely to miss the opportunity to alert professional human caregivers.

CrystalFeel has two broad categories of applications:

First, CrystalFeel outputs can be used as descriptive measures of sentiment and emotions in natural language. The five emotional intensity scores are psychologically meaningful. They can be directly used in analytic applications to measure, rank, sort, categorize, and aggregate a list of text data, independently on the text-based data stream or overlaid with other data streams.

Second, CrystalFeel can be used as predictive features to training and developing new predictive models while maintaining a reasonable degree of psychological interpretability. The scores also serve as valuable training features and develop new machine learning methods for advanced NLP and multimodal AI applications.

Check out some studies that used CrystalFeel in addressing various research questions:



For interest and enquires, please contact us.



Terms of Use    Privacy Statement