Dataset
Last updated
Last updated
There are a lot of memes and a myriad of them are created every day. We did some research on existing datasets and found "Memotion 2: Dataset on sentiment and emotion analysis of meme" which has the advantage of containing many memes but has limitations: mostly images without text (there are rarely captions) and always uses a macro model.
We also found further limitations: the analysis of sentiment is conducted exclusively on captions and the reference only indicates the source from which the meme was taken. These are significant limitations to our research, so we decided not to rely on existing datasets.
Moreover, as of now automatic knowledge extraction from memes presents significant challenges due to the inherently multimodal nature of memes, which combine visual and textual elements to convey nuanced meanings. Current AI tools struggle with the subtlety and context-dependent nature of memes, where the same text can have different implications based on the accompanying image. For example, benign sentences paired with certain images can transform into offensive or hateful messages. Moreover, the inclusion of "benign confounders"—alternative images or texts that flip the meaning of the meme—complicates the task further, as models must distinguish between these nuanced variations to correctly interpret the content. This necessity for sophisticated multimodal reasoning highlights the limitations of unimodal models, which often fail to capture the complex interplay between text and image. Consequently, there remains a substantial gap between human and machine performance in this domain, underscoring the ongoing need for advancements in multimodal AI capabilities.
For these reasons, we decided on opting for the creation of a synthetic dataset to test our ontology.
During the analysis of specific meme scenarios, we found problems with the use of the LLM, related to the limits of the topics covered and the generation of synthetic data.
The model tends to avoid controversial or sensitive topics in order to adhere to principles of political correctness, reducing the variety and depth of scenarios produced. This limits the LLM’s ability to provide a complete and accurate representation of the memes studied. In addition, the summary data generated may be limited and not representative of reality.
ChatGPT has very restrictive policies, particularly regarding current political events and other sensitive topics. In contrast, Cloud has proven to be more effective due to its less restrictive approach in these areas. This distinction is particularly significant when dealing with sarcastic and ironic memes or addressing personal and controversial opinions. The limitations imposed by ChatGPT can impede its ability to accurately capture and convey the nuances of such content, affecting the overall analysis of memes with these complex connotations.
To better align with our needs, we adjusted several values and labels. For example, ChatGPT encountered difficulties with the "Uncontroversial-Controversial" opinion, confusing it with "Controversial" and "Division," and attributing positive connotations to both. In our ontology, however, "Controversial" is defined to represent a provocative opinion that triggers debates and discussions, and it carries a "negative connotation." This means that within our framework, "Controversial" is associated with opinions that challenge norms and provoke strong reactions, which may not always be positive.
To produce the JSON file we used this prompt to be inserted in the LLM:
It is more useful to generate data separately for each template rather than creating a single JSON for all templates. This approach allows for better specification of requirements for data generation, resulting in more accurate and relevant outcomes for each template.
Specification of All Used Entries:
1. Eventuality
Type: Political Event. This specifies that the meme is related to a political situation or issue.
Specific: Brexit. The specific political event being referenced in the meme is Brexit, which is the United Kingdom's departure from the European Union.
2. Template: ChangeMyMind
3. Text: "Brexit negotiations are more confusing than IKEA instructions; Change my mind". The text is the caption of the meme. It humorously compares the complexity of Brexit negotiations to the often confusing assembly instructions of IKEA furniture, suggesting that the former is even more confusing.
4. Graphical Features
5. Connotation: Sarcastic. The meme uses sarcasm to downplay or mock the complexity of Brexit negotiations by comparing them to IKEA instructions, suggesting that the negotiations are needlessly complex.
6. Attitude
Opinionated: False. The meme does not present a strong personal opinion; rather, it uses humor and sarcasm.
Troll: True. The meme is intended to provoke or amuse.
7. Opinion: Uncontroversial
The meme’s stance is presented as relatively mild, aiming to entertain or lightly criticize without diving into deeper controversies.
This JSON structure allows for a detailed description of the meme, including its type, visual and textual components, and the nature of its commentary and attitude.
In order to perform queries on our dataset, we wrote a python script to transform the JSON file in graph database, according to the ontology that we defined.
