| Title: |
Embracing Diversity: A Multi-Perspective Approach with Soft Labels |
| Authors: |
Muscato, Benedetta, Bushipaka, Praveen, Gezici, Gizem, Passaro, Lucia, Giannotti, Fosca, Cucinotta, Tommaso |
| Publication Year: |
2025 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction |
| More Details: |
Prior studies show that adopting the annotation diversity shaped by different backgrounds and life experiences and incorporating them into the model learning, i.e. multi-perspective approach, contribute to the development of more responsible models. Thus, in this paper we propose a new framework for designing and further evaluating perspective-aware models on stance detection task,in which multiple annotators assign stances based on a controversial topic. We also share a new dataset established through obtaining both human and LLM annotations. Results show that the multi-perspective approach yields better classification performance (higher F1-scores), outperforming the traditional approaches that use a single ground-truth, while displaying lower model confidence scores, probably due to the high level of subjectivity of the stance detection task. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2503.00489 |
| Accession Number: |
edsarx.2503.00489 |
| Database: |
arXiv |
