<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="abzaliev-mihalcea-2024-unsupervised">
    <titleInfo>
        <title>Unsupervised Discrete Representations of American Sign Language</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Artem</namePart>
        <namePart type="family">Abzaliev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rada</namePart>
        <namePart type="family">Mihalcea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Many modalities are naturally represented as continuous signals, making it difficult to use them with models that expect discrete units, such as LLMs. In this paper, we explore the use of audio compression techniques for the discrete representation of the gestures used in sign language. We train a tokenizer for American Sign Language (ASL) fingerspelling, which discretizes sequences of fingerspelling signs into tokens. We also propose a loss function to improve the interpretability of these tokens such that they preserve both the semantic and the visual information of the signal. We show that the proposed method improves the performance of the discretized sequence on downstream tasks.</abstract>
    <identifier type="citekey">abzaliev-mihalcea-2024-unsupervised</identifier>
    <identifier type="doi">10.18653/v1/2024.emnlp-main.1104</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.1104/</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>19786</start>
            <end>19793</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Unsupervised Discrete Representations of American Sign Language
%A Abzaliev, Artem
%A Mihalcea, Rada
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F abzaliev-mihalcea-2024-unsupervised
%X Many modalities are naturally represented as continuous signals, making it difficult to use them with models that expect discrete units, such as LLMs. In this paper, we explore the use of audio compression techniques for the discrete representation of the gestures used in sign language. We train a tokenizer for American Sign Language (ASL) fingerspelling, which discretizes sequences of fingerspelling signs into tokens. We also propose a loss function to improve the interpretability of these tokens such that they preserve both the semantic and the visual information of the signal. We show that the proposed method improves the performance of the discretized sequence on downstream tasks.
%R 10.18653/v1/2024.emnlp-main.1104
%U https://aclanthology.org/2024.emnlp-main.1104/
%U https://doi.org/10.18653/v1/2024.emnlp-main.1104
%P 19786-19793

Markdown (Informal)

[Unsupervised Discrete Representations of American Sign Language](https://aclanthology.org/2024.emnlp-main.1104/) (Abzaliev & Mihalcea, EMNLP 2024)

• Unsupervised Discrete Representations of American Sign Language (Abzaliev & Mihalcea, EMNLP 2024)

ACL

• Artem Abzaliev and Rada Mihalcea. 2024. Unsupervised Discrete Representations of American Sign Language. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19786–19793, Miami, Florida, USA. Association for Computational Linguistics.