<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yin-etal-2024-empirical">
    <titleInfo>
        <title>Empirical Prior for Text Autoencoders</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yongjing</namePart>
        <namePart type="family">Yin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wenyang</namePart>
        <namePart type="family">Gao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Haodong</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jianhao</namePart>
        <namePart type="family">Yan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yue</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper explores the application of Variational Autoencoders (VAE) in text generation, focusing on overcoming challenges like posterior collapse and the limitations of simplistic prior distributions. We investigate a transition from VAE to text autoencoders (AE), which model a compact latent space and preserves the capability of the language model itself. Our method involves layer-wise latent vectors regularized by orthogonal constraints to encourage distinct semantic spaces. In particular, we estimate an empirical prior online from the learned latent vectors to support sampling during generation like VAE. Experimental results on standard benchmarks demonstrate that the autoencoders generate higher quality and more diverse text than the VAE-based Transformer baselines, offering an effective alternative for generative language modeling.</abstract>
    <identifier type="citekey">yin-etal-2024-empirical</identifier>
    <identifier type="doi">10.18653/v1/2024.findings-emnlp.796</identifier>
    <location>
        <url>https://aclanthology.org/2024.findings-emnlp.796/</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>13628</start>
            <end>13640</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Empirical Prior for Text Autoencoders
%A Yin, Yongjing
%A Gao, Wenyang
%A Wu, Haodong
%A Yan, Jianhao
%A Zhang, Yue
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Findings of the Association for Computational Linguistics: EMNLP 2024
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F yin-etal-2024-empirical
%X This paper explores the application of Variational Autoencoders (VAE) in text generation, focusing on overcoming challenges like posterior collapse and the limitations of simplistic prior distributions. We investigate a transition from VAE to text autoencoders (AE), which model a compact latent space and preserves the capability of the language model itself. Our method involves layer-wise latent vectors regularized by orthogonal constraints to encourage distinct semantic spaces. In particular, we estimate an empirical prior online from the learned latent vectors to support sampling during generation like VAE. Experimental results on standard benchmarks demonstrate that the autoencoders generate higher quality and more diverse text than the VAE-based Transformer baselines, offering an effective alternative for generative language modeling.
%R 10.18653/v1/2024.findings-emnlp.796
%U https://aclanthology.org/2024.findings-emnlp.796/
%U https://doi.org/10.18653/v1/2024.findings-emnlp.796
%P 13628-13640

Markdown (Informal)

[Empirical Prior for Text Autoencoders](https://aclanthology.org/2024.findings-emnlp.796/) (Yin et al., Findings 2024)

• Empirical Prior for Text Autoencoders (Yin et al., Findings 2024)

ACL

• Yongjing Yin, Wenyang Gao, Haodong Wu, Jianhao Yan, and Yue Zhang. 2024. Empirical Prior for Text Autoencoders. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13628–13640, Miami, Florida, USA. Association for Computational Linguistics.