2014
pdf
bib
abs
DCEP -Digital Corpus of the European Parliament
Najeh Hajlaoui
|
David Kolovratnik
|
Jaakko Väyrynen
|
Ralf Steinberger
|
Daniel Varga
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)
We are presenting a new highly multilingual document-aligned parallel corpus called DCEP - Digital Corpus of the European Parliament. It consists of various document types covering a wide range of subject domains. With a total of 1.37 billion words in 23 languages (253 language pairs), gathered in the course of ten years, this is the largest single release of documents by a European Union institution. DCEP contains most of the content of the European Parliament’s official Website. It includes different document types produced between 2001 and 2012, excluding only the documents already exist in the Europarl corpus to avoid overlapping. We are presenting the typical acquisition steps of the DCEP corpus: data access, document alignment, sentence splitting, normalisation and tokenisation, and sentence alignment efforts. The sentence-level alignment is still in progress but based on some first experiments; we showed that DCEP is very useful for NLP applications, in particular for Statistical Machine Translation.
2012
pdf
bib
abs
Rapid creation of large-scale corpora and frequency dictionaries
Attila Zséder
|
Gábor Recski
|
Dániel Varga
|
András Kornai
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
We describe, and make public, large-scale language resources and the toolchain used in their creation, for fifteen medium density European languages: Catalan, Czech, Croatian, Danish, Dutch, Finnish, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Serbian, Slovak, Spanish, and Swedish. To make the process uniform across languages, we selected tools that are either language-independent or easily customizable for each language, and reimplemented all stages that were taking too long. To achieve processing times that are insignificant compared to the time data collection (crawling) takes, we reimplemented the standard sentence- and word-level tokenizers and created new boilerplate and near-duplicate detection algorithms. Preliminary experiments with non-European languages indicate that our methods are now applicable not just to our sample, but the entire population of digitally viable languages, with the main limiting factor being the availability of high quality stemmers.
2008
pdf
bib
abs
Parallel Creation of Gigaword Corpora for Medium Density Languages - an Interim Report
Péter Halácsy
|
András Kornai
|
Péter Németh
|
Dániel Varga
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)
For increased speed in developing gigaword language resources for medium resource density languages we integrated several FOSS tools in the HUN* toolkit. While the speed and efficiency of the resulting pipeline has surpassed our expectations, our experience in developing LDC-style resource packages for Uzbek and Kurdish makes clear that neither the data collection nor the subsequent processing stages can be fully automated.
2007
pdf
bib
GYDER: Maxent Metonymy Resolution
Richárd Farkas
|
Eszter Simon
|
György Szarvas
|
Dániel Varga
Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007)
2006
pdf
bib
Web-based frequency dictionaries for medium density languages
András Kornai
|
Péter Halácsy
|
Viktor Nagy
|
Csaba Oravecz
|
Viktor Trón
|
Dániel Varga
Proceedings of the 2nd International Workshop on Web as Corpus
pdf
bib
abs
The JRC-Acquis: A Multilingual Aligned Parallel Corpus with 20+ Languages
Ralf Steinberger
|
Bruno Pouliquen
|
Anna Widiger
|
Camelia Ignat
|
Tomaž Erjavec
|
Dan Tufiş
|
Dániel Varga
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)
We present a new, unique and freely available parallel corpus containing European Union (EU) documents of mostly legal nature. It is available in all 20 official EU languages, with additional documents being available in the languages of the EU candidate countries. The corpus consists of almost 8,000 documents per language, with an average size of nearly 9 million words per language. Pair-wise paragraph alignment information produced by two different aligners (Vanilla and HunAlign) is available for all 190+ language pair combinations. Most texts have been manually classified according to the EUROVOC subject domains so that the collection can also be used to train and test multi-label classification algorithms and keyword-assignment software. The corpus is encoded in XML, according to the Text Encoding Initiative Guidelines. Due to the large number of parallel texts in many languages, the JRC-Acquis is particularly suitable to carry out all types of cross-language research, as well as to test and benchmark text analysis software across different languages (for instance for alignment, sentence splitting and term extraction).
pdf
bib
abs
Using a morphological analyzer in high precision POS tagging of Hungarian
Péter Halácsy
|
András Kornai
|
Csaba Oravecz
|
Viktor Trón
|
Dániel Varga
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)
The paper presents an evaluation of maxent POS disambiguation systems that incorporate an open source morphological analyzer to constrain the probabilistic models. The experiments show that the best proposed architecture, which is the first application of the maximum entropy framework in a Hungarian NLP task, outperforms comparable state of the art tagging methods and is able to handle out of vocabulary items robustly, allowing for efficient analysis of large (web-based) corpora.
2005
pdf
bib
Hunmorph: Open Source Word Analysis
Viktor Trón
|
Gyögy Gyepesi
|
Péter Halácsky
|
András Kornai
|
László Németh
|
Dániel Varga
Proceedings of Workshop on Software
1969
pdf
bib
Problem of Improving the Efficiency of Parsing Systems
D. Varga
International Conference on Computational Linguistics COLING 1969: Preprint No. 61
1965
pdf
bib
Syntactic Analysis in the Case of Highly Inflecting Languages
D. Varga
COLING 1965