Siddharth Patwardhan


2023

pdf bib
EELBERT: Tiny Models through Dynamic Embeddings
Gabrielle Cohn | Rishika Agarwal | Deepanshu Gupta | Siddharth Patwardhan
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track

We introduce EELBERT, an approach for compression of transformer-based models (e.g., BERT), with minimal impact on the accuracy of downstream tasks. This is achieved by replacing the input embedding layer of the model with dynamic, i.e. on-the-fly, embedding computations. Since the input embedding layer occupies a large portion of the model size, especially for the smaller BERT variants, replacing this layer with an embedding computation function helps us reduce the model size significantly. Empirical evaluation on the GLUE benchmark shows that our BERT variants (EELBERT) suffer minimal regression compared to the traditional BERT models. Through this approach, we are able to develop our smallest model UNO-EELBERT, which achieves a GLUE score within 4% of fully trained BERT-tiny, while being 15x smaller (1.2 MB) in size.

2016

pdf bib
Addressing Limited Data for Textual Entailment Across Domains
Chaitanya Shivade | Preethi Raghavan | Siddharth Patwardhan
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
The Role of Context Types and Dimensionality in Learning Word Embeddings
Oren Melamud | David McClosky | Siddharth Patwardhan | Mohit Bansal
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2013

pdf bib
Semantic Technologies in IBM Watson
Alfio Gliozzo | Or Biran | Siddharth Patwardhan | Kathleen McKeown
Proceedings of the Fourth Workshop on Teaching NLP and CL

pdf bib
Long-Distance Time-Event Relation Extraction
Alessandro Moschitti | Siddharth Patwardhan | Chris Welty
Proceedings of the Sixth International Joint Conference on Natural Language Processing

2012

pdf bib
When Did that Happen? — Linking Events and Relations to Timestamps
Dirk Hovy | James Fan | Alfio Gliozzo | Siddharth Patwardhan | Christopher Welty
Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics

2011

pdf bib
Using Syntactic and Semantic Structural Kernels for Classifying Definition Questions in Jeopardy!
Alessandro Moschitti | Jennifer Chu-Carroll | Siddharth Patwardhan | James Fan | Giuseppe Riccardi
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

2010

pdf bib
Learning to Predict Readability using Diverse Linguistic Features
Rohit Kate | Xiaoqiang Luo | Siddharth Patwardhan | Martin Franz | Radu Florian | Raymond Mooney | Salim Roukos | Chris Welty
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)

2009

pdf bib
A Unified Model of Phrasal and Sentential Evidence for Information Extraction
Siddharth Patwardhan | Ellen Riloff
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

2007

pdf bib
UMND1: Unsupervised Word Sense Disambiguation Using Contextual Semantic Relatedness
Siddharth Patwardhan | Satanjeev Banerjee | Ted Pedersen
Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007)

pdf bib
Effective Information Extraction with Semantic Affinity Patterns and Relevant Regions
Siddharth Patwardhan | Ellen Riloff
Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)

2006

pdf bib
Learning Domain-Specific Information Extraction Patterns from the Web
Siddharth Patwardhan | Ellen Riloff
Proceedings of the Workshop on Information Extraction Beyond The Document

pdf bib
Feature Subsumption for Opinion Analysis
Ellen Riloff | Siddharth Patwardhan | Janyce Wiebe
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

pdf bib
Using WordNet-based Context Vectors to Estimate the Semantic Relatedness of Concepts
Siddharth Patwardhan | Ted Pedersen
Proceedings of the Workshop on Making Sense of Sense: Bringing Psycholinguistics and Computational Linguistics Together

2005

pdf bib
SenseRelate::TargetWord—A Generalized Framework for Word Sense Disambiguation
Siddharth Patwardhan | Satanjeev Banerjee | Ted Pedersen
Proceedings of the ACL Interactive Poster and Demonstration Sessions

pdf bib
Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns
Yejin Choi | Claire Cardie | Ellen Riloff | Siddharth Patwardhan
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing

pdf bib
OpinionFinder: A System for Subjectivity Analysis
Theresa Wilson | Paul Hoffmann | Swapna Somasundaran | Jason Kessler | Janyce Wiebe | Yejin Choi | Claire Cardie | Ellen Riloff | Siddharth Patwardhan
Proceedings of HLT/EMNLP 2005 Interactive Demonstrations

2004

pdf bib
Generative models for semantic role labeling
Cynthia A. Thompson | Siddharth Patwardhan | Carolin Arnold
Proceedings of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text

pdf bib
WordNet::Similarity - Measuring the Relatedness of Concepts
Ted Pedersen | Siddharth Patwardhan | Jason Michelizzi
Demonstration Papers at HLT-NAACL 2004