NLP Reading Group

Iran University of Science and Technology

Moderator: Mohammad Taher Pilehvar

Wednesdays, 11am-12pm (online on Teams)

Contact the moderator if you are interested in joining the group.


Summer 2021

Date Presenters Topic / Paper
8 September Hosein Mohebbi, Zahra Sayedi
  • Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience, EMNLP 2021. [slides]
  • BoB: BERT Over BERT for Training Persona-based Dialogue Models, ACL 2021.
1 September Mahdi Zakizadeh, Amin Pourdabiri
  • Towards Emotional Support Dialog Systems, ACL 2021. [slides]
18 August National holiday --
11 August Hosein Mohebbi, Mohsen Tabasi
  • Probing BERT in Hyperbolic Spaces, ICLR 2021. [slides]
  • Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning, ACL 2021.
4 August Tohid Abedini, Mohammad Ali Modarressi
  • Combating Adversarial Misspellings with Robust Word Recognition, ACL 2019. [slides]
  • On Attention Redundancy: A Comprehensive Study, NAACL 2021.
28 July Houman Mehrafarin, Maryam Sadat Hashemi
  • An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models, TACL 2020.
  • Charformer: Fast Character Transformers via Gradient-based Subword Tokenization, Arxiv 2021.
21 July National holiday --
14 July Kiamehr Rezaee, Zahra Sayedi
  • Generationary or: “How We Went beyond Word Sense Inventories and Learned to Gloss”, EMNLP 2020.
  • All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text, ACL 2021.
7 July Hosein Mohebbi, Mohsen Tabasi
  • MPNet: Masked and Permuted Pre-training for Language Understanding , NeurIPS 2020. [slides]
  • Comparing Test Sets with Item Response Theory, ACL 2021.
30 June Amin Pourdabiri, Sara Rajaee
  • SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations, Arxiv 2021. [slides]
  • Positional Artefacts Propagate Through Masked Language Model Embeddings, ACL 2021. [slides]
23 June Houman Mehrafarin, Maryam Sadat Hashemi
  • Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work? ACL 2020.
  • Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions, NAACL 2021. [slides]


Spring 2021

Date Presenters Topic / Paper
16 June Hosein Mohebbi, Sara Rajaee
  • TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference, NAACL 2021. [slides]
  • How transfer learning impacts linguistic knowledge in deep NLP models? ACL 2021. [slides]
9 June Mohsen Tabasi, Mohammad Ali Modarressi
  • Entailment as Few-Shot Learner, Arxiv 2021.
  • BERT Busters: Outlier LayerNorm Dimensions that Disrupt Transformers, Findings of ACL 2021.
2 June Sara Rajaee, Maryam Sadat Hashemi
  • A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space, ACL 2021.
  • FNet: Mixing Tokens with Fourier Transforms, Arxiv 2021. [slides]
26 May Kiamehr Rezaee
  • Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer, ACL 2020.
19 May Amin Pourdabiri
  • Situated and Interactive Multimodal Conversations, COLING 2020. [slides]
12 May EMNLP deadline --
5 May Zahra Sayedi, Houman Mehrafarin
  • Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue, ICLR 2020.
  • What Happens To BERT Embeddings During Fine Tuning, BlackboxNLP 2020.
28 April Hosein Mohebbi, Mohammad Ali Modarressi
  • DirectProbe: Studying Representations without Classifiers, NAACL 2021. [slides]
  • Telling BERT’s Full Story: from Local Attention to Global Aggregation, EACL 2021.
21 April Mohsen Tabasi, Zahra Sayedi
  • Static Embeddings as Efficient Knowledge Bases? NAACL 2021.
  • Towards a Human-like Open-Domain Chatbot, Arxiv 2020.
14 April Amin Pourdabiri, Sara Rajaee
  • Emotion Dynamics in Movie Dialogues, Arxiv 2021. [slides]
  • Infusing Finetuning with Semantic Dependencies, TACL 2021.
7 April Kiamehr Rezaee, Houman Mehrafarin
  • AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts, EMNLP 2020.
  • Making Pre-trained Language Models Better Few-shot Learners, Arxiv 2020.
  • Probing What Different NLP Tasks Teach Machines about Function Word Comprehension, *SEM 2019.


Winter 2020/2021

Date Presenters Topic / Paper
17 March Hosein Mohebbi, Mohammad Ali Modarressi
  • Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference. EACL 2021. [slides]
  • How Many Data Points is a Prompt Worth?. NAACL 2021.
  • It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (Arxiv 2020).
10 March Mohsen Tabasi, Sara Rajaee
  • Interpretation of NLP models through input marginalization. EMNLP 2020.
  • Designing and Interpreting Probes with Control Tasks. EMNLP 2019.
3 March Amin Pourdabiri, Sara Rajaee
  • Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation. ACL 2020. [slides]
  • Analyzing Individual Neurons in Pre-trained Language Models. EMNLP 2020. [slides]
24 February Houman Mehrafarin, Kiamehr Rezaee
  • Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning. Rep4NLP 2020.
  • On Identifiability in Transformers. ICLR 2020.
17 February Hosein Mohebbi, Mohammad Ali Modarressi
  • Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. TACL 2021. [slides]
  • First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT. EACL 2021.
10 February National holiday --
3 February Mohsen Tabasi, Mahsa Razavi
  • Generalization through Memorization: Nearest Neighbor Language Models. ICLR 2020.
  • SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training. EMNLP 2020. [slides]
27 January ACL deadline --
20 January Amin Pourdabiri, Sara Rajaee
  • Fine-grained Emotion and Intent Learning in Movie Dialogues (Arxiv 2020). [slides]
  • Unsupervised Distillation of Syntactic Information from Contextualized Word Representations. BlackBoxNLP 2020. [slides]
13 January Houman Mehrafarin, Maryam Sadat Hashemi
  • Attention Interpretability Across NLP Tasks (Arxiv 2019).
  • When BERT Plays the Lottery, All Tickets Are Winning. EMNLP 2020. [slides]
6 January (2021) Kiamehr Rezaee, Mohammad Ali Modarressi
  • Attention is Not Only Weight: Analyzing Transformers with Vector Norms. EMNLP 2020.
  • How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.
30 December Mohsen Tabasi, Hosein Mohebbi
  • Experience Grounds Language. EMNLP 2020.
  • Intrinsic Probing through Dimension Selection. EMNLP 2020. [slides]
23 December Student project proposals --

Fall 2020

Date Presenters Topic / Paper
16 December Sara Rajaee, Mahsa Razavi
  • Topology of Word Embeddings: Singularities Reflect Polysemy. *SEM 2020. [slides]
  • Diversifying Dialogue Generation with Non-Conversational Text. ACL 2020. [slides]
9 December Houman Mehrafarin, Amin Pourdabiri
  • Assessing BERT's Syntactic Abilities. 2019.
  • Personal Information Leakage Detection in Conversations. EMNLP 2020. [slides]
2 December Maryam Sadat Hashemi, Kiamehr Rezaee
  • Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. EMNLP 2020. [slides]
  • Linguistic Profiling of a Neural Language Model. COLING 2020.
25 November Hosein Mohebbi, Mohsen Tabasi, Sara Rajaee
  • The elephant in the interpretability room. Why use attention as explanation when we have saliency methods? BlackboxNLP 2020. [slides]
  • How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking. EMNLP 2020.
  • Asking without Telling: Exploring Latent Ontologies in Contextual Representations. EMNLP 2020. [slides]
18 November Mohammad Ali Modarressi, Samin Fatehi
  • Pretrained Language Model Embryology: The Birth of ALBERT. EMNLP 2020.
  • ETC: Encoding Long and Structured Inputs in Transformers. EMNLP 2020. [slides]
11 November Amin Pourdabiri, Mahsa Razavi
  • Hierarchical Reinforcement Learning for Open-Domain Dialog. AAAI 2020. [slides]
  • Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness. EMNLP 2020. [slides]
4 November Houman Mehrafarin, Kiamehr Rezaee, Maryam Sadat Hashemi
  • Are Sixteen Heads Really Better Than One? NIPS 2019. [slides]
  • Do Explicit Alignments Robustly Improve Multilingual Encoders. EMNLP 2020.
  • Rethinking attention with performers. [slides]
28 October Hosein Mohebbi, Mohammad Ali Modarressi
  • A Tale of a Probe and a Parser. ACL 2020.
  • What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding. EMNLP 2020.
21 October Sara Rajaee, Samin Fatehi, Mahsa Razavi
  • BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance. EMNLP 2020. [slides]
  • Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. ACL 2020.
  • Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network. ACL 2018.
14 October Sara Rajaee, Maryam Sadat Hashemi
  • Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. NAACL 2019. [slides]
  • Cross-Modality Relevance for Reasoning on Language and Vision. ACL 2020. [slides]
7 October Houman Mehrafarin, Kiamehr Rezaee, Mohsen Tabasi
  • What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. TACL 2020. [slides]
  • Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference.
  • Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information. ACL 2020.
30 September Amin Pourdabiri, Maryam Sadat Hashemi, Samin Fatehi
  • Big Bird: Transformers for Longer Sequences. [slides]
  • Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. ECCV 2020. [slides]
  • Attention over Parameters for Dialogue Systems. NeurIPS 2020 ConvAI workshop. [slides]
23 September Ali Modarressi and Hosein Mohebbi
  • Quantifying Attention Flow in Transformers. ACL 2020. [slides]
  • DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering. ACL 2020. [slides]

Summer 2020

Date Presenters Topic / Paper
4 July Houman Mehrafarin, Hosein Mohebbi, Amin Pourdabiri, Ali Modarressi
  • Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
  • BERT Loses Patience: Fast and Robust Inference with Early Exit. [slides]
  • Accelerating Natural Language Understanding in Task-Oriented Dialog
  • Fine-tune BERT with Sparse Self-Attention Mechanism. [slides]
11 July Houman Mehrafarin, Sara Rajaee
  • Linguistic Knowledge and Transferability of Contextual Representations. [slides]
  • Finding Universal Grammatical Relations in Multilingual BERT. [slides]
15 July Mohsen Tabasi, Kiamehr Rezaee, Zahra Sayedi
  • How does BERT’s attention change when you fine-tune? An analysis methodology and a case study in negation scope. [slides]
  • Similarity of Neural Network Representations Revisited
  • Consistent Dialogue Generation with Self-supervised Feature Learning
22 July Hosein Mohebbi, Amin Pourdabiri, Ali Modarressi
  • BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. [slides]
  • Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access
  • FastBERT: a Self-distilling BERT with Adaptive Inference Time. [slides]
29 July Sara Rajaee, Maryam Sadat Hashemi, Samin Fatehi
  • On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines. [slides]
  • VisualBERT: A Simple and Performant Baseline for Vision and Language. [slides]
  • Parameter-Efficient Transfer Learning for NLP. [slides]
5 August Mohsen Tabasi, Kiamehr Rezaee, Zahra Sayedi
  • Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions. [slides]
  • Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
  • You Impress Me: Dialogue Generation via Mutual Persona Perception
12 August Houman Mehrafarin, Hosein Mohebbi, Mohammad Ali Modarresi
  • Probing Linguistic Systematicity, ACL 2020. [slides]
  • ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators, ICLR 2020. [slides]
  • BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension, ACL 2020. [slides]
19 August Sara Rajaee, Samin Fatehi, Mahsa Razavi
  • A Mixture of h-1 Heads is Better than h Heads, ACL 2020. [slides]
  • Longformer: The Long-Document Transformer, 2020. [slides]
  • Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems, ACL 2019.
26 August Mohsen Tabasi, Kiamehr Rezaee, Amin Pourdabiri
  • The Sensitivity of Language Models and Humans to Winograd Schema Perturbations, ACL 2020. [slides] [notebook]
  • Learning to Speak and Act in a Fantasy Text Adventure Game. EMNLP 2019. [slides]
  • Emerging Cross-lingual Structure in Pretrained Language Models. ACL 2020.
2 September Amin Pourdabiri, Maryam Sadat Hashemi, Samin Fatehi
  • Deploying Lifelong Open-Domain Dialogue Learning. [slides]
  • Reformer: The Efficient Transformer, ICLR 2020. [slides]
  • Revealing the Dark Secrets of BERT, EMNLP 2019. [slides]
9 September Houman Mehrafarin, Kiamehr Rezaee
  • Do Neural Language Models Show Prefrences for Syntactic Formalisms? ACL 2020. [slides]
  • Multilingual Alignment of Contextual Word Representations. ICLR 2020.
16 September Mohsen Tabasi, Maryam Sadat Hashemi, Sara Rajaee
  • Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020. [slides]
  • MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices. ACL 2020. [slides]
  • Knowledge Enhanced Contextual Word Representations. EMNLP 2019. [slides]

Series 1

Spring 2020

Date Presenters Topic / Paper
20 April Mohsen Tabasi
  • BERT Rediscovers the Classical NLP Pipeline. [slides]
  • Universal Adversarial Triggers for Attacking and Analyzing NLP. [slides]
27 April Hosein Mohebbi
  • oLMpics -- On what Language Model Pre-training Captures
  • Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. [slides]
4 May Houman Mehrafarin
  • What Does BERT Look At? An Analysis of BERT's Attention. [slides]
11 May Amin Pourdabiri
  • Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
18 May Kiamehr Rezaee
  • How Multilingual is Multilingual BERT?
25 May - -
1 June Ali Modarressi
  • BERT-based Lexical Substitution. [slides]
  • Contextual Embeddings: When Are They Worth It? [slides]