NLP Reading Group

Tehran Institute for Advanced Studies (TeIAS)

Moderator: Mohammad Taher Pilehvar

Wednesdays, 11am-12pm (online on Teams)

Contact the moderator if you are interested in joining the group.

Winter 2023

Date	Presenters	Topic / Paper
1 February	Majid Zarharan
25 January	Mohammad AkbariTajari Mehri
18 January	Fereidoon Mehri	Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection, ACL 2020.
11 January	Sadegh Jafari	Improving Neural Language Generation with Spectrum Control, ICLR 2020.
4 January	Mohammad AkbarTajari	Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation, ICLR 2022.
28 December	Fereidoon Mehri	Discovering Latent Knowledge in Language Models Without Supervision, ICLR 2023 submission.

Fall 2022

Date	Presenters	Topic / Paper
21 December	Mohammad AkbarTajari	Incorporating Residual and Normalization Layers into Analysis of Masked Language Models, EMNLP 2021.
7 December	Mohammad AkbarTajari	Attention is Not Only a Weight: Analyzing Transformers with Vector Norms, EMNLP 2020.
30 November	Fereidoon Mehri	Chain of Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022.
23 November	Sara Rezaeimanesh	Is Sparse Attention More Interpretable?, ACL 2021.
16 November	Sadegh Jafari	Temporal Analysis of Language through Neural Language Models, ACL 2014.
9 November	Fereidoon Mehri	PEER: A Collaborative Language Model, ICLR 2023 submission.
2 November	Mahdi Zakizadeh, Sara Rezaeimanesh	DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference, ACL 2020. Metaphor Generation with Conceptual Mapping, ACL 2021.
19 October	Zahra Delbari	Understanding Dataset Difficulty with V-Usable Information, ICML 2022.

Summer 2022

Date	Presenters	Topic / Paper
31 August	Mahdi Zakizadeh	Exploring Gender Bias in Retrieval Models, Arxiv 2022.
24 August	Zahra Delbari	DACT-BERT: Differentiable Adaptive Computation Time for an Efficient BERT Inference, NLP Power! 2022. DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference, ACL 2020.
3 August	Mahdi Zakizadeh	Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information, GeBNLP 2022.
27 July	Zahra Delbari	Compacter: Efficient Low-Rank Hypercomplex Adapter Layers, NeurIPS 2021.
13 July	Zahra Delbari	A Flexible Multi-Task Model for BERT Serving, ACL 2022.
6 July	Kaveh Eskandai	Perturbation Augmentation for Fairer NLP, Arxiv 2022.
29 June	Mehrdad Nasser, Mahdi Zakizadeh	Structured Prediction as Translation between Augmented Natural Languages, Arxiv 2021. Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias, GeBNLP 2022.

Spring 2022

Date	Presenters	Topic / Paper
1 June	Maryam Sadat Hashemi	Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Models, GeBNLP 2022.
25 May	ACL 2022
18 May	Mahdi Zakizadeh	How Gender Debiasing Affects Internal Model Representations, and Why It Matters.
11 May	Kaveh Eskandari	OPT: Open Pre-Trained Transformers.
4 MAy	--
27 April	Mahdi Zakizadeh	Towards a Unified View of Parameter-Efficient Transfer Learning.
20 April	Mehrdad Nasser, Kaveh ESkandari	Transformer Memory as a Differentiable Search Index. BRIO: Bringing Order to Abstractive Summarization.
13 April	Mohammad Hossein Khojasteh, Maryam Sadat Hashemi	Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable. Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers.
6 April	Sara Rajaee	Efficient One-Pass End-to-End Entity Linking for Questions.
6 April	Sara Rajaee	Efficient One-Pass End-to-End Entity Linking for Questions.

Winter 2021

Date	Presenters	Topic / Paper
16 March	Mahdi Zakizadeh, Mehrdad Nasser	Multitask Prompted Training Enables Zero-Shot Task Generalization. Efficient One-Pass End-to-End Entity Linking for Questions.
9 March	Mehrdad Nasser, Kaveh Eskandari	Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings. Challenges in Detoxifying Language Models.
2 March	Houman Mehrafarin, Maryam Sadat Hashemi	Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly. Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers.
23 February	Mohammad Hossein Khojasteh, Mohammad Ali Modarressi	Reinforced History Backtracking for Conversational Question Answering. What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression.
16 February	Mohammad Hossein Khojasteh, Mahdi Zakizadeh	QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. Assessing the Reliability of Word Embedding Gender Bias Measures.
9 February	Kaveh Eskandari, Sara Rajaee	Do Transformers Encode a Foundational Ontology? Probing Abstract Classes in Natural Language. Information-Theoretic Measures of Dataset Difficulty.
2 February	Zahra Dehghani, Houman Mehrafarin	Don't Search for a Search Method—Simple Heuristics Suffice for Adversarial Text Attacks. Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks.
26 January	Mehrdad Nasser, Kaveh Eskandari	REALM: Retrieval-Augmented Language Model Pre-Training. Simple, Interpretable and Stable Method for Detecting Words with Usage Change Across Corpora.
19 January	Maryam Sadat Hashemi, Amin Pourdabiri	Good Prompt Is Worth Millions of Parameters? Low-resource Prompt-based Learning for Vision-Language Models. MEDCOD: A Medically-Accurate, Emotive, Diverse, and COntrollable Dialog System.
12 January	Mehrdad Nasser, Sara Rajaee	Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases. Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models.
5 January	Zahra Dehghani, Mohammad Hossein Khojasteh	Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! Neural Unification for Logic Reasoning over Natural Language.
29 December	Houman Mehrafarin, Maryam Sadat Hashemi	Introducing Orthogonal Constraint in Structural Probes. SimVLM Simple Visual Language Model Pretraining with Weak Supervision
22 December	Rabeeh Karimi Mahabadi	PERFECT: Prompt-free and Efficient Language Model Fine-Tuning, Arxiv 2021.
22 December	Sara Rajaee, Mohammad Ali Modarressi	Transformer Feed-Forward Layers Are Key-Value Memories, EMNLP 2021. Learning from others' mistakes: Avoiding dataset biases without modeling them, ICLR 2021.

Fall 2021

Date	Presenters	Topic / Paper
15 December	Kaveh Eskandari	Words of Wisdom: Representational Harms in Learning From AI Communication, LearnTec4EDI 2021. Evaluating Debiasing Techniques  for Intersectional Biases, EMNLP 2021.
8 December	Fangyu Liu	Visually Grounded Reasoning across Languages and Cultures, EMNLP 2021.
24 November	Mohammad Hossein Khojasteh, Houman Mehrafarin	Editing Factual Knowledge in Language Models, EMNLP 2021. Factual Probing Is [MASK]: Learning vs. Learning to Recall, NAACL 2021.
17 November	Zahra Sayedi, Maryam Sadat Hashemi	Augmenting Data for Sarcasm Detection with Unlabeled Conversation Context, FigLang, ACL 2020. Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language Models, EMNLP 2021.
10 November	Zahra Dehghani	Universal Adversarial Attacks with Natural Triggers for Text Classification, NAACL 2021. [slides]
3 November	Mahdi Zakizadeh, Amin Pourdabiri	Fairness without Demographics through Adversarially Reweighted Learning, NeurIPS 2020. [slides] An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog, Findings of EMNLP 2021. [slides]
27 October	Kaveh Eskandari, Sara Rajaee	Grokking: Generalization Beyond Overfitting On Small Algorithmic Data Sets, MathAI, ICLR 2021. EBERT: Efficient BERT Inference with Dynamic Structured Pruning, Findings of ACL 2021.
20 October	Amin Pourdabiri, Mohammad Ali Modarressi	Enriching Pre-trained Language Model with Entity Information for Relation Classification, Arxiv 2019. Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression, EMNLP 2021.
13 October	Houman Mehrafarin, Maryam Sadat Hashemi	Multilingual BERT Post-Pretraining Alignment, NAACL 2021. Multimodal Few-Shot Learning with Frozen Language Models, Arxiv 2021.
6 October	Zahra Dehghani, Mahdi Zakizadeh	Style is NOT a single variable: Case Studies for Cross-Style Language Understanding, ACL 2021. Mitigating Unwanted Biases with Adversarial Learning, Arxiv 2018. [slides]
29 September	Kaveh Eskandari, Mohammad Ali Modarressi	Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search, EMNLP 2021. [slides] Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers, BlackboxNLP 2021.
22 September	Tohid Abedini, Houman Mehrafarin	Constituency Parsing with a Self-Attentive Encoder, ACL 2018. Discourse Probing of Pretrained Language Models, NAACL 2021.

Summer 2021

Date	Presenters	Topic / Paper
15 September	Sara Rajaee, Maryam Sadat Hashemi	Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications, NAACL 2021. [slides] All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality, EMNLP 2021. Finetuned Language Models Are Zero-Shot Learners, Arxiv 2021. [slides]
8 September	Hosein Mohebbi, Zahra Sayedi	Enjoy the Salience: Towards Better Transformer-based Faithful Explanations with Word Salience, EMNLP 2021. [slides] BoB: BERT Over BERT for Training Persona-based Dialogue Models, ACL 2021.
1 September	Mahdi Zakizadeh, Amin Pourdabiri	On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning, NAACL 2021. Towards Emotional Support Dialog Systems, ACL 2021. [slides]
18 August	National holiday	--
11 August	Hosein Mohebbi, Mohsen Tabasi	Probing BERT in Hyperbolic Spaces, ICLR 2021. [slides] Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning, ACL 2021.
4 August	Tohid Abedini, Mohammad Ali Modarressi	Combating Adversarial Misspellings with Robust Word Recognition, ACL 2019. [slides] On Attention Redundancy: A Comprehensive Study, NAACL 2021.
28 July	Houman Mehrafarin, Maryam Sadat Hashemi	An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models, TACL 2020. Charformer: Fast Character Transformers via Gradient-based Subword Tokenization, Arxiv 2021. [slides]
21 July	National holiday	--
14 July	Kiamehr Rezaee, Zahra Sayedi	Generationary or: “How We Went beyond Word Sense Inventories and Learned to Gloss”, EMNLP 2020. All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text, ACL 2021.
7 July	Hosein Mohebbi, Mohsen Tabasi	MPNet: Masked and Permuted Pre-training for Language Understanding , NeurIPS 2020. [slides] Comparing Test Sets with Item Response Theory, ACL 2021.
30 June	Amin Pourdabiri, Sara Rajaee	SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations, Arxiv 2021. [slides] Positional Artefacts Propagate Through Masked Language Model Embeddings, ACL 2021. [slides]
23 June	Houman Mehrafarin, Maryam Sadat Hashemi	Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work? ACL 2020. Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions, NAACL 2021. [slides]

Spring 2021

Date	Presenters	Topic / Paper
16 June	Hosein Mohebbi, Sara Rajaee	TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference, NAACL 2021. [slides] How transfer learning impacts linguistic knowledge in deep NLP models? ACL 2021. [slides]
9 June	Mohsen Tabasi, Mohammad Ali Modarressi	Entailment as Few-Shot Learner, Arxiv 2021. BERT Busters: Outlier LayerNorm Dimensions that Disrupt Transformers, Findings of ACL 2021.
2 June	Sara Rajaee, Maryam Sadat Hashemi	A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space, ACL 2021. FNet: Mixing Tokens with Fourier Transforms, Arxiv 2021. [slides]
26 May	Kiamehr Rezaee	Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer, ACL 2020.
19 May	Amin Pourdabiri	Situated and Interactive Multimodal Conversations, COLING 2020. [slides]
12 May	EMNLP deadline	--
5 May	Zahra Sayedi, Houman Mehrafarin	Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue, ICLR 2020. What Happens To BERT Embeddings During Fine Tuning, BlackboxNLP 2020.
28 April	Hosein Mohebbi, Mohammad Ali Modarressi	DirectProbe: Studying Representations without Classifiers, NAACL 2021. [slides] Telling BERT’s Full Story: from Local Attention to Global Aggregation, EACL 2021.
21 April	Mohsen Tabasi, Zahra Sayedi	Static Embeddings as Efficient Knowledge Bases? NAACL 2021. Towards a Human-like Open-Domain Chatbot, Arxiv 2020.
14 April	Amin Pourdabiri, Sara Rajaee	Emotion Dynamics in Movie Dialogues, Arxiv 2021. [slides] Infusing Finetuning with Semantic Dependencies, TACL 2021.
7 April	Kiamehr Rezaee, Houman Mehrafarin	AUTOPROMPT: Eliciting Knowledge from Language Models with Automatically Generated Prompts, EMNLP 2020. Making Pre-trained Language Models Better Few-shot Learners, Arxiv 2020. Probing What Different NLP Tasks Teach Machines about Function Word Comprehension, *SEM 2019.

Winter 2020/2021

Date	Presenters	Topic / Paper
17 March	Hosein Mohebbi, Mohammad Ali Modarressi	Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference. EACL 2021. [slides] How Many Data Points is a Prompt Worth?. NAACL 2021. It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (Arxiv 2020).
10 March	Mohsen Tabasi, Sara Rajaee	Interpretation of NLP models through input marginalization. EMNLP 2020. Designing and Interpreting Probes with Control Tasks. EMNLP 2019.
3 March	Amin Pourdabiri, Sara Rajaee	Generate, Delete and Rewrite: A Three-Stage Framework for Improving Persona Consistency of Dialogue Generation. ACL 2020. [slides] Analyzing Individual Neurons in Pre-trained Language Models. EMNLP 2020. [slides]
24 February	Houman Mehrafarin, Kiamehr Rezaee	Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning. Rep4NLP 2020. On Identifiability in Transformers. ICLR 2020.
17 February	Hosein Mohebbi, Mohammad Ali Modarressi	Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. TACL 2021. [slides] First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT. EACL 2021.
10 February	National holiday	--
3 February	Mohsen Tabasi, Mahsa Razavi	Generalization through Memorization: Nearest Neighbor Language Models. ICLR 2020. SMRT Chatbots: Improving Non-Task-Oriented Dialog with Simulated Multiple Reference Training. EMNLP 2020. [slides]
27 January	ACL deadline	--
20 January	Amin Pourdabiri, Sara Rajaee	Fine-grained Emotion and Intent Learning in Movie Dialogues (Arxiv 2020). [slides] Unsupervised Distillation of Syntactic Information from Contextualized Word Representations. BlackBoxNLP 2020. [slides]
13 January	Houman Mehrafarin, Maryam Sadat Hashemi	Attention Interpretability Across NLP Tasks (Arxiv 2019). When BERT Plays the Lottery, All Tickets Are Winning. EMNLP 2020. [slides]
6 January (2021)	Kiamehr Rezaee, Mohammad Ali Modarressi	Attention is Not Only Weight: Analyzing Transformers with Vector Norms. EMNLP 2020. How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models.
30 December	Mohsen Tabasi, Hosein Mohebbi	Experience Grounds Language. EMNLP 2020. Intrinsic Probing through Dimension Selection. EMNLP 2020. [slides]
23 December	Student project proposals	--

Fall 2020

Date	Presenters	Topic / Paper
16 December	Sara Rajaee, Mahsa Razavi	Topology of Word Embeddings: Singularities Reflect Polysemy. *SEM 2020. [slides] Diversifying Dialogue Generation with Non-Conversational Text. ACL 2020. [slides]
9 December	Houman Mehrafarin, Amin Pourdabiri	Assessing BERT's Syntactic Abilities. 2019. Personal Information Leakage Detection in Conversations. EMNLP 2020. [slides]
2 December	Maryam Sadat Hashemi, Kiamehr Rezaee	Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. EMNLP 2020. [slides] Linguistic Profiling of a Neural Language Model. COLING 2020.
25 November	Hosein Mohebbi, Mohsen Tabasi, Sara Rajaee	The elephant in the interpretability room. Why use attention as explanation when we have saliency methods? BlackboxNLP 2020. [slides] How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking. EMNLP 2020. Asking without Telling: Exploring Latent Ontologies in Contextual Representations. EMNLP 2020. [slides]
18 November	Mohammad Ali Modarressi, Samin Fatehi	Pretrained Language Model Embryology: The Birth of ALBERT. EMNLP 2020. ETC: Encoding Long and Structured Inputs in Transformers. EMNLP 2020. [slides]
11 November	Amin Pourdabiri, Mahsa Razavi	Hierarchical Reinforcement Learning for Open-Domain Dialog. AAAI 2020. [slides] Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness. EMNLP 2020. [slides]
4 November	Houman Mehrafarin, Kiamehr Rezaee, Maryam Sadat Hashemi	Are Sixteen Heads Really Better Than One? NIPS 2019. [slides] Do Explicit Alignments Robustly Improve Multilingual Encoders. EMNLP 2020. Rethinking attention with performers. [slides]
28 October	Hosein Mohebbi, Mohammad Ali Modarressi	A Tale of a Probe and a Parser. ACL 2020. What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding. EMNLP 2020.
21 October	Sara Rajaee, Samin Fatehi, Mahsa Razavi	BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover's Distance. EMNLP 2020. [slides] Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. ACL 2020. Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network. ACL 2018.
14 October	Sara Rajaee, Maryam Sadat Hashemi	Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. NAACL 2019. [slides] Cross-Modality Relevance for Reasoning on Language and Vision. ACL 2020. [slides]
7 October	Houman Mehrafarin, Kiamehr Rezaee, Mohsen Tabasi	What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. TACL 2020. [slides] Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference. Breaking Through the 80% Glass Ceiling: Raising the State of the Art in Word Sense Disambiguation by Incorporating Knowledge Graph Information. ACL 2020.
30 September	Amin Pourdabiri, Maryam Sadat Hashemi, Samin Fatehi	Big Bird: Transformers for Longer Sequences. [slides] Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. ECCV 2020. [slides] Attention over Parameters for Dialogue Systems. NeurIPS 2020 ConvAI workshop. [slides]
23 September	Ali Modarressi and Hosein Mohebbi	Quantifying Attention Flow in Transformers. ACL 2020. [slides] DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering. ACL 2020. [slides]

Summer 2020

Date	Presenters	Topic / Paper
4 July	Houman Mehrafarin, Hosein Mohebbi, Amin Pourdabiri, Ali Modarressi	Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT BERT Loses Patience: Fast and Robust Inference with Early Exit. [slides] Accelerating Natural Language Understanding in Task-Oriented Dialog Fine-tune BERT with Sparse Self-Attention Mechanism. [slides]
11 July	Houman Mehrafarin, Sara Rajaee	Linguistic Knowledge and Transferability of Contextual Representations. [slides] Finding Universal Grammatical Relations in Multilingual BERT. [slides]
15 July	Mohsen Tabasi, Kiamehr Rezaee, Zahra Sayedi	How does BERT’s attention change when you fine-tune? An analysis methodology and a case study in negation scope. [slides] Similarity of Neural Network Representations Revisited Consistent Dialogue Generation with Self-supervised Feature Learning
22 July	Hosein Mohebbi, Amin Pourdabiri, Ali Modarressi	BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. [slides] Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access FastBERT: a Self-distilling BERT with Adaptive Inference Time. [slides]
29 July	Sara Rajaee, Maryam Sadat Hashemi, Samin Fatehi	On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines. [slides] VisualBERT: A Simple and Performant Baseline for Vision and Language. [slides] Parameter-Efficient Transfer Learning for NLP. [slides]
5 August	Mohsen Tabasi, Kiamehr Rezaee, Zahra Sayedi	Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions. [slides] Beyond Accuracy: Behavioral Testing of NLP Models with CheckList You Impress Me: Dialogue Generation via Mutual Persona Perception
12 August	Houman Mehrafarin, Hosein Mohebbi, Mohammad Ali Modarresi	Probing Linguistic Systematicity, ACL 2020. [slides] ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators, ICLR 2020. [slides] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension, ACL 2020. [slides]
19 August	Sara Rajaee, Samin Fatehi, Mahsa Razavi	A Mixture of h-1 Heads is Better than h Heads, ACL 2020. [slides] Longformer: The Long-Document Transformer, 2020. [slides] Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems, ACL 2019.
26 August	Mohsen Tabasi, Kiamehr Rezaee, Amin Pourdabiri	The Sensitivity of Language Models and Humans to Winograd Schema Perturbations, ACL 2020. [slides] [notebook] Learning to Speak and Act in a Fantasy Text Adventure Game. EMNLP 2019. [slides] Emerging Cross-lingual Structure in Pretrained Language Models. ACL 2020.
2 September	Amin Pourdabiri, Maryam Sadat Hashemi, Samin Fatehi	Deploying Lifelong Open-Domain Dialogue Learning. [slides] Reformer: The Efficient Transformer, ICLR 2020. [slides] Revealing the Dark Secrets of BERT, EMNLP 2019. [slides]
9 September	Houman Mehrafarin, Kiamehr Rezaee	Do Neural Language Models Show Prefrences for Syntactic Formalisms? ACL 2020. [slides] Multilingual Alignment of Contextual Word Representations. ICLR 2020.
16 September	Mohsen Tabasi, Maryam Sadat Hashemi, Sara Rajaee	Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020. [slides] MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices. ACL 2020. [slides] Knowledge Enhanced Contextual Word Representations. EMNLP 2019. [slides]

Series 1

Spring 2020

Date	Presenters	Topic / Paper
20 April	Mohsen Tabasi	BERT Rediscovers the Classical NLP Pipeline. [slides] Universal Adversarial Triggers for Attacking and Analyzing NLP. [slides]
27 April	Hosein Mohebbi	oLMpics -- On what Language Model Pre-training Captures Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping. [slides]
4 May	Houman Mehrafarin	What Does BERT Look At? An Analysis of BERT's Attention. [slides]
11 May	Amin Pourdabiri	Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
18 May	Kiamehr Rezaee	How Multilingual is Multilingual BERT?
25 May	-	-
1 June	Ali Modarressi	BERT-based Lexical Substitution. [slides] Contextual Embeddings: When Are They Worth It? [slides]