Interview Practice Questions

Compare and contrast the tokenization methods BPE and WordPiece.

Why are contextual embeddings used in ELMo and how do they differ from static embeddings?

What is RLHF and how is it applicable in modern NLP models?

What are the main principles of how the T5 model works and what makes it unique?

Explain what BERT is and what it is used for in natural language processing tasks.

What is word2vec and how does the Skip-gram method work?

Compare the tokenization methods BPE, WordPiece, and SentencePiece. What are their main differences?

Why is positional encoding important in transformers?

Explain how the self-attention mechanism works in transformers and how it differs from cross-attention.

What are the advantages and disadvantages of using GPT compared to BERT?

When should the RAG (Retrieval-Augmented Generation) approach be used in text processing tasks?

How does Reinforcement Learning from Human Feedback (RLHF) work and where can it be useful?

What is LoRA and how does it help in training large language models?

What is TF-IDF and how does it work?

Explain how the context-dependent word vector representation method Word2Vec works.

What is BERT and how do its Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) mechanisms affect its performance?

Compare the tokenization approaches of BPE and WordPiece. What are their main differences?

How does the attention mechanism work in transformers, and what is the difference between self-attention and cross-attention?

When and why should one use the BM25 method over TF-IDF in search tasks?

What is FastText and what advantages does it offer compared to other word embedding models?