Interview Practice Questions

How do text generation methods like temperature sampling and top-k sampling work? When should each be used?

What is the role of positional encoding in transformers, and what are its variants?

Explain how ELMo works and how it differs from other word embedding models.

What is TF-IDF and how is it used in text processing?

What is NER and what is it used for?

Compare BM25 and TF-IDF. In which cases is it better to use each of them?

Explain how positional encoding is used in transformers.

Why does BERT use masked language modeling (MLM)?

Explain how Reinforcement Learning from Human Feedback (RLHF) works in the context of language models.

Compare the beam search and greedy approaches for sequence generation.

Explain how the self-attention mechanism works in transformers and what role it plays in sequence processing.

How would you apply sentiment analysis to LLM responses to rank products?

Why is vanilla BERT insufficient for sentiment analysis, and why is fine-tuning needed?

Which text tokenization algorithms are used in transformers?

What model would you train to generate text descriptions if prompting does not work?

How do tokenization and embedding generation work in transformers?

Describe your experience with LLMs: how have you used them, and what problems have you solved?

Why can't you simply replace the main LLM with the LLM judge if the judge produces better answers?

What are the main approaches and technologies in the field of speech-to-text conversion and audio processing?

What are the main approaches to configuring LLM for moderation? Are System Prompt or fine-tuning used?