NLP interview questions

Explain how positional encoding is used in transformers.

Why does BERT use masked language modeling (MLM)?

Explain how Reinforcement Learning from Human Feedback (RLHF) works in the context of language models.

Compare the beam search and greedy approaches for sequence generation.

Explain how the self-attention mechanism works in transformers and what role it plays in sequence processing.

How would you apply sentiment analysis to LLM responses to rank products?

Why is vanilla BERT insufficient for sentiment analysis, and why is fine-tuning needed?

Which text tokenization algorithms are used in transformers?

What model would you train to generate text descriptions if prompting does not work?

How do tokenization and embedding generation work in transformers?

Describe your experience with LLMs: how have you used them, and what problems have you solved?

Why can't you simply replace the main LLM with the LLM judge if the judge produces better answers?

What are the main approaches and technologies in the field of speech-to-text conversion and audio processing?

What are the main approaches to configuring LLM for moderation? Are System Prompt or fine-tuning used?

Describe your implementation of an AI agent (a RAG system) in detail.

Сейчас. Ты говорил, что работал с VLM. Зачем вообще он нужен, если есть точку Generate с Transformers?

Why Word2vec, а не Berth какой-нибудь?

Можешь сказать, что из себя представляет эмбейдинг и Why он нам интересен в контексте языковой модели?

For example, a distilled BERT with a small output embedding and around 10–20 million parameters.

Then compare the cosine distance between our text and the product to determine how relevant the conversation is to the product.