ML System Design interview questions

Design a system that will collect and store features for a machine learning model. Ensure that the system can handle 10 million records per day and must provide access to features with a latency of no more than 100 ms. The team consists of 3 members.

Build a system for monitoring and alerting on data drift. The system should handle real-time data with a volume of 1 million events per day and must send alerts in case of drift detection with a delay of no more than 5 minutes. The budget is $5000 per month.

Design a system for experiment tracking using MLflow. The system should support up to 1000 concurrent experiments with a total database size of up to 500 GB and must provide access to results with a latency of no more than 10 seconds.

Build an A/B testing system capable of handling 5 million requests per day with two user groups. The system must collect and analyze user interaction data with a delay of no more than 1 hour and provide the ability to dynamically adjust tests.

Design a CI/CD architecture for deploying machine learning models. The system should support 20 deployments per day and have the ability to roll back to a previous model version within 15 minutes. The budget should not exceed $3000 per month.

Design an active learning system that will work with a dataset of 10 TB. The system must select 1% of the data for labeling each day and be able to process requests with a latency of no more than 200 ms. The team consists of 5 members.

Design a system for deploying machine learning models with support for batch processing. The system must handle 100,000 requests per day with a latency of no more than 1 hour and provide the ability to update models at least once a week.

Design a system for storing and managing machine learning models using a model registry. The system should support up to 500 models and provide access to model metadata with a latency of no more than 5 seconds.

Design a system that will monitor for concept drift in machine learning models. The system must process 2 million predictions per day and send alerts upon detecting drift with a delay of no more than 30 minutes. The budget is $4000 per month.

Design a system that will handle real-time ranking requests for 500,000 users simultaneously. The system must provide latency of no more than 200 ms and have the capability of dynamically adding new features without downtime.

Design a system that serves as a feature store for a machine learning model capable of handling data from 1 million users daily. The store should support feature versioning and ensure data access with a latency of less than 100 ms. How would you organize the architecture of the store considering these requirements?

Design a system for training a machine learning model with a dataset size of 10 TB that requires real-time labeling. The system should allow for 50 queries per second for data annotation. What would be the main components of this system and how would you manage the load?

You need to build an A/B testing infrastructure for a model that will serve 10,000 users concurrently and can handle up to 2,000 requests per minute. How would you organize data collection and analysis, and ensure proper traffic distribution between variants?

Design a monitoring system for a machine learning model that will track both data drift and concept drift. The system should process data in real-time and issue alerts with a delay of no more than 5 minutes. What key metrics would you monitor?

Design a system for experiment tracking using MLflow for a team of 5 people. The system should support up to 100 experiments per month and integrate with CI/CD processes. How would you organize the data structure and what metrics would you store?

You need to create a model registry for a team of 10 researchers. The registry should store at least 200 models and provide access to models with a latency of no more than 200 ms. How would you manage model versioning and ensure secure access?

Design a CI/CD system for ML that should handle 5 new models per week and ensure automated deployment and testing. The team consists of 8 people. How would you structure the deployment process and what tools would you use?

Design an active learning system for a model that will be used in analytics with a daily data volume of 1 TB. The system should be able to select the 20% most informative samples for annotation. How would you evaluate the effectiveness of the active learning?

Design a batch-serving system for a model capable of handling 1 million requests per day with a latency of no more than 1 second. The system should be able to handle different model versions and provide performance monitoring. What would be the key components of your system?

Design a system that integrates data from multiple sources and ensures their preprocessing for model training. The data volume reaches up to 500 GB per day. How do you plan to organize this process considering the processing time constraints and the necessary data for training?