A Day in the Life of a Machine Learning Engineer (at a *small* startup)

Summary

In this video, Daniel Burke, co-founder and machine learning engineer at Nutrify, gives a quick tour of their office, sharing insights into their work developing an iOS app focused on recognizing whole foods. He introduces team members, discusses the machine learning models they use, and explains how they aim to upgrade from image classification to object detection. The team blends work with exercise and healthy food preparation, and Daniel highlights their technical workflow, model training process, and data labeling efforts. The video offers a behind-the-scenes look at life in a startup.

Takeaways

• 🤖 Daniel Burke introduces himself as a machine learning engineer and co-founder of Nutrify, an iOS app focused on Whole Foods.
• 📱 Josh, the head iOS engineer, is responsible for all interactive elements in the Nutrify app.
• 🏢 The office setup includes whiteboards, data labeling stations, and model training systems in a home-converted workspace.
• 🥑 Nutrify focuses on image classification of Whole Foods, with plans to upgrade to object detection.
• 📚 Daniel highlights 'Designing Machine Learning Systems' as a key resource for machine learning engineers.
• 💪 Daniel practices regular exercise throughout the workday, incorporating bodyweight movements like pull-ups and squats.
• 🍽 Lunch at Nutrify HQ often features homemade Whole Foods meals, such as roasted sweet potatoes and ground beef.
• 📊 The team uses Weights & Biases for tracking model training metrics and logs, with ongoing improvements to their food image dataset.
• 💻 The setup includes a Titan RTX and RTX 4090 GPUs for training machine learning models and running inference tasks.
• ⚙️ The future goal is to build a fully automatic data labeling engine for identifying multiple foods in images, mimicking Tesla’s data approach for self-driving cars.

Q & A

What is Nutrify HQ and who introduced the video?

Nutrify HQ is the headquarters of Nutrify, an iOS app focused on learning about whole foods. The video was introduced by Daniel Burke, a machine learning engineer and co-founder of Nutrify.

What is the main function of the Nutrify app?

The main function of the Nutrify app is to help users learn about whole foods. Currently, it uses an image classification model to identify food, with plans to upgrade it to object detection.

What role does Josh play in Nutrify?

Josh is the head iOS engineer at Nutrify. He is responsible for anything users can interact with on the iOS app, including text detection and image matching features.

What is the machine learning process at Nutrify focused on?

The machine learning process at Nutrify involves labeling food images, training models, and refining the data pipeline to go from single food per image to multi-food object detection. They also perform manual data labeling to ensure high quality.

How does Daniel Burke describe his daily routine?

Daniel starts his day early, usually between 7:00 and 8:00 a.m., often preparing breakfast for himself and others. After setting priorities for the day, he reads articles on machine learning and spends time labeling data, training models, and refining systems. He integrates regular exercise into his workday with short bodyweight workouts.

What challenges did Daniel encounter during model training?

During one model training session, Daniel encountered a dimensionality issue related to test accuracy calculations. He speculated that it might be due to unnecessary use of 'torch.argmax'.

What does the term 'Geni Jutsu' mean and how is it relevant to Nutrify?

'Geni Jutsu' is a term from Toyota's production system that means 'real things, real places.' Daniel applies this concept to Nutrify, emphasizing the importance of testing the app in real-life scenarios to ensure it works as intended.

What hardware is used to train the Nutrify models?

Nutrify uses two computers for model training. The primary machine is a Titan RTX, which handles most training tasks. They also have a newer PC with an Nvidia RTX 4090, primarily used for running inference tasks over large datasets.

What are some of the meals prepared at Nutrify HQ, and how is food related to the app's mission?

At Nutrify HQ, meals are prepared using whole foods, such as ground beef, roasted sweet potatoes, guacamole, and broccolini. Since Nutrify is a food app, the team emphasizes preparing and consuming simple, nutritious meals that align with their app's mission.

What is Daniel's perspective on the importance of data in machine learning?

Daniel emphasizes that much of the work in machine learning involves refining the dataset, not just building models. His goal is to maintain a continuous cycle of data labeling, training, and upgrading the dataset, comparing it to the data engine used by Tesla.