Big Theory versus Big Data, CNS 2018 : Alona Fyshe

Summary

In this engaging presentation, Alanna Fish, a computer scientist, explores the synergy between data and theory in model development. Through examples like coin flipping and image classification, she illustrates how blending theory with data leads to more accurate results. Fish emphasizes the importance of allowing data to challenge theoretical assumptions, particularly in fields like computer vision, where deep learning algorithms have significantly outperformed traditional methods. Her key takeaway is that the best models are those that let data guide decisions, advocating for a flexible theoretical approach and the willingness to 'kill your darlings' when the data suggests a better solution.

Takeaways

• 😀 Data and theory are not opposing forces, but complementary elements that can be integrated to improve model performance.
• 😀 The Maximum A Posteriori (MAP) estimate allows us to blend theoretical beliefs (priors) with empirical data to create more accurate models.
• 😀 In Bayesian inference, prior beliefs (e.g., a coin being fair) can be adjusted as more data is observed (e.g., coin flips).
• 😀 Using the coin flip example, Alanna demonstrates how Bayesian methods help refine estimates as more evidence is gathered, making the model more data-driven over time.
• 😀 Convolutional Neural Networks (CNNs) revolutionized image classification by letting the data dictate what features are important, instead of relying on handcrafted feature extraction.
• 😀 The performance boost in computer vision came with the introduction of CNNs in the ImageNet competition, where they outperformed traditional methods by a significant margin.
• 😀 Alanna emphasizes the importance of allowing data to inform feature selection, contrasting it with older, manual approaches where features were pre-programmed.
• 😀 The transition from traditional computer vision methods to CNNs reflects a broader shift towards more flexible, data-driven approaches in machine learning.
• 😀 The concept of 'killing your darlings' in creative writing parallels the need to let go of rigid theories when they no longer serve the data or model optimization.
• 😀 Alanna advocates for theory that is flexible enough to incorporate and adapt to new data, rather than being confined by outdated assumptions.
• 😀 Advances in deep learning and computational models, such as CNNs and language models, align with how the human brain processes information, highlighting the convergence between machine learning and cognitive science.

Q & A

What is the main theme of Alanna Fish's talk?

The main theme is the integration of data and theory in data science, emphasizing how both can work together, particularly in machine learning and computer vision tasks.

What is Maximum A Posteriori (MAP) estimation?

MAP estimation is a method that blends data with prior beliefs, allowing a model to adjust its predictions based on observed evidence while respecting previous assumptions or theoretical models.

How does MAP estimation work with the coin-flipping example?

In the coin-flipping example, MAP estimation allows the blending of prior belief (that the coin is fair) with observed data (flipping a biased coin), leading to an updated probability estimate for the coin being fair.

What role does the 'prior belief' play in MAP estimation?

The prior belief represents what is known or assumed before observing any data. It is combined with the observed data to form a more informed estimate, as seen in the coin-flipping example where the prior belief was that the coin is fair.

How does increasing the amount of data affect the MAP estimate?

As more data is collected, the MAP estimate becomes more focused on the data, reducing the influence of the prior belief and leading to a more accurate estimate based on the observed evidence.

What challenge does image classification pose in computer vision?

Image classification is challenging due to the large variety of objects, different views, and granularity of classes (e.g., distinguishing between very similar types of trees or cats), making it difficult for algorithms to accurately classify images.

How did traditional computer vision approaches handle image classification?

Traditional approaches used handcrafted pipelines, where key features were pre-programmed by researchers. These features were manually selected and used to classify images based on their patterns.

What is the significance of convolutional neural networks (CNNs) in image classification?

CNNs revolutionized image classification by allowing the data to define its own features. Instead of relying on human-defined key features, CNNs extract relevant features from images, leading to much better performance and accuracy.

How did CNNs outperform traditional computer vision algorithms in 2012?

In 2012, CNNs outperformed traditional algorithms by nearly 10%, as they allowed the model to learn the relevant features from the data itself, rather than relying on pre-programmed assumptions, leading to improved accuracy in tasks like image classification.

What is the connection between convolutional neural networks and human brain activity?

The hidden representations generated by CNNs during image processing have been found to have a relationship with areas of the human brain involved in visual processing, suggesting that CNNs may be simulating or drawing inspiration from human visual processing systems.

What does Alanna Fish mean by 'kill your darlings' in the context of machine learning?

'Kill your darlings' is a metaphor suggesting that researchers should be willing to discard their preconceived theories or assumptions when they are no longer helpful, allowing the data to guide the development of more accurate and flexible models.

Why is the dichotomy between theory and data considered false?

The dichotomy is false because both theory and data are necessary for building robust models. Instead of seeing them as opposing forces, they should complement each other, with flexible theories that can adapt based on the insights provided by data.