What is machine learning?

It does whatever you train it to do

time_travel

You can use ML for everything

resnet_mnist

So what's deep about deep learning?

scaling

How big are these deep networks?

Deep Learning

The old days

Not too long ago this was totally true

img

People used to be tougher

img

There was a time for Caffee...

img

There was a time for Tensorflow...

img

Increasing Performance

The bitter lesson is true (also, not bitter imo)

img

Also, try on transfer learning:

img

If you overfit, add regularization

img

Try different hyper-parameters. Sometimes you hit the jackpot

img

Sometimes you don't hit the jackpot

img

If nothing else works

img

Debugging Models

Finding bugs in DL is hard

img

You don't know what went wrong, the answer is just bad

img

DL Research

We still don't really understand many elements of deep learning

img

For example, we know Batchnorm is helpful in many situations, but the explanation for why keeps changing

img

If you're in RL...

img

Let's make things simpler to understand (unlike how f(x) = max(x, 0) became "rectified linear unit")

img

Research into Production

Bringing research into production

img

More on "cutting-edge" research

img

Implementing your own version is also tricky

img

When you consider not just the model, but the whole ML pipeline, things go wrong:

The underlying data

Totally underappreciated: curating your own dataset.

img

Yepp, someone labelled those datasets

img

Infrastructure

Sklearn still has its uses

img

Colab is actually quite awesome

img

Sofwater Engineering

I often go back and forth

img

I can relate to both

img

Some things never change

img

True, not just in deep learning

img

The difference often matters

img

Things look tidy until you look under the hood

img

Well, I do like my own implementation of everything...

img

Data Science and Statistics

img

What is a "data scientist"?

img

"Rigorous statistics"

img

Precision vs recall

img

Okay, this is pure math, but still good

img

AI in the world

Since we need a broader impact section.

How to impress people with AI

img

Lots of appetite for AI startups: