Questions
You're at a strategy meeting with the stakeholders. They want to make sure you have the data required to answer the questions they're most interested in.
Be prepared to answer the following questions:
Transfer learning for from scratch
Karl, Head of AI
Obviously you'll be using a convolutional neural network to build your model, but will you be using an existing architecture as a starting point, or do you think it'll be better to design your own?
Based on your initial analysis of the data, your team feels:
- VGG-16 would be a good architecture to use on this data.
- ResNet would be a good architecture to use on this data.
- Inception would be a good architecture to use on this data.
- For best results, we should design a custom architecture.
Preprocessing
Johnny, Data Science Intern
The training and test images have three color channels, (Red, Green, and Blue), with pixel values for each channel ranging from 0 to 255.
Do you think we need to do any preprocessing before using the data to train the model?
Data Augmentation
Emma, CEO of GehirnWagen
I'm concerned that the model will only be able to recognize signs that look exactly like the ones we have images for.
I understand from Johnny that data augmentation can help with this problem. What strategy would you suggest?
Model Evaluation
Johnny, Data Science Intern
This seems like one of those cases where straight accuracy might not be the best metric for model evaluation, but what do you think?
Confidence
Karl, Head of AI
We'd like you to actually ride in the car for a test drive, based on automation built on your sign recognition model. What performance would you require before you’d ride in the car?
Based on your initial analysis of the data, your team feels:
- Only an F1 score of 1.00 on all signs would be appropriate.
- A recall score of 1.00 for all stop signs.
- An accuracy of 95% would be good enough.
- An accuracy of 85% or above for all warning/information signs, and 98% or above for all other prohibition and traffic signs.