Download as PDF, PPTX
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/85/Distributed-Deep-Learning-with-Docker-at-Salesforce-9-320.jpg)
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”
Data Science Team](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/85/Distributed-Deep-Learning-with-Docker-at-Salesforce-10-320.jpg)
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”
Data Science Team Systems Team](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/85/Distributed-Deep-Learning-with-Docker-at-Salesforce-11-320.jpg)
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/75/Distributed-Deep-Learning-with-Docker-at-Salesforce-9-2048.jpg)
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”
Data Science Team](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/75/Distributed-Deep-Learning-with-Docker-at-Salesforce-10-2048.jpg)
![Deep Learning at Salesforce
Discusses
cat
preprocess
[0.2, 0.71, 0.89, 0.6]
[0.85, 0.15]
inference
postprocess“Hello! My
cat is
friendly.”
Data Science Team Systems Team](https://image.slidesharecdn.com/jeffhajewski-distributeddeeplearning-200529022235/75/Distributed-Deep-Learning-with-Docker-at-Salesforce-11-2048.jpg)
Uploaded byDocker, Inc.
Distributed Deep Learning with Docker at Salesforce
The document outlines how Salesforce employs Docker for distributed deep learning model deployment, addressing challenges such as team specialization, interaction with model servers, and testing. It emphasizes the importance of preprocessing, inference, and postprocessing in the deep learning workflow and presents solutions like containerizing functions and using Docker Compose for testing. Key takeaways include the scalability of deep learning processes through Docker and its ability to simplify complex systems.
More Related Content
Similar to Distributed Deep Learning with Docker at Salesforce
![UiPath Automation Suite Installation (Hands-On) [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/automationsuitecommunitysession2-251015095633-a6d862f1-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Distributed Deep Learning with Docker at Salesforce
- 1. Distributed Deep Learning withDocker at Salesforce
- 2.
- 3. Caveats ● These myown views and opinions, not those of Salesforce ● This is how one team at Salesforce deploys deep learning models ● When I use the term Docker I am referring to the technology, not the company ● Some of these designs are simplified
- 4. ● What isdeep learning and why is it difficult? ● Deep learning at Salesforce ● Challenges ○ Designing for team specialization ○ Interacting with the model server ○ Testing ● Key takeaways About this talk
- 5. The core taskof deep learning is function approximation. Neural networks can approximate any function. Neural networks are expensive to evaluate. ● Linear regression: ~1,000 parameters ● Deep neural network: 100M - 1B parameters (100,000 - 1M x linear reg.) Deep Learning Review
- 6. How should wedesign distributed systems for deep learning? high latency tasks
- 7. We use deeplearning models to provide our customers useful information about their sales process. They send us this data as a firehose of streaming data. The faster we get this data to our customers, the more useful and actionable it is for their sales teams. Deep Learning at Salesforce
- 8. There are threesteps to this process 1. Preprocessing - cleaning and formatting the data 2. Inference - running the data through the model 3. Postprocessing - interpreting the output from the model Deep Learning at Salesforce
- 9. Deep Learning atSalesforce Discusses cat preprocess [0.2, 0.71, 0.89, 0.6] [0.85, 0.15] inference postprocess“Hello! My cat is friendly.”
- 10. Deep Learning atSalesforce Discusses cat preprocess [0.2, 0.71, 0.89, 0.6] [0.85, 0.15] inference postprocess“Hello! My cat is friendly.” Data Science Team
- 11. Deep Learning atSalesforce Discusses cat preprocess [0.2, 0.71, 0.89, 0.6] [0.85, 0.15] inference postprocess“Hello! My cat is friendly.” Data Science Team Systems Team
- 12.
- 13. Requirements 1. The datascience team shouldn’t need to know about the system. They just want to define a sequence of computation. 2. The systems engineers shouldn’t need to know anything about the computation. They just want to scale the system. Designing for team specialization
- 14. Challenges 1. Some functionstakes longer to execute than others (e.g., model inference) 2. The order of execution is important Designing for team specialization
- 15. Solution: map functionsto containers Designing for team specialization postprocess(inference(preprocess(x))) preprocess inference postprocess
- 16. What about throughput? preprocessinference postprocess 0110010 1001111 0110010 1000011 It’s a cat! 1,000 QPS 500 QPS 300 QPS 1,000 QPS Max throughput
- 17. inference inferenceinferencepreprocess What about throughput? preprocessinference postprocess 0110010 1001111 0110010 1000011 It’s a cat! 1,000 QPS 2x 500 QPS 4x 300 QPS 1,000 QPS Max throughput Docker enables us to easily scale out each individual stage
- 18. inferenceinferencepreprocess What about throughput? preprocessinference postprocess 0110010 1001111 0110010 1000011 It’s a cat! 1,000 QPS 2x 500 QPS 2x 300 QPS 1,000 QPS Max throughput Kafka gives stage-wise checkpointing
- 19.
- 20. Model servers providea way to query the model, typically via gRPC or HTTP. What is the best way to deploy and interact with these model servers? Serving deep learning models
- 21. Challenge: 1. Model serversare designed as a standalone process. 2. How should we best utilize multiple GPUs? 3. What about networking? Interacting with the model server We want to keep deployment simple!
- 22. Solution: Deploy modelserver images as part of a “pod” or “group” with a coordinator service Interacting with the model server JVM Manager Model Server Model Server Model Server... Pod
- 23. 1. Who ownsthe model server? 2. How should we handle model versions? Where are they stored locally? 3. What are the addresses of the model servers? This solves additional challenges Data science team Docker shared volume http://localhost via Docker private networking
- 24.
- 25. Challenge: how shouldwe test these systems? 1. Deep learning models are probabilistic 2. Interservice interactions can be quite complex Testing
- 26. Solution: Docker Compose ●Makes it easy to swap out the model server with a mock service ● Deploying the entire system locally is easy ● Integrates well with Maven and Gradle Testing
- 27. We haven’t spenta lot of time discussing the details of Docker That is precisely the point!
- 28. ● Docker allowsus to simplify many aspects of our design. ● Docker stays out of the way. ● Docker provides a simple alternative to a much more complex solution. Docker simplifies our lives