Download to read offline
Uploaded byDan Sullivan, Ph.D.
With Automated ML, is Everyone an ML Engineer?
The document discusses the machine learning workflow and modern tools for building, deploying, and monitoring ML models, particularly using Google Cloud services. It emphasizes the importance of defining problems, identifying data sources, and evaluating model fairness throughout the ML process. Additionally, it highlights services like AutoML, AI Platform, and Cloud Composer that facilitate various stages of ML development while maintaining an emphasis on data quality and monitoring practices.
More Related Content
![[AI] ML Operationalization with Microsoft Azure](https://cdn.slidesharecdn.com/ss_thumbnails/wds-mlops-trainer-kyle-akepanidtaworn-v03-190924094657-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to With Automated ML, is Everyone an ML Engineer?
![[Giovanni Galloro] How to use machine learning on Google Cloud Platform](https://cdn.slidesharecdn.com/ss_thumbnails/mlcapabilitiesongcp-190115085455-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![Agentic Systems and Compliance - A brief intro [1.2]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticsystemsandcompliace-1-251018025303-958a42ec-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
With Automated ML, is Everyone an ML Engineer?
- 1. With AutomatedML, Is EveryoneanML Engineer? Dan Sullivan- DevFest Northeast 2020 – October 31,2020
- 2. Bio • Principle Engineer,PEAK6 Technologies • Author • Instructor • Udemy • Google Cloud • LinkedIn Learning • Data Science • Machine Learning • Databases & Data Modeling
- 4. Overview • Machine LearningWorkflow • Formulating an ML Problem • Building ML Models in GCP • Data Engineering • Monitoring and Evaluating Fairness
- 5. 1. Formulating anML Problem
- 6. 0. Machine LearningWorkflow • Formulate problem • Identity data sources • Prepare data • Train, evaluate, and tune model • Deploy model • Use model in production • Monitor and Evaluate Fairness https://thenounproject.com/term/workflow/2409348/
- 7. Define the Problemto Be Solved • Informal description • What is the value of solving the problem? • How can the problem be solved • Regression • Classification https://static.thenounproject.com/png/230138-200.png
- 8. IdentifyData Sources • Amountof data available • Quality • Rate of generation • Requirements to access • Limitations on the use of data https://commons.wikimedia.org/wiki/File:Data_types_-_en.svg
- 9.
- 10. Services for BuildingModels •Cloud AutoML • AI Platform Training • Kubeflow • Dataproc with Spark ML • BigQuery ML
- 11. CloudAutoML • Designed formodel builders with limited ML experience • GUI for training, evaluating, and tuning • Services for sight, language, and structured data • AutoML Tables uses structured data to build regression and classification models
- 12. AI Platform Training •Trains and runs models built in • Tensorflow • Scikit Learn • XGBoost • Hosted frameworks but can run custom containers • Service provisions compute resources needed for a job and then executes the job
- 13. Kubeflow • Kubeflow ismachine learning toolkit for Kubernetes • Packages models like applications • Compose, deploy, and manage ML workflows
- 14. Dataproc and SparkML • Dataproc is managed Spark and Hadoop service • Spark ML is machine learning library • ML Algorithms • Feature engineering • Pipelines • Persistence • Utilities
- 15. BigQueryML • BigQuery isserverless analytical database • BigQuery ML brings machine learning functions to SQL • Key advantages are: • Ability to train and run models in BigQuery • Use SQL, not Python or ML frameworks
- 16.
- 17. CloudComposer • Managed ApacheAirflow Service • Executes workflows defined in directed acyclic graphs (DAGs) • Accessed through console or command line (gcloud composer environments)
- 18. Apache AirflowDAG • Workflowis a collection of tasks with dependencies. • DAGs stored in Cloud Storage • Supports custom plugins for operators, hooks, and interfaces • Python dependencies (packages)
- 19. DAGs are PythonPrograms Source: https://cloud.google.com/composer/docs/how-to/using/writing-dags
- 20. CloudComposer Environments • Deployedin environments, which are collections of GCP service components based on Kubernetes Engine • Uses a combination of tenant and customer project resources
- 21.
- 22. CloudData Fusion • Managedservice based on open source CDAP data analytics platform • Code-free ETL/ELT development tool • Over 160 connectors and transformations • Drag-and-drop ETL/ELT construction
- 23. ExecutionEnvironment • Cloud DataFusion deployed as an instance • Two editions • Basic – visual designer, transformations, SDK, etc. • Enterprise – Basic plus streaming pipelines, integration metadata repository, high availability, triggers, schedules, etc.
- 24.
- 25. 4. Monitoringand EvaluatingFairness
- 26. Objective • Understand performanceof model • Metrics to monitor: • Traffic patterns • Error rates • Latency • Resource utilization • Configure alerts in Cloud Monitoring
- 27. MonitoringAI Platforms • AIPlatform exports metrics to Cloud Monitoring • Metrics • Error count • Latencies • Accelerator utilization • Memory utilization • CPU utilization • Network • Prediction counts
- 28. MonitoringML ModelBest Practices •Monitor for data skew • Watch for changes in dependencies • Models are refreshed as needed • Assess model prediction quality • Test for unfairness
- 29. Fairness • Anti-classification • Protectedattributes not used in model • Example: gender • Classification parity • Measures of predictive performance are equal across groups • Calibration • Outcomes are independent of protected attributes
- 30. Fairness Resources • Google’sMachine Learning Fairness • https://developers.google.com/machine- learning/fairness-overview • AI Fairness 360 https://github.com/IBM/AIF360 • FairML https://github.com/adebayoj/fairml • What-If Tool https://pair-code.github.io/what-if-tool/
- 31. QuickSummary • Machine learningworkflows are multi-step • Automated ML addresses some, but not all steps • Lots of data engineering and monitoring still required