AI/ML Program Playbook (McCombs)

Task and Activities Hide Comments and Delete Rows as needed Status Start Date End Date Duration Predece Respo Account Support Consult Informed
ssors nsible able ing ed
1 Project Charter In Progress 09/11/23 10/12/23 24d
2 Identify Project Steering Committee Required for all Alignment Gates In Progress 09/11/23 10/12/23 24d
3 Identify Project Resources Across all impacted functions Not Started
4 Establish Project Timeline Be aggressive but realistic Complete
5 Craft Project Mission Statement Why this project and what do you want to achieve? Delayed
6 Define Project Goals and Objectives Short and long terms goals for this project At Risk
7 Develop and approve RASCI Chart See columns to far right Blocked
8 Develop and approve Communication Plan Every stakeholder is your customer- treat them as such Select One
9 Develop and approve "Definition of Done" Define project success in advance Select One
10 Develop and approve "Quit Criteria" Know in advance when to fold your cards Select One
11 Summary Presentation Prepared for Steering Committee Select One
12 ALIGNMENT GATE Steering Committee Approval Required Select One
13 Project Discovery Not Started
14 Gather Requirements Look under every stone Select One
15 Identify and Develop Use Case(s) See 8/13 Industry Session - Slides 21-24 Select One
16 Perception Use Cases(s) Collect Data - Detect Signals Select One
17 Inference Use Cases(s) Make Sense of the World Select One
18 Decision Use Cases(s) Decide on a Course of Action Select One
19 Action Use Case(s) Act in the Real World Select One
20 Itemize Data Sources by Use Case Identify total data universe Select One
21 Quantitative Data Any information that can be quantified, counted or measured, and given a numerical value. Select One
22 Discrete The data has a finite value and can be counted (for ex. # of users) Select One
23 Continuous The data has an infinite number of possible values that can be measured (for ex. avg monthly temp) Select One
24 Qualitative Data Any information that is descriptive in nature, expressed in terms of language rather than numerical values. Select One
25 Nominal The data can only be categorized (for ex. male/female) Select One
26 Ordinal The data can be categorized and ranked (for ex. happy, neutral, unhappy) Select One
27 Scope Analysis What data do we need and how much? Select One
28 Quality Assurance How good is the data and is it useful to us? Select One
29 Identify Math Models by Use Case Determine which math models are applicable to the use case Select One
30 Natural Language Processing Models that process and understand human language Select One
31 Knowledge Representation Models and structures to represent information and knowledge Select One
32 Automated Reasoning Models that deduce an outcome based on a mathematical truth and proof. Select One
33 Machine Learning Models that take data, find patterns, and then summarizes them with mathematical precision. Select One
34 Supervised Learning Mathematical model using data that contains both the inputs and the desired outputs. Select One
35 Neural Net Deep learning across multiple data nodes, adaptive system, learns from mistakes Select One
36 Decision Trees (Bagging/Boosting) Models that make predictions based on how previous questions or inputs were answered. Select One
37 Unsupervised Learning Mathematical model using data that contains ONLY inputs and no desired outputs. Select One
38 Clustering Output results in the grouping of data Select One
39 Dimensional Reduction Output results in the reduction or compression of data (tighter/cleaner data set) Select One
40 Associative Models Output is learned rules: If X, then Y. If Not X, then Z. Select One
41 Semi-Supervised Learning Models with a large amount of unlabeled data and small amount of labeled data (extrapolate) Select One
42 Reinforcement Learning Models that incorporate feedback in the learning technique (perform/see result/learn) Select One
43 Representation Learning Models that extract meaningful patterns from raw data and create insightful representations. Select One
44 Computer Vision Models that interpret, analyze, and understand the visual world Select One
45 Robotics Models that instruct robots to perform tasks without human intervention. Select One
46 Statistical Models Models that generate sample data and make predictions about the real world Select One
47 Itemize Technical Tools by Use Case Identify all required technology Select One
48 Hardware Requirements What hardware is required? (on-prem vs cloud) Select One
49 Software Requirements What software is required? (local vs SaaS vs custom) Select One
50 Microservices Requirements What APIs or other microservices will be required? Select One
51 Data Storage Requirements How will we manage data storage and security? Select One
52 Model Building Requirements Tools specific to building the model Select One
53 Model Training Requirements Tools specific to training and maintaining the model Select One
54 Identify Required Resources Identify everyone who needs to be involved Select One
55 Business Roles Business or Functional Analysts and Domain Experts Select One
56 Required Skills Data Visualization, Business Intelligence, SQL, Domain Expertise Select One
57 Employees Available internal resources Select One
58 Contractors Consultants, and Subject Matter Experts (SMEs) Select One
59 Data Roles Data Analysts, Data Engineers, and Data Architects Select One
60 Required Skills Ruby, Python, JavaScript, C/C++, SQL, noSQL, Perl, Hive, Pig, Hadoop, Spark Select One
61 Employees Available internal resources Select One

Page 1 of 4
 Task and Activities Hide Comments and Delete Rows as needed Status Start Date End Date Duration Predece Respo Account Support Consult Informed
ssors nsible able ing ed
62 Contractors Developers, Consultants, and Data Scientists Select One
63 Model Building Roles Engineers, Statisticians, and Data Scientists Select One
64 Required Skills Ruby, Python, JavaScript, C/C++, SQL, noSQL, Perl, Hive, Pig, Hadoop, Spark Select One
65 Employees Available internal resources Select One
66 Contractors Developers, Consultants, and Data Scientists Select One
67 Model Management Roles Deployment Engineers, AI/ML Ops Engineers, Tech Leads Select One
68 Required Skills Ruby, Python, JavaScript, C/C++, SQL, noSQL, Perl, Hive, Pig, Hadoop, Spark Select One
69 Employees Available internal resources Select One
70 Contractors Developers, Consultants, and Data Scientists Select One
71 Initial ROI Analysis by Use Case Pre-deep-dive business case(s) Select One
72 Investments What is it going to cost? Select One
73 Technology Costs Software Licenses, SaaS, Hardware, Tools, Storage, DevOps Select One
74 Human Resources Costs Payroll, Contractors, Training, Ongoing Maintenance & Support Select One
75 Returns What are we going to get for the effort? Select One
76 Productivity Gains Itemized by function Select One
77 Time Savings Itemized by role Select One
78 Cost Savings Itemized by P&L category Select One
79 Summary Presentation Prepared for Steering Committee Select One
80 ALIGNMENT GATE Steering Committee Approval Required Select One
81 Feasibility Analysis Not Started
82 Use Case 1 Business Fundamentals Select One
83 Technical Feasibility Analyze the technology and expertise needed for the project. Select One
84 Technical Tools Do we have the right technology to do this properly? Select One
85 Technical Expertise Do we have the right/enough expertise to do this right? Select One
86 Operational Feasibility Can we realistically implement and support this model/solution? Select One
87 Resource Analysis Which and how many resources do we have to support this project? Select One
88 Skills Analysis What are the skills needed to support this project and do we have them? Select One
89 Competency Analysis What are our collective resources and skills capable of achieving? Select One
90 Market Feasibility Is there internal OR external demand for this model/solution? Select One
91 Market Analysis What other options are available? Select One
92 Competitive Analysis How does this solution stack up? Select One
93 Revenue/Benefit Projections Whats the expected benefit to the (internal or external) market? Select One
94 Financial Feasibility Identify the financial commitment required and the projected return. Select One
95 Cost/Benefit Analysis What is the expected benefit to the business? Select One
96 Expected ROI ROI Analysis Select One
97 Economic Value Added (EVA) FP&A assessment Select One
98 Summary and Recommendation Feasibility Study + Recommendation Select One
99 Pro Forma P&L Forecasted Project P&L Select One
100 Use Case 2 Business Fundamentals Select One
118 Use Case 3 Business Fundamentals Select One
136 Summary Presentation Prepared for Steering Committee Select One
137 ALIGNMENT GATE (Go vs No Go) Steering Committee Approval Required Select One
138 Deploy ML Operations Not Started
139 Provision Required Hardware On-prem and/or cloud Select One
140 Implement Required Software Local and/or SaaS Select One
141 Integrate Required Microservices APIs and webhooks Select One
142 Structure and Consolidate Data Sources Prepare data for EDA Select One
143 Systems Integration Testing (SIT) Make sure the integrations work Select One
144 User Acceptance Testing (UAT) Make sure the data scientists can use the tools Select One
145 Declare IAM and Security Protocols Identity and Access Management, Information Security Select One
146 Summary Presentation Prepared for Steering Committee Select One
147 ALIGNMENT GATE Steering Committee Approval Required Select One
148 Build Data Model(s) Not Started
149 Data Ingestion and Preprocessing Secure, consolidate, and import all data sources Select One
150 Data Transformation Organize and clean up the data Select One
151 Missing Values Handle missing values found in table cells Select One
152 Numeric Outliers Detect and manage the outliers for each data source (Boxplot IQR) Select One
153 Row Filter Filter rows based on selected criteria Select One
154 Duplicate Row Filter Remove duplicate rows Select One
155 Color Manager Assign color codes to Numeric and/or Nominal (Categorical) values Select One
156 Exploratory Data Analysis (EDA) Analyzing Numeric and Nominal (Categorical) variables Select One
157 Univariate Analysis Used when one variable is analyzed at a time Select One

Page 2 of 4
 Task and Activities Hide Comments and Delete Rows as needed Status Start Date End Date Duration Predece Respo Account Support Consult Informed
ssors nsible able ing ed
158 Data Explorer Analyzes data, observe distributions, and provides summary statistics Select One
159 Histogram Tells us how evenly distributed or normally/uniformly/right/left skewed the data is. Select One
160 Box Plot Provides a 5 point summary: min, lower quartile, median, upper quartile, max (+ outliers) Select One
161 Bar Chart Comparative understanding of the frequency count for each Nominal (Categorical) variable Select One
162 Bivariate Analysis Used to analyze the relationship between two variables Select One
163 Conditional Box Plot Used to observe the distribution of numeric variables with respect to multiple categories of nominal variables. Select One
164 Bar Chart Compare the 'average' or 'sum' of the numeric variables corresponding to the categories of a nominal variable. Select One
165 GroupBy Observe different statistical measures of numeric variables corresponding to each category of a nominal variable. Select One
166 CrossTab (local) Compare the percentage/frequency distribution across the categories of two Nominal variables. Select One
167 Scatter Plot Observe the relationship of one numeric variable to another numeric variable Select One
168 Linear Correlation Draw insights into the association between two variables (Positive, Negative, None) Select One
169 Covariance Analysis Measure the joint variability of two random variables from their respective means. Select One
170 Model Building Determine the appropriate Machine Learning technique Select One
171 Supervised Learning Mathematical model using data that contains both the inputs and the desired outputs. Select One
172 Linear Regression Desired output is a continuous number Select One
173 Logistic Regression Desired output is a category Select One
174 Unsupervised Learning Mathematical model using data that contains ONLY inputs and no desired outputs. Select One
175 Clustering Output results in the grouping of data Select One
176 Dimension Reduction/Compression Output results in the reduction or compression of data (tighter/cleaner data set) Select One
177 Association Rules Output is learned rules: If X, then Y. If Not X, then Z. Select One
178 Semi-Supervised Learning Models with a large amount of unlabeled data and small amount of labeled data (extrapolate) Select One
179 Reinforcement Learning Models that incorporate feedback in the learning technique (perform/see result/learn) Select One
180 Representation Learning Models that extract meaningful patterns from raw data and create insightful representations. Select One
181 Model Performance Testing Measure and calibrate the effectiveness of the model Select One
182 Numeric Scoring (Linear Regression) Analyze scoring results Select One
183 Training Data Measure performance against training data (70% ) - Should be perfect Select One
184 Test Data Measure performance against test data (30%) - Should be very close to perfect Select One
185 Performance Metrics (Linear Regression) Measure the accuracy of your Linear Regression model Select One
186 R-squared Measurement Between .5 and .8 is considered acceptable in social sciences Select One
187 Adjusted R-squared Measurement Better than .7 is considered good (.95 if medical or scientific) Select One
188 Confusion Matrix (Logistical Regression) Table that is used to measure and define the performance of a classification algorithm. Select One
189 True Positive (TP) Analysis The values predicted to be True that are actually True. Select One
190 True Negative (TN) Analysis The values predicted to be False that are actually False. Select One
191 False Positive (FP) Analysis The values predicted to be True that are actually False. Select One
192 False Negative (FN) Analysis The values predicted to be False that are actually True. Select One
193 Performance Metrics (Logistical Regression) Measure the accuracy of your Logistical Regression model Select One
194 Precision Measure of Correctness (formula = TP / TP+FP) Select One
195 Recall (Sensitivity) Measure of Completeness (formula = TP / TP+FN) Select One
196 Specificity Measure of False Positive Rate (formula = TP / TP+FN) Select One
197 F1 Score Combines Precision and Recall (formula = (2*Precision*Recall) / (Precision+Recall) Select One
198 Best Fit Determination Tweak model to find and strike ideal balance Select One
199 Summary Presentation Prepared for Steering Committee Select One
200 ALIGNMENT GATE Steering Committee Approval Required Select One
201 Implement Model in Production Not Started
202 One Shot Prediction Crafted per use case Select One
203 Batch Prediction Crafted per use case Select One
204 Software as a Service Requires its own Program Plan Select One
205 Program Plan Insert Plan Select One
206 Change Management Insert Plan Select One
207 Communication Management Insert Plan Select One
208 Systems Integration Testing (SIT) Insert Plan Select One
209 User Acceptance Testing (UAT) Insert Plan Select One
210 Go Live Insert Plan Select One
211 Hyper-Support Insert Plan Select One
212 Summary Presentation Prepared for Steering Committee Select One
213 ALIGNMENT GATE Steering Committee Approval Required Select One
214 Model Maintenance and Support Not Started
215 One Shot Prediction Crafted per use case Select One
216 Batch Prediction Crafted per use case Select One
217 Software as a Service Requires its own Operating Plan Select One
218 Summary Presentation Prepared for Steering Committee Select One
219 ALIGNMENT GATE Steering Committee Approval Required Select One

Page 3 of 4
 Task and Activities Hide Comments and Delete Rows as needed Status Start Date End Date Duration Predece Respo Account Support Consult Informed
ssors nsible able ing ed
220 Kitchen Sink Not Started

Page 4 of 4