Data Science & ML with GenAI Training

• Data Types & Variables (String, Integer, Float, Complex, Boolean, None)
• Input and Output Functions
• Working with the format() method, f-strings, & escape sequences
• Basic Arithmetic & Operators
• Type casting, type checking, & validation

• Conditional Statements (if, else, elif)
• Loops (for, while)
• Looping over tuples, strings, & dictionaries
• Special loops in Python (for/else)
• Using nested loops and flow control through conditions
• Resolving real-world problems to improve skills
• Special Statements: pass, continue, break

AI Tool:

• Google Colab - Gemini

Lists:

• Overview & fundamental operations
• Indexing, slicing, & negative indexing
• Looping through lists & conditions
• List methods like .insert(), .append(), .remove(), .sort(), etc.
• List comprehension with conditions

Tuples:

• Introduction & operations
• Indexing, slicing, & looping
• List versus Tuple
• Switching between lists and tuples
• Tuple unpacking

Sets:

• Introduction & set operations
• Adding, removing, & discarding items
• Set operations: union, intersection, and difference
• Frozenset versus set

Dictionaries:

• Introduction to dictionaries & methods like .get(), .update(), .keys(), .pop(), etc.
• Dictionary comprehension
• Nested dictionaries

AI Tool:

• Gemini or Codeium

• Defining functions through def keyword
• Parameters, Arguments, & Return Statements
• Returning multiple values
• Default & keyword arguments
• Anonymous functions (lambda)
• Nested functions & closures
• Scopes in Python: Local and Global

Text File Operations:

• Reading & writing text files
• Modes of file (r, w, a, rb, wb)
• File path handling with the os module

Working with CSV Files:

• Basics of CSV format and operations
• Reading & writing CSV files with csv.reader & csv.writer
• Using dictionaries in CSV files

Working with JSON:

• Introduction to JSON & its structure
• Reading & writing JSON data with the json module
• Parsing JSON strings

AI Tool:

• Using ChatGPT for prompt engineering

• Classes & Objects
• Class versus Object attributes
• Initializing object attributes with __init__()
• self keyword
• Inheritance: single, multiple, and multi-level
• Polymorphism & operator overloading
• Function overriding & encapsulation

AI Tools:

• Pythontutor.com

• Try-except blocks
• Catching specific exceptions
• Using else & finally
• Generating and creating custom exceptions
• Problem-solving strategies

• Lambda Functions
• Generators & Iterators
• List comprehensions
• Working with *args & **kwargs

Standard libraries: os, random, math, functools, etc.

Data manipulation with Pandas

• Working with DataFrames
• Reading & writing CSV files
• Data manipulation techniques

Data Visualization:

• Using Matplotlib, Seaborn, and Plotly

AI Tools:

• Pandas Profiling

• Designing and changing databases and tables
• CRUD operations (CREATE, SELECT, UPDATE, DELETE)
• Filtering data with the WHERE clause

AI Tools:

• DBeaver for SQL queries
• Optimizing & explaining SQL queries with ChatGPT

• Introduction to APIs and HTTP methods (GET, POST, PUT, DELETE requests)
• Sending parameters, headers, and JSON data
• Handling responses and status codes
• API authentication (API keys, tokens)
• Error handling and exceptions
• Working with sessions and pagination
• Consuming public APIs (practical examples)

• Functions as first-class objects
• Closures and nested functions
• Creating and using decorators
• Decorators with arguments
• Using functools.wraps
• Built-in decorators (@staticmethod, @classmethod, @property)
• Practical use cases (logging, timing, authorization)

• Introduction to type annotations
• Function and variable type hints
• Built-in and collection types
• typing module (Optional, Union, Any)
• Type hints in classes and methods
• Static type checking basics (mypy)
• Best practices for clean and safe code

• Installing & configuring Git
• Setting up local & remote repositories
• Making commits & branching
• Integrating local repositories to GitHub
• Cloning repositories and pushing

AI Tools:

• GitHub Copilot for Git commands

1. Web Scraping + Database + File Operations: Scrape data, store it in SQL, & export to CSV/JSON
2. Desktop Application (Data Entry System): Develop an application to manage data in JSON/CSV format
3. CLI Application with CRUD Operations: Design a CLI app with basic CRUD operations & database integration

Topics

• Data Science project lifecycle
• Problem definition
• Data collection
• Data understanding
• Data cleaning
• Exploratory Data Analysis
• Feature engineering
• Model development
• Model evaluation
• Deployment
• Monitoring and improvement
• Business understanding and communication
• CRISP-DM framework
• Agile approaches in Data Science projects
• Version control basics (Git) for Data Science projects

Practical: Create a workflow for a real-world problem such as:

• Customer churn
• House price prediction
• Sales analysis
• Student performance
• Fraud detection
• Identify the lifecycle stage for a set of real project scenarios

Topics

• Basic probability
• Events and sample spaces
• Addition and multiplication rules
• Conditional probability
• Independent and dependent events
• Bayes’ theorem: intuitive understanding
• Random variables
• Normal distribution
• Bernoulli and binomial distributions
• Poisson distribution
• z-scores
• Law of Large Numbers
• Uniform and exponential distributions
• Central tendency of probability distributions

Practical

• Simulate simple probability experiments.
• Visualize common probability distributions.
• Calculate and interpret z-scores.
• Compare simulated results with theoretical probability distributions

Topics

• Sampling methods
• Sampling bias
• Sampling distribution
• Central Limit Theorem
• Point estimation
• Confidence intervals
• Margin of error
• Statistical uncertainty
• Stratified and cluster sampling
• Bootstrapping: conceptual introduction
• Standard error

Practical

• Compare population and sample statistics.
• Calculate confidence intervals.
• Demonstrate the effect of sample size.
• Perform a simple bootstrap resampling exercise

Topics

• Null and alternative hypotheses
• Significance level
• p-value
• Type I and Type II errors
• One-sample and two-sample t-tests
• Chi-square test
• Pearson correlation
• Spearman correlation
• Correlation vs. causation
• Statistical vs. practical significance
• ANOVA: conceptual introduction
• Effect size
• Multiple comparisons: brief overview

Practical

• Conduct a t-test.
• Conduct a chi-square test.
• Calculate and interpret correlations.
• Write conclusions in plain language.
• Run and interpret a one-way ANOVA

Topics

• COUNT
• SUM
• AVG
• MIN and MAX
• GROUP BY
• HAVING
• CASE statements
• Date functions
• String functions
• Business questions using SQL
• COALESCE and ROUND
• Nested aggregations
• Indexing: conceptual introduction

Practical: Answer questions such as:

• Which products generate the most revenue?
• Which customers purchase most frequently?
• Which regions have the highest sales?
• What is the average order value?
• Identify how indexing could speed up a slow query

Topics

• INNER JOIN
• LEFT JOIN
• Self-join
• Joining multiple tables
• Subqueries
• Common Table Expressions
• Data relationship analysis
• RIGHT JOIN and FULL OUTER JOIN
• UNION and UNION ALL
• Recursive CTEs: conceptual introduction

Practical: Combine customer, order, and product information to create an analytical dataset. Additionally, use UNION to merge results from multiple queries into one multi-source report.

Topics

• SQL for data preparation
• Filtering data before analysis
• Creating analytical datasets
• SQL and Data Science workflow
• SQL with analytical tools
• Basic window functions:
  • ROW_NUMBER
  • RANK
  • Running totals
• Exporting query results
• LAG, LEAD, and NTILE window functions
• Views
• Connecting SQL to Python (SQLAlchemy, pandas.read_sql)

Practical: Create a complete SQL analysis of a business dataset. Additionally, load the exported query results into a Python notebook using pandas.read_sql for further analysis.

Topics

• Common file formats: CSV, JSON, Parquet
• Row-based vs. columnar storage: conceptual introduction
• Reading and writing files programmatically
• Working with REST APIs
• API authentication basics
• Pagination and rate limits
• Combining data from multiple sources

Practical: Write a script that ingests data from a public API and merges it with a local file-based dataset.

Topics

• Databases vs. data warehouses: conceptual introduction
• OLTP vs. OLAP: conceptual introduction
• Star schema basics
• Data orchestration and scheduling: conceptual introduction
• Data quality monitoring at scale
• Data lineage: conceptual introduction
• Cloud data storage: conceptual overview

Practical: Design a simple star schema for a sales or business dataset and outline a scheduled refresh with basic quality checks.

Topics

• Data transformation
• Encoding categorical variables
• Scaling numerical variables
• Date and time features
• String-based features
• Binning
• Log transformation
• Feature creation
• Feature selection basics
• Data leakage
• One-hot, label, and target encoding
• Min-max vs. standard scaling
• Feature interactions

Practical: Prepare a dataset for machine learning. Additionally, compare model-ready features created with different scaling and encoding methods.

Topics

• Purpose of EDA
• Univariate analysis
• Bivariate analysis
• Multivariate analysis
• Numerical analysis
• Categorical analysis
• Distribution analysis
• Group-based analysis
• Identifying trends and patterns
• Finding anomalies
• Pairplots and correlation matrices
• Automated EDA tools (pandas-profiling / ydata-profiling, Sweetviz)

Practical: Perform structured EDA using a real dataset. Additionally, generate an automated EDA report using a tool such as ydata-profiling or Sweetviz and compare it with the manual analysis.

Topics

• Principles of effective visualization
• Choosing the right chart
• Bar charts
• Line charts
• Histograms
• Box plots
• Scatter plots
• Count plots
• Heatmaps
• Relationship visualization
• Avoiding misleading charts
• Interactive visualization with Plotly
• Color theory for data visualization
• Small multiples

Practical: Create visualizations for:

• Comparison
• Distribution
• Trend
• Relationship
• Category analysis
• Correlation
• Build one interactive chart using Plotly

Topics

• Multivariate analysis
• Correlation analysis
• Time-based EDA
• Segment-based analysis
• Identifying important findings
• From chart to insight
• Data storytelling
• Communicating results to non-technical audiences
• Business recommendations
• Cohort analysis
• Funnel analysis: conceptual introduction
• Executive summary writing

Practical: Convert an exploratory analysis into a structured data story:

• Problem
• Data overview
• Important patterns
• Key findings
• Recommendations
• Build a simple cohort analysis for a sales or subscription dataset

Topics

• End-to-end EDA workflow
• Data cleaning
• Descriptive statistics
• Analytical questions
• Visualization
• Statistical interpretation
• Business recommendations
• Peer review of EDA projects

Practical: Complete an EDA project using a dataset from:

• Sales
• Banking
• Healthcare
• Education
• Housing
• E-commerce
• Customer behavior
• Review and provide feedback on a peer's EDA project

Topics

• Regression problems
• Simple linear regression
• Multiple linear regression
• Best-fit line
• Predictions
• Residuals
• Model coefficients
• Basic assumptions
• Multicollinearity
• Feature scaling for regression
• Interpreting standardized coefficients

Practical: Build a model to predict:

• House prices
• Sales
• Salary
• Delivery time
• Check multicollinearity using the Variance Inflation Factor (VIF)

Topics

• MAE
• MSE
• RMSE
• R²
• Train-test split
• Cross-validation introduction
• Polynomial regression
• Ridge regression
• Lasso regression
• Bias and variance
• Elastic Net
• Learning curves
• Adjusted R²

Practical: Compare multiple regression models and identify overfitting. Additionally, plot learning curves for at least one model to diagnose bias and variance.

Topics

• Classification problems
• Binary and multiclass classification
• Logistic function
• Probability prediction
• Decision threshold
• Confusion matrix
• Precision and recall
• Multiclass strategies (One-vs-Rest, Softmax)
• Class weighting for imbalanced data
• Odds ratio interpretation

Practical: Build a classification model for:

• Customer churn
• Loan approval
• Student performance
• Spam detection
• Apply class weighting to handle an imbalanced classification dataset

Topics

• Decision tree structure
• Splitting concept
• Gini impurity: practical understanding
• Tree depth
• Overfitting
• Random Forest
• Feature importance
• Advantages and limitations
• Bagging vs. boosting: conceptual introduction
• Gradient Boosting / XGBoost overview
• Key hyperparameters (n_estimators, max_depth)

Practical:Train and compare Decision Tree and Random Forest models. Additionally, train a Gradient Boosting model and compare its performance with the Random Forest model.

Topics

• K-Nearest Neighbors
• Distance-based learning
• Importance of feature scaling
• Selecting K
• Support Vector Machines
• Hyperplanes and margins
• Kernel concept
• Strengths and limitations
• Curse of dimensionality
• Soft margin and regularization in SVM

Practical: Train a KNN or SVM classifier and compare its performance with Logistic Regression. Additionally, visualize the decision boundaries produced by the KNN and SVM models.

Topics

• Accuracy
• Precision
• Recall
• F1-score
• Specificity
• ROC-AUC
• Confusion matrix
• Class imbalance
• Stratified splitting
• Threshold adjustment
• Precision-Recall curve
• Macro vs. weighted averaging
• Cohen's Kappa

Practical: Evaluate a classification model on an imbalanced dataset. Additionally, plot ROC and Precision-Recall curves to compare model performance.

Topics

• Missing-value imputation
• Numerical scaling
• Categorical encoding
• Feature selection
• Feature creation
• Preprocessing pipelines
• Column transformation
• Preventing data leakage
• scikit-learn Pipeline and ColumnTransformer
• Custom transformers
• Saving and loading models (joblib / pickle)

Practical: Build a complete preprocessing and modeling pipeline. Additionally, save the trained pipeline to disk using joblib and reload it to confirm it works correctly.

Topics

• Parameters vs. hyperparameters
• Cross-validation
• Grid search
• Randomized search
• Model selection
• Hyperparameter search spaces
• Error analysis
• Model improvement
• Bayesian optimization: conceptual introduction
• Nested cross-validation: brief overview
• AutoML: conceptual introduction

Practical: Tune a Random Forest, SVM, or another selected model. Additionally, compare the runtime and results of Grid Search and Randomized Search on the same model.

Topics

• Unsupervised learning
• K-Means clustering
• Centroids
• Elbow method
• Silhouette score
• PCA: conceptual introduction
• Model interpretation
• Complete ML workflow
• Hierarchical clustering: conceptual introduction
• DBSCAN: conceptual introduction
• Anomaly detection basics

Practical: Perform customer segmentation or another clustering task. Additionally, compare the K-Means results with a hierarchical clustering solution on the same dataset.

Topics

• Writing clear prompts
• Providing context
• Defining constraints
• Specifying output format
• Few-shot examples
• Iterative prompt improvement
• Asking for assumptions
• Debugging generated code
• Verifying AI output
• Chain-of-thought prompting
• Role-based prompting
• Reusable prompt templates

Practical: Create prompts for:

• Data analysis
• SQL
• Visualization
• Statistical interpretation
• Machine learning
• Documentation
• Build a prompt template for automated report generation

Topics

• Natural-language data analysis
• AI-generated analytical questions
• AI-assisted visualization
• AI-generated reports
• Human verification
• Reproducibility
• Privacy and confidential data
• Limitations of AI-generated conclusions
• Introduction to AI agents in Data Science
• AI coding assistants in Data Science workflows
• Evaluating AI-generated code for correctness
• Bias in AI-generated insights

Practical: Complete an AI-assisted data analysis workflow and verify every important result. Additionally, use an AI coding assistant to speed up one part of the workflow, then review and correct its output.

Topics

• Streamlit layout
• Widgets
• Filters
• Tables
• Charts
• User input
• Loading datasets
• Displaying model predictions
• Basic error handling
• Caching with st.cache_data
• Session state basics
• Multi-page apps

Practical: Build a dashboard or prediction interface. Additionally, add caching to the application to improve data-loading performance.