Projects

Here you can find multi-part series and in-depth articles.

MLOps: Designing a Modular Pipeline

project thumbnail

Transformations as the Backbone of a Modular MLOps Pipeline

Poor code organization leads to “pipeline spaghetti,” where data ingestion, cleaning, feature engineering, and modeling code are tangled together. This tangle often arises when code is developed in a linear fashion (for example, in one giant notebook) rather than separated into reusable modules for each pipeline stage. The result is code that is hard to test or reuse.
project thumbnail

The Integration Of MLflow In This Project

MLflow is central to this project’s experiment tracking, artifact management, and reproducible model development. It is integrated through Hydra configurations, S3 synchronization scripts, and Python modeling code that leverages MLflow’s Pythonic API.
project thumbnail

Exploring dvc.yaml The Engine of a Reproducible Pipeline

The dvc.yaml file is the central point for defining a DVC-based pipeline. It specifies each stage, along with the command, dependencies, and outputs. In this project, the entire pipeline—spanning raw data ingestion, transformations, feature engineering, and modeling—is consolidated into a single dvc.yaml.
project thumbnail

Automating DVC Pipelines with Jinja2 Templates

This article explains how Jinja2 templates are structured, how Python scripts generate the final dvc.yaml, and why this method has significantly improved the maintainability of a large MLOps pipeline.
project thumbnail

A Comprehensive Look at Modular Code in an MLOps Pipeline

Modular code refers to designing each pipeline stage—data ingestion, preprocessing, model training, evaluation, and deployment—as a distinct module with well-defined inputs, outputs, and responsibilities.
project thumbnail

A Comprehensive Look at Logging in a Modular MLOps Pipeline

This article explores how logging is integrated into the pipeline, referencing relevant Python scripts in dependencies/logging_utils, Hydra configs under configs/logging_utils, and how each stage’s logs tie back to DVC and MLflow runs.
project thumbnail

A Comprehensive Look at Hyperparameter Tuning with Hydra and Optuna in an MLOps Pipeline

This article explores hyperparameter tuning best practices within a modern MLOps pipeline that integrates Hydra, Optuna, and MLflow, alongside DVC for reproducibility.
project thumbnail

A Comprehensive Look at Feature Engineering in a Modular MLOps Pipeline

Effective feature engineering is central to building high-performing and maintainable machine learning pipelines. This project’s codebase illustrates several practices that keep transformations organized, version-controlled, and consistent across training and inference stages.

Machine Learning in Action

project thumbnail

Navigating Global Data Challenges: Insights from My Experience in File System Mastery

Unlocking the secrets of file system mastery in ML: A real-world journey through data management and scripting challenges.

Machine Learning Process

project thumbnail

Optimal Machine Learning Models for Structured vs Unstructured Data: A Comparative Analysis

Structured data is searchable, suitable for traditional ML. Unstructured data suits deep learning models.
project thumbnail

In-depth Analysis: Support Vector Machines vs XGBoost

Comprehensive literature review comparing Support Vector Machines and XGBoost, focusing on their distinctive features, strengths, and limitations in predictive modeling.

Tabular Data

project thumbnail

Video Data Mining: Real-Estate Prediction Models and Problem-Solving in Action

In the current era of data-driven decision-making, understanding the intricacies of data extraction, cleansing, validation, and enrichment is a valuable asset for businesses across all sectors.
project thumbnail

Integrating MySQL with AWS Redshift for Data Analysis

Demonstrates the integration of Python, Pandas, and MySQL in an AWS Redshift environment, guiding through the process of data import and formulation of MySQL queries for effective data retrieval.
project thumbnail

Hyperparameter Optimization in Deep Learning for Kaggle, Part 7

Kaggle Submission 2: tabular_learner deep learning estimator optimized using manual hyperparameter optimization. XGBRegressor using RandomizedSearchCV and sampling from continuous parameter distributions.
project thumbnail

Kaggle Challenge Strategy: RandomForestRegressor and Deep Learning, Part 6

Strategy and techniques for Kaggle competitions, focusing on RandomForestRegressor and fastai deep learning models, including hyperparameter optimization and preprocessing.
project thumbnail

Addressing the Out-of-Domain Problem in Feature Selection, Part 5

Comprehensive exploration of feature importance in tackling the out-of-domain problem, including identification, significance, and mitigation strategies.
project thumbnail

Statistical Interpretation of Tabular Data, Part 4

Advanced statistical visualization techniques for data interpretation, including dendrograms, Spearman rank correlation, and partial dependence plots.
project thumbnail

RandomForestRegressor Performance Analysis, Part 3

In-depth analysis of RandomForestRegressor, focusing on feature importance and out-of-bag error as key performance indicators.
project thumbnail

Decision Tree Feature Selection Methodology, Part 2

Detailed exploration of feature selection using sklearn's DecisionTreeRegressor. It includes testing the reliability of the feature_importances_ method.
project thumbnail

Advanced Missing Value Analysis in Tabular Data, Part 1

Comprehensive techniques for visualizing and handling missing values in large datasets. The article demonstrates advanced strategies for filling missing values and employing categorical embeddings.
project thumbnail

Expert Analysis in Lasso Regression: The Datacamp Concrete Challenge

This article documents my approach to the Datacamp Concrete Challenge, completed within an hour. It offers detailed explanations for each step, emphasizing my proficiency with the Lasso regression model in a time-constrained environment.

Data Preprocessing

project thumbnail

Advanced Geospatial Feature Creation

Demonstrates my expertise in data cleaning and transformation for generating geospatial features. The process involves converting raw GPS data into decimal degrees format and identifying key transportation hubs in Hamburg, Germany.
project thumbnail

Advanced Data Cleaning and Validation: Batch Processing with Pandas, Part 4

Showcases advanced techniques in data cleaning and validation, using batch processing with pandas and regular expressions. It exemplifies how to handle large volumes of tabular data efficiently.
project thumbnail

Geospatial Engineering in Pandas: Creating Valid GPS Columns, Part 3

Highlights my skills in geospatial data engineering using pandas. This part focuses on cleaning, validating, and creating GPS data columns by merging longitude and latitude fields into Point geometry objects.
project thumbnail

PyJanitor Proficiency: Efficient String Data Cleaning, Part 2

Demonstrates my proficiency with pyjanitor in streamlining string data cleaning. This installment details method chaining and advanced string manipulation techniques for optimizing data preprocessing.
project thumbnail

Mastery in Pandas: In-Depth Data Exploration, Part 1

This article showcases my expertise in using pandas for advanced data exploration. It focuses on analyzing a 47-column dataset, providing insights into leveraging pandas for complex tabular data management.

Deep Learning

project thumbnail

Essential Tasks for Machine Learning Projects with Unstructured Data

A concise overview of the six vital tasks in machine learning projects handling unstructured data, covering aspects from feature engineering to workflow optimization.
project thumbnail

Fundamental Concepts of Stochastic Gradient Descent Explained

A comprehensive exploration of the mathematical foundations of Stochastic Gradient Descent, crucial for understanding its application in optimization algorithms in deep learning.
project thumbnail

Automation Using A Test Harness
For Deep Learning:
Part 2

Continuation of the series, delving into empirical experimentation using fastai on a unique dataset. This segment focuses on handling a small dataset with 850 images and two target labels, demonstrating advanced image data handling and classification skills.
project thumbnail

Automation Using A Test Harness
For Deep Learning:
Part 1

Outlines the development and application of a custom test harness to optimize deep learning testing workflows. This approach enhances GPU utilization and accelerates model building and parameter testing, using the fastai library.

Statistics

project thumbnail

Understanding and Detecting Multicollinearity in Data Analysis

A thorough examination of multicollinearity in statistical datasets, explaining covariance and Pearson correlation coefficient techniques for identifying inter-variable relationships.
project thumbnail

Economic Analysis of Hearthstone: Euro to In-Game Currency

Delving into the economic aspects of Hearthstone, this article provides a detailed simulation of in-game currency value, offering insights into the real-world monetary worth of Hearthstone decks.