🏙️ NYC Airbnb Dynamic Pricing Service

---

1. Project Overview

This repository contains the Capstone 1 Project for the DataTalks.Club Machine Learning Zoomcamp.

The project is an end-to-end machine learning service that predicts the optimal nightly price for an Airbnb listing in New York City based on its characteristics (location, amenities, room type, etc.). The final product is a containerized REST API built with FastAPI that serves predictions from a tuned XGBoost model.

1.1. The Problem & Business Value

In the competitive short-term rental market of NYC, hosts often struggle to price their listings effectively.

• Overpricing leads to high vacancy rates and lost revenue.
• Underpricing leaves money on the table and attracts lower-quality guests.

The Solution: This project implements a "Dynamic Pricing Engine" that analyzes market data to suggest a competitive price. This helps hosts maximize occupancy and revenue (Yield Management).

---

2. Tech Stack

• Language: Python 3.10
• Dependency Management: Pipenv
• Machine Learning: Scikit-Learn (Pipeline), XGBoost (Gradient Boosting)
• API Framework: FastAPI with Uvicorn
• Data Validation: Pydantic
• Containerization: Docker & Docker Compose
• Cloud Deployment: Google Cloud Run (Serverless)

---

3. Project Methodology & Workflow

This project follows a structured MLOps workflow to ensure reproducibility and performance.

1. Research & EDA (notebooks/01_eda_and_engineering.ipynb):

   • Analyzed the "New York City Airbnb Open Data" from Kaggle.
   • Feature Engineering:
     • Target Transformation: Applied log1p to the price variable to handle the long-tail distribution.
     • Geospatial: Calculated Haversine Distance to city center (Times Square) and used K-Means clustering to identify "micro-neighborhoods."
     • NLP Lite: Extracted keywords from listing titles (e.g., "Luxury", "View").
   • Cleaning: Handled outliers (prices > $2000) and missing values.

2. Model Training & Tuning (notebooks/02_modeling.ipynb):

   • Established a Linear Regression Baseline (RMSE: ~0.45).
   • Trained a Challenger XGBoost Model.
   • Hyperparameter Tuning: Used Hyperopt (Bayesian Optimization) to tune max_depth, learning_rate, subsample, etc.
   • The final XGBoost model achieved an RMSE of 0.40 (on log price), significantly outperforming the baseline.
   • The logic was exported to src/train.py for automated training.

3. Deployment (src/main.py & Dockerfile):

   • Built a FastAPI service with Pydantic schemas for strict input validation.
   • Implemented Lifespan Events to load the model efficiently on startup.
   • Containerized the application using a multi-stage Dockerfile with a non-root user for security.
   • Deployed to Google Cloud Run for serverless scalability.

---

4. Live Deployment (Google Cloud Run)

The service is currently live and accessible via the following URL.

API Base URL: https://airbnb-pricing-service-644458477502.us-central1.run.app

4.1. Interactive Documentation (Swagger UI)

You can test the API directly in your browser using the auto-generated Swagger UI: 👉 Open Live Swagger UI

4.2. Sample cURL Command

Copy and paste this into your terminal to get a real-time price prediction:

curl -X 'POST' \
  'https://airbnb-pricing-service-644458477502.us-central1.run.app/predict' \
  -H 'Content-Type: application/json' \
  -d '{
  "neighbourhood_group": "Manhattan",
  "neighbourhood": "Midtown",
  "room_type": "Entire home/apt",
  "latitude": 40.75,
  "longitude": -73.98,
  "minimum_nights": 2,
  "availability_365": 100,
  "name": "Luxury Suite near Times Square"
}'

Expected JSON Response:

{
  "suggested_price": 244.13
}

---

5. How to Run the Project Locally

The entire application is containerized. You can spin it up with a single command.

Prerequisites:

• Git installed
• Docker installed and running.

Step 1: Clone the Repository

git clone <your-repo-url>
cd nyc-airbnb-pricing

Step 2: Build and Run with Docker Compose

docker compose up --build

The API will be available at http://localhost:9696.

Step 3: Test the Local API

Option A: Interactive Docs Open http://localhost:9696/docs in your browser.

Option B: Python Script You can use the provided test script (if created) or curl:

curl -X 'POST' \
  'http://localhost:9696/predict' \
  -H 'Content-Type: application/json' \
  -d '{
  "neighbourhood_group": "Brooklyn",
  "neighbourhood": "Williamsburg",
  "room_type": "Private room",
  "latitude": 40.71,
  "longitude": -73.96,
  "minimum_nights": 3
}'

---

6. Reproducibility Guide (Manual Setup)

If you prefer to run without Docker (e.g., for development), follow these steps.

1. Install Dependencies (Pipenv):

   pip install pipenv
   pipenv install --dev
   pipenv shell

2. Download Data: Ensure AB_NYC_2019.csv is in the data/ folder.

   • Download Link (Kaggle)

3. Train the Model: This script will train the model and save models/model_pipeline.pkl.

   python -m src.train

4. Run the Server:

   uvicorn src.main:app --host 0.0.0.0 --port 9696 --reload

---

7. Course Deliverables Checklist

This section maps the project files to the course evaluation criteria.

• Problem Description: README.md (Section 1).
• EDA: notebooks/01_eda_and_engineering.ipynb (Analysis of target, features, and geospatial data).
• Model Training: notebooks/02_modeling.ipynb (XGBoost vs Linear, Hyperopt tuning).
• Export to Script: src/train.py (Reproducible training logic).
• Reproducibility: Pipfile & Pipfile.lock included.
• Model Deployment: src/main.py (FastAPI service).
• Containerization: Dockerfile & docker-compose.yml.
• Cloud Deployment: Service deployed to Google Cloud Run (URL in Section 4).