Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/current_recommendation_system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Ember Current Recommendation System

The Ember app currently utilizes a hybrid approach for its music recommendations, combining external collaborative filtering via YouTube Music's API with local heuristic-based scoring.

## Architecture

The recommendation engine spans both the Dart backend services (`RecommenderService`) and Python microservices (`PythonService` communicating with `catalog.py`).

### Stage 1: Candidate Generation (Retrieval)
When a song finishes or the user skips tracks, the system must choose a new track to queue. It compiles a "candidate pool" from two primary sources:

1. **YT Music Similar Graph:**
* The app sends the `videoId` of the currently playing track to the Python backend (`android_api.py` -> `catalog.py`).
* The Python backend fetches the "watch playlist" from YouTube Music, effectively acting as an external collaborative filtering system that knows what other users listened to alongside that track.
2. **Local Favorites:**
* To inject user preference directly, the system randomly samples up to 5 tracks from the user's local database of favorite songs.

### Stage 2: Ranking Pipeline
Once candidates are pooled, they pass through a lightweight, synchronous ranking algorithm running in Dart:

1. **Base Scoring:**
* Songs are scored initially based on their order in the candidate list. Higher-ranked candidates from YT Music start with a slightly higher base score (`100.0 - (index * 1.5)`).
* Tracks that appear in both candidates (e.g., in favorites and similar) get a `+1.0` fusion bonus.
2. **Artist Affinity Multiplier:**
* The most significant step is personalization based on the `_artistAffinityCache`.
* The app dynamically calculates an "affinity score" for each artist based on:
* **Play History:** `+1` point every time a song by that artist is played.
* **Favorites:** `+3` points for each favorited song by that artist.
* **Cold Start:** `+10` points for artists selected during the initial app onboarding.
* If a candidate track belongs to an artist with a recorded affinity, its final score significantly boosts (`score += affinity * 2.0`).
3. **Fatigue Penalty:**
* To prevent endless loops, the engine checks the 20 most recently played `videoId`s. If a candidate was recently played, it receives a `-200.0` penalty, ensuring it sinks to the bottom of the recommendations.

### Limitations of Current Model
- Lacks semantic understanding of music (like tempo, genre, or mood).
- Heavily relies on internet connectivity for the initial "similar graph" generation.
- The scoring model only uses artists, completely ignoring genres or collaborative clustering based on the user's explicit listening sessions locally.
28 changes: 28 additions & 0 deletions docs/music_recommendation_research.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Modern Music Recommendation Algorithms

This document summarizes state-of-the-art approaches to building music recommendation systems, moving beyond simple popularity metrics to highly personalized, context-aware models.

## 1. Collaborative Filtering (CF)
Collaborative filtering predicts what a user will like based on the preferences of similar users or the relationships between items.
- **User-User CF:** Finds users with similar tastes and recommends tracks they have enjoyed.
- **Item-Item CF:** Analyzes co-occurrence of songs in listening sessions (e.g., if Song A and Song B are often played together, recommend B when A is playing).
- **Matrix Factorization (MF):** A mathematical technique that decomposes user-item interactions into lower-dimensional latent features, allowing the system to uncover hidden affinities even when data is sparse.

## 2. Content-Based Filtering
This approach analyzes the actual properties of the items to recommend similar tracks. It is essential for solving the "cold start" problem (new tracks with no listening history).
- **Metadata Analysis:** Using TF-IDF or text embeddings on track titles, artist names, genres, and lyrics.
- **Audio Feature Extraction:** Analyzing acoustic features like BPM (tempo), danceability, energy, and key using neural networks.

## 3. Deep Learning & Sequence Modeling
Modern streaming giants (like Spotify and YouTube Music) utilize deep learning to understand complex user patterns.
- **Sequence Modeling:** Since music is consumed in sequential sessions, models like Recurrent Neural Networks (RNNs), Transformers (like BERT4Rec), and Markov Chains analyze the *order* of songs to predict the perfect "next track".
- **Embeddings:** Utilizing models like Word2Vec to map tracks and users into a high-dimensional vector space. The distance between vectors represents similarity.

## 4. Context-Aware Recommendations
Recommendations shouldn't just depend on historical taste, but also the current situation.
- **Temporal Context:** Adjusting weights based on the time of day (e.g., energetic tracks in the morning, lo-fi beats at night).
- **Recent Fatigue:** Applying heavy negative weights to songs heard in the last 24 hours to prevent repetitive loops.

## How to Optimize with Python & Dart in Hybrid Apps
- **Python's Role:** Python excels at vector mathematics and machine learning. Libraries like `scikit-learn` or `numpy` can be used to locally train Matrix Factorization models or compute TF-IDF similarities on user databases as a background task.
- **Dart's Role:** Dart (Flutter) excels at immediate, stateful UI updates and lightweight heuristics. Dart should handle the final recommendation ranking loop (applying immediate context, fatigue penalties, and cache lookups) to ensure near-zero latency when skipping tracks.
97 changes: 97 additions & 0 deletions ember/recommendation_engine.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import json
import logging
import math
from collections import defaultdict
from typing import Any, Dict, List

log = logging.getLogger("recommendation_engine")

def build_cooccurrence_matrix(history: List[Dict[str, Any]], session_threshold_ms: int = 30 * 60 * 1000) -> Dict[str, Dict[str, float]]:
"""
Builds an item-item co-occurrence matrix from play history based on sessions.
Args:
history: List of play history track definitions containing 'videoId', 'artist', 'timestamp'
session_threshold_ms: Time difference in ms to consider tracks as part of the same session
Returns:
JSON serializable dict mapping artist to related artists and their affinity scores.
"""
if not history:
return {}

# Sort history by timestamp just in case
def get_ts(x: Dict[str, Any]) -> int:
try:
return int(x.get("timestamp", 0))
except (ValueError, TypeError):
return 0

sorted_history = sorted(history, key=get_ts)

sessions = []
current_session = []
last_ts = 0

for track in sorted_history:
ts = get_ts(track)
artist = track.get("artist")
if not artist or artist.lower() == "unknown artist":
continue

if not current_session:
current_session.append(artist)
last_ts = ts
else:
if ts - last_ts <= session_threshold_ms:
current_session.append(artist)
else:
sessions.append(current_session)
current_session = [artist]
last_ts = ts

if current_session:
sessions.append(current_session)

# Build co-occurrence map
co_occurrence = defaultdict(lambda: defaultdict(int))
artist_counts = defaultdict(int)

for session in sessions:
# Unique artists in this session to avoid self-loop bias inside a single long playlist
unique_artists = list(set(session))
for i in range(len(unique_artists)):
artist_a = unique_artists[i]
artist_counts[artist_a] += 1
for j in range(i + 1, len(unique_artists)):
artist_b = unique_artists[j]
co_occurrence[artist_a][artist_b] += 1
co_occurrence[artist_b][artist_a] += 1

# Normalize into a graph with similarities based on Jaccard-like index
affinity_graph = {}
for artist_a, related in co_occurrence.items():
edges = {}
for artist_b, count in related.items():
# Standard association rule metric: count / (freq(a) + freq(b) - count)
# We scale it up so it's a usable integer/float multiplier for Dart
score = (count / float(artist_counts[artist_a] + artist_counts[artist_b] - count)) * 100.0
if score > 0:
edges[artist_b] = round(score, 2)
if edges:
affinity_graph[artist_a] = edges

return affinity_graph

def analyze_history(history_json: str) -> str:
"""
Parses the JSON play history string from Dart and builds the graph.
Returns stringified JSON.
"""
try:
data = json.loads(history_json)
if not isinstance(data, list):
return "{}"
graph = build_cooccurrence_matrix(data)
return json.dumps(graph)
except Exception as exc:
log.error("Failed to build affinity graph: %s", exc)
return "{}"
3 changes: 3 additions & 0 deletions ember_flutter/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,6 @@ app.*.map.json

# Widget Preview related
.widget_preview/

# Secrets
.env
1 change: 1 addition & 0 deletions ember_flutter/android/app/src/main/AndroidManifest.xml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
android:name="com.ryanheise.audioservice.AudioService"
android:foregroundServiceType="mediaPlayback"
android:exported="true"
android:stopWithTask="true"
tools:ignore="Instantiatable">
<intent-filter>
<action android:name="android.media.browse.MediaBrowserService" />
Expand Down
4 changes: 4 additions & 0 deletions ember_flutter/lib/blocs/audio/audio_bloc.dart
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ class AudioBloc extends Bloc<AudioEvent, AudioState> {
queue: queue,
nativeIndexOffset: offset,
currentTrack: currentTrack,
isFetchingStream: true,
),
);

Expand All @@ -170,6 +171,9 @@ class AudioBloc extends Bloc<AudioEvent, AudioState> {
await player.setAudioSource(_playlist);

final success = await _addTrackToPlaylist(offset, queue);

emit(state.copyWith(isFetchingStream: false));

if (success) {
if (state.resumePositionMs != null && state.resumePositionMs! > 0) {
await player.seek(Duration(milliseconds: state.resumePositionMs!));
Expand Down
6 changes: 6 additions & 0 deletions ember_flutter/lib/blocs/audio/audio_state.dart
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ class AudioState extends Equatable {
final Color? dominantColor;
final dynamic playerState; // just_audio PlayerState
final int? resumePositionMs;
final bool isFetchingStream;

const AudioState({
this.queue = const [],
Expand All @@ -17,6 +18,7 @@ class AudioState extends Equatable {
this.dominantColor,
this.playerState,
this.resumePositionMs,
this.isFetchingStream = false,
});

AudioState copyWith({
Expand All @@ -26,6 +28,7 @@ class AudioState extends Equatable {
Color? dominantColor,
dynamic playerState,
int? resumePositionMs,
bool? isFetchingStream,
}) {
return AudioState(
queue: queue ?? this.queue,
Expand All @@ -34,6 +37,7 @@ class AudioState extends Equatable {
dominantColor: dominantColor ?? this.dominantColor,
playerState: playerState ?? this.playerState,
resumePositionMs: resumePositionMs ?? this.resumePositionMs,
isFetchingStream: isFetchingStream ?? this.isFetchingStream,
);
}

Expand All @@ -48,6 +52,7 @@ class AudioState extends Equatable {
playerState: playerState,
currentTrack: currentTrack ? null : this.currentTrack,
dominantColor: dominantColor ? null : this.dominantColor,
isFetchingStream: isFetchingStream,
);
}

Expand All @@ -59,5 +64,6 @@ class AudioState extends Equatable {
dominantColor,
playerState,
resumePositionMs,
isFetchingStream,
];
}
2 changes: 2 additions & 0 deletions ember_flutter/lib/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@ import 'package:ember_flutter/widgets/mini_player.dart';
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:just_audio_background/just_audio_background.dart';
import 'package:flutter_dotenv/flutter_dotenv.dart';

void main() async {
WidgetsFlutterBinding.ensureInitialized();
await dotenv.load(fileName: ".env");

await JustAudioBackground.init(
androidNotificationChannelId: 'com.example.ember.channel.audio',
Expand Down
48 changes: 23 additions & 25 deletions ember_flutter/lib/screens/about/about_screen.dart
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import 'package:ember_flutter/screens/settings/settings_screen.dart';
import 'package:ember_flutter/theme.dart';
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:url_launcher/url_launcher.dart';

import 'package:ember_flutter/theme.dart';
import 'package:ember_flutter/screens/settings/settings_screen.dart';
import 'package:package_info_plus/package_info_plus.dart';
import 'package:url_launcher/url_launcher.dart';

class AboutScreen extends StatelessWidget {
const AboutScreen({super.key});
Expand Down Expand Up @@ -179,28 +178,27 @@ class AboutScreen extends StatelessWidget {
),
const SizedBox(height: 8),
FutureBuilder<PackageInfo>(
future: PackageInfo.fromPlatform(),
builder: (context, snapshot) {
final version = snapshot.hasData ? "v${snapshot.data!.version}" : "v1.0.1";
return Container(
padding: const EdgeInsets.symmetric(horizontal: 12, vertical: 4),
decoration: BoxDecoration(
color: Colors.white.withValues(alpha: 0.08),
borderRadius: BorderRadius.circular(20),
border: Border.all(color: Colors.white.withValues(alpha: 0.05)),
),
child: Text(
version,
style: const TextStyle(
fontSize: 12,
color: Colors.white70,
fontWeight: FontWeight.w600,
letterSpacing: 0.5,
future: PackageInfo.fromPlatform(),
builder: (context, snapshot) {
final version = snapshot.hasData ? "v${snapshot.data!.version}" : "v1.0.1";
return Container(
padding: const EdgeInsets.symmetric(horizontal: 12, vertical: 4),
decoration: BoxDecoration(
color: Colors.white.withValues(alpha: 0.08),
borderRadius: BorderRadius.circular(20),
border: Border.all(color: Colors.white.withValues(alpha: 0.05)),
),
),
);
}
),
child: Text(
version,
style: const TextStyle(
fontSize: 12,
color: Colors.white70,
fontWeight: FontWeight.w600,
letterSpacing: 0.5,
),
),
);
}),
],
),
],
Expand Down
3 changes: 2 additions & 1 deletion ember_flutter/lib/screens/artist/artist_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,12 @@ class _ArtistScreenState extends State<ArtistScreen> {
Future<void> _load() async {
final browseId = widget.artistInfo['videoId'] ?? widget.artistInfo['browseId'];
if (browseId == null || browseId.isEmpty) {
if (mounted)
if (mounted) {
setState(() {
_loading = false;
_error = "Invalid artist link.";
});
}
return;
}

Expand Down
3 changes: 2 additions & 1 deletion ember_flutter/lib/screens/download/bloc/download_bloc.dart
Original file line number Diff line number Diff line change
Expand Up @@ -143,14 +143,15 @@ class DownloadBloc extends Bloc<DownloadEvent, DownloadState> {
final downloads = await DatabaseService.instance.getAllDownloads();

// Ensure we invoke emit safely
if (!isClosed)
if (!isClosed) {
emit(
state.copyWith(
progress: finalProgress,
completed: finalCompleted,
downloadedTracks: downloads,
),
);
}
} catch (e) {
_failDownload(videoId);
}
Expand Down
17 changes: 2 additions & 15 deletions ember_flutter/lib/screens/history/history_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -46,19 +46,6 @@ class _HistoryScreenState extends State<HistoryScreen> {

void _playSong(BuildContext context, Map<String, String> track) {
context.read<AudioBloc>().add(AudioPlayQueue([track], startIndex: 0));

Navigator.push(
context,
PageRouteBuilder(
pageBuilder: (context, animation, secondaryAnimation) => const FullPlayerScreen(),
transitionsBuilder: (context, animation, secondaryAnimation, child) {
const begin = Offset(0.0, 1.0);
const end = Offset.zero;
var tween = Tween(begin: begin, end: end).chain(CurveTween(curve: Curves.fastOutSlowIn));
return SlideTransition(position: animation.drive(tween), child: child);
},
),
);
}

Widget _buildListeningStats() {
Expand Down Expand Up @@ -111,7 +98,7 @@ class _HistoryScreenState extends State<HistoryScreen> {
Container(
padding: const EdgeInsets.symmetric(horizontal: 12, vertical: 4),
decoration: BoxDecoration(
color: YTColors.primary.withOpacity(0.2),
color: YTColors.primary.withValues(alpha: 0.2),
borderRadius: BorderRadius.circular(12),
),
child: Text(
Expand All @@ -122,7 +109,7 @@ class _HistoryScreenState extends State<HistoryScreen> {
],
),
);
}).toList(),
}),
],
),
);
Expand Down
Loading
Loading