BSAN3204 Project Proposal
Optimising Spotify Playlist
Recommendations with the Million
Song Dataset
1 Introduction
This proposal uses the Million Song Dataset (MSD) to improve Spotify playlist recom- mendations. Spotify is a leading music streaming service. It has over 500 million users in
2025. The company is based in Stockholm, Sweden. Its success comes from strong recom- mendation algorithms. These power features like Discover Weekly and Daily Mixes. They keep users happy and coming back. The Spotify Data Analysis Team is the audience. They are experts who improve these algorithms. Their goal is to raise user satisfaction. They also aim to increase listening time and loyalty. The MSD has data on one million songs. It includes metadata and audio details. This project will find useful patterns. These will help make better playlists for Spotify users.
The main goal is to boost user engagement. We focus on music from different times and styles. This brings benefits. Users get better song picks. They listen longer. They stay loyal to Spotify. We use key variables for this. These are year, duration, loudness, tempo, artist name, artist familiarity, and song hotttnesss. They show what users like. They also track music trends. Early findings give clues. Songs are getting longer now. Loudness is up in new tracks. Some artists have many songs. Popular artists often have hit songs. This data helps Spotify stand out.
2 Background and Rationale
Spotify faces competition. Rivals include Apple Music, Amazon Music, and YouTube Music. They all want more users. Spotify’s recommendation system is key. It drives about 70% of streams. This comes from industry reports. Features like Discover Weekly and Release Radar use it. Personalised playlists depend on it too. This keeps Spotify ahead. The MSD offers data on one million songs. It covers many years. This helps see music patterns clearly.
Users want playlists that fit them. Studies show this keeps them engaged. Good recommendations raise retention by 20%. Spotify’s own data backs this up. The MSD can spot trends. It shows popular tempos and artist eras. It tracks song length changes too. This is vital now. Users mix old songs from the 1980s with new ones. Spotify’s Wrapped reports show this trend. The MSD helps with new songs too. These lack user data at first. Old song patterns can guide them. This project keeps Spotify strong.
3 Methodology
This project analyses the MSD with R. R is good for big data. The dataset is MSD. RData. It has one million songs and 18 variables. We start by loading it. Then we check the data. We fix issues like missing year values. Some are 0 and need removal. We handle song hotttnesss blanks too. Next, we get stats. These include averages and ranges.
We make 10 plots with ggplot2. This tool makes clear charts. Plots include bar plots and histograms. We use scatter plots and line plots too. Boxplots, density plots, and heatmaps are also made. Each plot shows one part of the data. We filter bad data first. We group some numbers like duration. This makes plots easy to read. We save them as PNG files with ggsave. These are rough now. Later, we add better colours and labels.
4 Expected Outcomes
This project will find trends. Songs are longer now than before. Loudness is higher in new music. Top artists have many tracks. Popular songs link to famous artists. Spotify can use this data. It will improve playlists. Users will enjoy them more. They will listen for longer. Engagement may rise by 10-15%. This matches industry goals. Over time, it helps with new music styles. It keeps Spotify ahead in the market.
5 Proposed Plots
These 10 plots come from the MSD. They help improve Spotify playlists. Each shows a different part of the data. Below are details on what they mean and why they help.
5.1 Plot 1: Number of Songs by Release Year
Figure 1: Bar plot of song counts per release year.
This bar plot counts songs by year. The 2000s have the most, over 300,000 tracks.
The 1960s have less, around 50,000. This shows more music from recent times. Spotify can use this info. It can pick more songs from the 2000s. Users might enjoy newer hits more. Older songs still matter for variety. This plot picks the best years. It helps make playlists users love. Engagement goes up with popular eras.
5.2 Plot 2: Distribution of Song Duration
This histogram shows song lengths. Most are 180 to 300 seconds. That’s 3 to 5 minutes. Very short songs under 100 seconds are rare. Very long ones over 600 seconds are few too. Spotify can choose these common lengths. Users like this range best. It fits their daily listening. Playlists stay smooth and easy. This plot sets a time guide. It keeps users happy with good flow.
Figure 2: Histogram of song durations.
5.3 Plot 3: Loudness vs Tempo
Figure 3: Scatter plot of loudness versus tempo.
This scatter plot looks at loudness and tempo. Loudness goes from -50 to 0 decibels.
Tempo is 50 to 200 beats per minute. No clear link shows up. Some songs are loud and fast, like -5 decibels at 180 BPM. Others are quiet and slow, like -20 at 80 BPM. This mix is good. Spotify can try different styles. It fits many user moods. This plot adds variety to playlists.
5.4 Plot 4: Average Song Duration by Year
This line plot shows average length by year. In 1970, it’s about 200 seconds. By 2010, it’s near 240 seconds. The rise starts in the 1990s. Songs grow longer over time. Spotify
Figure 4: Line plot of average song duration by year.
can match this trend. Old songs can fit new lengths. Users prefer current styles. This keeps playlists fresh. This plot tracks time changes. It helps update song picks.
5.5 Plot 5: Top 10 Artists by Song Count
Figure 5: Bar plot of top 10 artists by song count.
This bar plot lists top artists. Some have over 500 songs. They are big in the MSD.
Names stand out like stars. Spotify can use them a lot. Fans know these artists well. Playlists with them get more plays. This plot finds the best artists. It boosts playlist fame. Users stick around for big names.
Figure 6: Boxplot of song duration distribution.
5.6 Plot 6: Boxplot of Song Duration
This boxplot shows length spread. Middle songs are near 200 seconds. Some hit 600 seconds or more. These are outliers. Most stay under 400 seconds. Spotify can avoid long ones. Users might skip them. Shorter tracks keep flow. This plot spots odd times. It helps make tight playlists.
5.7 Plot 7: Loudness Distribution by Year
Figure 7: Boxplot of loudness by year.
This boxplot tracks loudness by year. Old songs average -15 decibels. New ones hit -5 decibels. Loudness rises over time. The gap is clear. Spotify can fix old tracks. It makes sound even. Users enjoy steady volume. This plot shows loudness shifts. It improves playlist sound.
5.8 Plot 8: Density of Song Tempo
Figure 8: Density plot of song tempos.
This density plot shows tempo range. Most are 100 to 150 beats per minute. That’s medium speed. Slow ones under 80 BPM are few. Fast ones over 200 BPM are rare too. Spotify can use this range. It’s what users like most. It fits lots of tastes. This plot picks top tempos. It boosts playlist energy.
5.9 Plot 9: Heatmap of Duration by Year
Figure 9: Heatmap of duration bins by year.
This heatmap maps duration by year. Boxes show ranges like 100-200 seconds. Dark areas have more songs. The 2000s show longer tracks, up to 300 seconds. Old years have shorter ones. Spotify can tweak by era. It fits old and new styles. This plot shows time patterns. It customises playlists well.
5.10 Plot 10: Song Hotttnesss vs Artist Familiarity
Figure 10: Scatter plot of song hotttnesss versus artist familiarity.
This scatter plot ties popularity to fame. Song scores go from 0 to 1. Artist fame does too. High scores match, like 0.8 for both. Famous artists have popular songs. Spotify can pick these hits. Users love known tracks. It gets more listens. This plot links fame and success. It drives playlist wins.