︎
#eye #eye

A Guide to Your First Python Data Analysis Project

Table of Contents:
  1. Understanding the Fundamentals of Data Analysis
  2. Setting Up Your Python Analysis Environment
  3. Loading and Exploring the Spotify Dataset
  4. Identifying and Handling Missing Data
  5. Filtering and Subsetting Data
  6. Analytical Questions and Insights
  7. Visualizing Data
  8. Exploring Relationships Using Regression
  9. Summary and Next Steps
  10. Glossary

Analyzing Spotify Song Attributes with Pandas, Matplotlib, and Seaborn

This guide walks you through a full, real-world data analysis workflow using Python. Our goal is to explore a large dataset of Spotify songs and uncover patterns in characteristics like energy, danceability, popularity, and genre. Along the way, we will learn essential analysis skills including loading data, examining its structure, cleaning it, filtering it, summarizing it, visualizing relationships, and interpreting a simple regression.

1. Understanding the Fundamentals of Data Analysis

Before writing any code, it is useful to understand a few foundational concepts. These concepts shape how we analyze, filter, and visualize data.

Types of Data in a Dataset

Every dataset is made up of different kinds of variables, and each type determines what you can do analytically.

Numerical Variables

These include values that can be counted or measured, such as tempo, energy, loudness, danceability, and popularity.

Numerical variables allow you to compute:

  • averages
  • correlations
  • minimum and maximum values
  • distributions
  • regression models

They also work well in visualizations like histograms, line charts, bar charts, heatmaps, and scatterplots.

Categorical Variables

These represent groups, categories, or labels, such as genre, artist_name, key, or mode. They are essential for:

  • filtering subsets of the data
  • grouping values by category
  • comparing differences between groups
  • computing category-level statistics, such as average danceability by genre

Together, numerical and categorical variables form the backbone of most data analysis tasks. Recognizing them helps you choose appropriate methods and avoid errors. For example, you would not calculate the “mean genre,” nor would you plot text as a scatterplot axis.

What is Pandas? When do We Use It?

What is a library in Python?

A library is a collection of pre-written code that allows us to use tools someone else already made instead of building everything from scratch.

Pandas is a Python library designed for working with tabular data. If Excel could be expanded and connected to other analytical tools, that would be Pandas. It allows us to:

  • Load data from CSV files, URLs, Excel files, and more
  • Organize data into tables called DataFrames
  • Filter, sort, and group data
  • Inspect and summarize tables
  • Clean and fix messy datasets
  • Compute aggregated statistics
  • Reshape and merge datasets


2. Setting Up Your Python Analysis Environment

Before we begin working with the Spotify dataset, let’s practice writing and running a few basic lines of Python code. This will help you become comfortable with code cells, variables, lists, and simple calculations.

Printing text: The print() function displays output.
Python 
print("I am learning Python for data analysis.")

Creating variables: A variable stores information that we can reuse later.
Python 
song_title = "Blinding Lights"artist = "The Weeknd"popularity = 95

print(song_title)print(artist)print(popularity)

Python can store different types of information, including text and numbers. Text values are placed inside quotation marks, while numbers do not need quotation marks.

Doing basic math: Python can also be used like a calculator.
Python 
x = 10y = 5

print(x + y)print(x - y)print(x * y)print(x / y)

Creating lists: A list stores multiple values in one variable.
Python 
genres = ["Pop", "Rock", "Jazz", "Hip-Hop"]print(genres)

You can access a single item from a list using its position. Python starts counting at 0.
Python 
print(genres[0])print(genres[1])

"Pop" is in position 0, and "Rock" is in position 1.

Python Comparison & Logic Operators
  • = → to store a value in a variable
  • == → equal to
  • != → unequal to
  • > → greater than
  • < → smaller than
  • & → and
  • | → or
To complete this project, you will use Python along with three key libraries: Pandas for data handling, Matplotlib for basic visualizations, and Seaborn for statistical graphics built on top of Matplotlib.

If you are working in Google Colab or Jupyter Notebook, these libraries are typically pre-installed. If working locally, they can be installed using:

Python 
pip install pandas matplotlib seaborn

Once installed, you can import them. Importing a library as something gives it a nickname, so you can refer to it with a shorter name in your code.

Python 
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

The imports establish the tools you will rely on throughout the rest of the guide.

3. Loading and Exploring our Example Dataset

The dataset for this project is a CSV file. A CSV file stands for “comma-separated values” and stores data in a simple table format, where each row is a line and each value is separated by a comma. Our Spotify dataset includes more than one hundred thousand tracks from Spotify across multiple genres. Each song includes attributes like acousticness, tempo, loudness, energy, popularity, and danceability.

First, download the Spotify dataset from the Dropbox link and save it somewhere easy to find on your computer, such as your Downloads folder or the same folder as your notebook.

Then, use the file path to load the CSV file. A file path tells Python where a file is saved on your computer. If the CSV file is in the same folder as your notebook, you can use just the file name. If it is saved somewhere else, such as your Downloads folder, you need to give Python the full path to that location.

Python 
df = pd.read_csv("Users/erc/Downloads/spotify_tracks_dataset.csv")
df

You now have a DataFrame, Pandas’ core data table object, ready for analysis.

Examining the Structure of Your Dataset

Exploration is a critical step because it helps you understand what you are working with before making any assumptions or conducting deeper analysis.

Previewing the First Few Rows

Python 
df.head(5)

This gives you a sense of what types of variables exist.

Listing All Column Names

Python 
df.columns

This helps you understand the set of variables in the dataset that you can use to ask questions.

Understanding Data Types and Missing Values

Python 
df.info()

This command allows us to see:

  • which columns are integers, floats, or text
  • whether any columns contain missing values
  • the overall size of the dataset

The DType tells us what kind of data is stored in each column. Object columns usually contain text, float columns contain decimal numbers, and int columns contain whole numbers.

Summary Statistics for Numerical Variables

Python 
df.describe()

This provides key descriptive measures including:

  • mean
  • standard deviation
  • quartiles
  • min and max values

Beginning our analysis with these summaries will help us have an idea of the data’s distribution and scale.

4. Identifying and Handling Missing Data

Almost all real-world datasets contain missing information. Missing values can interrupt mathematical calculations, skew charts, and cause functions to fail.

Checking for Missing Values

Python 
df.isna().sum()

This produces a count of missing entries in each column. If there are few, you can drop rows without significant loss.

Removing Missing Data

Python 
df = df.dropna()

This removes all rows containing missing values. Be cautious – doing so without inspection may unintentionally remove important or rare data points.

5. Filtering and Subsetting Data

How can we create a smaller dataset that only includes Rock songs? We filter by genre!

Filtering by Genre

This line checks each row in the genre column to see if it equals "Rock." The matching rows are kept and saved in a new DataFrame called rock_songs.

Remember: use two equals signs, ==, to check for equality. A single equals sign, =, is used to assign a value.
Python 
rock_songs = df[df['genre'] == 'Rock']
rock_songs.head()

Filtering by a Numerical Threshold

Now let’s create a dataset that only includes songs with an energy level higher than 0.8. Energy is a numeric variable, meaning it is stored as a number. Because of this, we can use comparison symbols like greater than > or less than < to filter the data.

Python 
high_energy = df[df['energy'] > 0.8]
high_energy.head()

high_energy keeps only songs with an energy level above 0.8.

Filtering by Multiple Conditions

What if we wanted to look at songs that are both Rock songs and high energy?

Python 
high_energy_rock = df[(df['genre'] == 'Rock') & (df['energy'] > 0.8)]
high_energy_rock.head()

This narrows the data to just the rows meeting both conditions: the genre must be "Rock" and the energy value must be greater than 0.8. The & symbol means “and,” so both conditions must be true for a song to be included. Each condition is placed inside parentheses to help Python understand the full filter.

6. Analytical Questions and Insights

Once the dataset is clean and well understood, we can begin exploring meaningful questions.

What Are the Most Popular Songs?

Python 
top_10_popular = df.sort_values(by='popularity', ascending=False)
top_10_popular[['track_name', 'artist_name', 'popularity']].head(10)

Sorting helps identify rankings within the dataset. Saving the result as a new variable makes it easier to reuse later.

What Are the Most Danceable Songs?

Python 
top_5_danceable = df.sort_values(by='danceability', ascending=False)
top_5_danceable[['track_name', 'artist_name', 'danceability']].head(5)

What Is the Average Danceability or Energy for Each Genre?

Pandas’ groupby() function allows us to calculate statistics for categories. Conceptually, the operation splits the data into groups, applies a summary function like mean, and recombines the results into a new table.

Python 
genre_analysis = df.groupby('genre')[['danceability', 'energy']].mean()
genre_analysis.head(10)

This creates a profile of each genre, revealing patterns such as which genres tend to be more energetic or danceable.

Counting Songs per Genre

Python 
df.groupby('genre').size()

This shows which genres dominate the dataset.

7. Visualizing Data

Visualizations convert raw numbers into insights. They help you identify patterns that are not obvious from tables alone.

Bar Chart of Average Danceability by Genre

Python 
genre_analysis['danceability'].plot(
    kind='bar',
    title='Average Danceability by Genre',
    xlabel='Genre',
    ylabel='Danceability'
)
plt.xticks(rotation=75)
plt.show()

Bar charts are effective for comparing categories.

Scatter Plot of Energy vs Danceability

Python 
plt.figure(figsize=(8, 6))
plt.scatter(df['energy'], df['danceability'], alpha=0.3)
plt.title('Energy vs Danceability')
plt.xlabel('Energy')
plt.ylabel('Danceability')
plt.show()

Scatterplots help visualize relationships between two numerical variables. However, because this dataset is large, many points overlap on top of each other, making the chart difficult to interpret. When the points are too crowded, it becomes hard to distinguish individual songs or see whether there is a clear pattern between the variables.

Addressing Overplotting with Hexbin Charts

Python 
df.plot.hexbin(
    x='energy',
    y='danceability',
    gridsize=25,
    cmap='Blues',
    figsize=(8, 6),
    sharex=False
)
plt.title('Density of Songs: Energy vs Danceability')
plt.show()

Hexbin charts reveal areas where many songs share similar values. The areas with the darker colors are the ones with the highest concentration.

Another way to explore the relationship between energy and danceability is to group songs by energy level and calculate the average danceability for each group. This gives us a simpler view of the overall trend instead of showing every individual song.

Python 
# Calculate the average danceability for each energy group
avg_danceability = df.groupby('energy_level')['danceability'].mean()

# Plot the results
avg_danceability.plot(kind='bar', figsize=(10, 6))

plt.title('Average Danceability by Energy Level')
plt.xlabel('Energy Level')
plt.ylabel('Average Danceability')
plt.show()

8. Exploring Relationships Through Regressions

Regression analysis helps reveal underlying relationships between variables. In this case, we want to know whether a song’s energy level predicts its danceability. While regression can be complex, Seaborn's lmplot() offers an accessible entry point: it visualizes trends without requiring mathematical background.

Basic Regression Plot

Python 
sns.lmplot(
    data=df,
    x='energy',
    y='danceability',
    height=6,
    aspect=1.2
)
plt.title("Linear Regression: Energy vs Danceability")
plt.show()

The resulting chart should show:

  • A fitted line that captures the overall trend
  • A cloud of points representing the data
  • A shaded confidence interval showing uncertainty

Using Sampling to Improve Visibility

Because the dataset is large, sampling helps reduce visual clutter:

Python 
sample_df = df.sample(1000)

sns.lmplot(
    data=sample_df,
    x='energy',
    y='danceability',
    scatter_kws={'color': 'pink'},
    height=6,
    aspect=1.2
)
plt.title("Regression Using a Sample of 1,000 Songs")
plt.show()

Comparing Regression Lines Across Genres

This helps highlight whether the relationship between energy and danceability varies across musical styles:

Python 
sns.lmplot(
    data=df.sample(5000),
    x='energy',
    y='danceability',
    hue='genre',
    scatter_kws={'alpha': 0.2},
    height=7,
    aspect=1.3
)
plt.title("Energy vs Danceability Across Genres")
plt.show()

Different slopes indicate different underlying patterns across genres.

Observation/note: The chart above includes many different genres, which makes it difficult to tell the colors and trend lines apart. To make the chart easier to read, we could use a sample of the data, focus on fewer genres, or choose a color palette with stronger contrast.

Extracting Numerical Regression Results

For a more technical summary:

Python 
from scipy.stats import linregress

result = linregress(df['energy'], df['danceability'])
print(result)

This provides the slope, intercept, correlation, and statistical significance, also known as the p-value.

9. Guide Summary and Next Steps

In this guide, you learned how to complete an end-to-end data analysis workflow in Python. This included:

  • Importing and loading data
  • Exploring structure and content
  • Identifying and cleaning missing values
  • Filtering subsets of data
  • Computing descriptive statistics
  • Comparing categories
  • Creating visualizations
  • Interpreting regression results

You now have the tools to extend this analysis by exploring other Spotify features, testing more relationships, or even building predictive models!

10. Glossary of Terms


  • CSV file: a plain-text file that stores table-like data using rows and columns, with values separated by commas
  • File path: the location of a file on your computer that Python uses to find and open the file
  • Python notebook: a document that lets you combine Python code, written notes, and code output in one place
  • Code cell: a section in a notebook where you write and run Python code
  • Comment: any text following a hashtag symbol (#), which Python does not recognize as code
  • Function: a pre-defined block of code designed to perform a specific task
  • Variable: a name used to store information
  • DataFrame: a table-like structure in pandas that organizes data into rows and columns
  • Row: a horizontal entry in a DataFrame, usually representing one observation, such as one song
  • Column: a vertical section in a DataFrame, usually representing one variable, such as genre, energy, or danceability
  • DType: short for data type; it tells us what kind of information is stored in a column
  • Object data: text or mixed character data, such as song titles, artist names, or genres
  • Float data: numeric data with decimal places, such as 0.85 or 0.42
  • Integer data: whole-number data without decimal places, such as 10, 85, or 100
  • Numeric data: numbers that can be used for calculations, including integers and decimals
  • Boolean data: data with only two possible values, True or False
  • Indexing: accessing specific rows, columns, or values in a DataFrame using square brackets
  • Filtering: selecting only the rows that meet a specific condition
  • Subsetting: creating a smaller version of a dataset by selecting specific rows or columns
  • Condition: a rule that Python checks, such as whether a song’s genre is Rock or whether its energy is greater than 0.8
  • Missing value: a blank or empty value in a dataset where information is not available
  • Overplotting: when too many points overlap in a chart, making it difficult to distinguish individual data points or patterns
  • Hexbin chart: a chart that groups nearby points into hexagon-shaped areas and uses color to show where the data is most concentrated
  • Regression line: a line that shows the general trend or relationship between two numerical variables
  • Sample: a smaller portion of a dataset used to make analysis or visualization easier

By: Rayhana Mouaouia