“Music is a world within itself, it is a language we all understand.” – Stevie Wonder
Like Stevie Wonder said, music is a powerful language that has united the world through culture, politics, and ultimately as a form of expression. Its power throughout time cannot be disputed, specifically in its ability to show the societal, social, and political changes over time. By analyzing song lyrics, our team decided to conduct an analysis on the evolution of music in order to identify some of these changes. Based on the analysis of Kaylin Walker’s “50 Years of Pop,” we analyzed 50 years of the Top 100 Billboard songs (1965 – 2015), applying topic clustering to identify main themes and evaluate key trends in song content.
We used SAS Enterprise Miner to identify the main topics that were observed across all song lyrics. Here is what we found, in order of relevance.
- Love – baby, love, come on, lovin, good
- Heart Break – heart, cry, break, tear, fall
- Dance – dance, come on , music, move, shake
- Home – little, home, down, night, day
- Dream – love, dream, eye, sweet, sky
- Romance – wanna, tonight, want, night, right
- Life – live, world, life, people, find
- Girl – girl, wanna, want, man, look
- Memories – time, back, day, night, remember
When you think of the songs you hear on the radio or on a Spotify playlist, of course you expect to hear a love song about how someone got their heart broken or a jam that just makes you want to get up and dance. But what happens when you drill down by the decade?
This graph shows the overall trend for each topic across the decades from the 1960s to the 2010s. The horizontal axis indicates each topic, and each bar represents the average topic relevance for each decade. The left side of the graph shows the topics with the highest average topic weight across the entire time period. Thus, themes like love, memories, and dream dominate the Billboards regardless of the decade. However, this cannot be said for all of the topics.
Here is the same graph, but now focusing on the other side of the spectrum. These topics, like dance and romance, seem to follow an uptrend over time. Looking at dance for example, it might be inferred that the initial spike in the 1970s signals the beginning of the age of a “boogie wonderland.” And who would not get up to dance in the 2000s with songs like “Boom Boom Pow” by the Black Eyed Peas or “Single Ladies” by Beyoncé? In order to further evaluate the results by time, we decided to look at the data in another way.
In this graph, the data is flipped. Each decade is on the horizontal axis and each bar represents the specific topic. In the 1960s, the era of psychedelic rock and soul were captured through love, memories, and dreams. The disco dancing and the emerging R&B music in the 70s brought about an increase in dancing and desire, which continued throughout the 1980s and 1990s. And finally, we saw more dancing, dreams, and memories that dominated more of the recent decades.
We also wanted to analyze the use of filler words over time. We thought this analysis would be interesting, since it seemed that more often than not that songs have a lot more “ooos,” “umms,” and “las.” To our surprise, as can be seen by the chart above, was that there were not a lot of filler words in the songs overall, and they had an approximately even distribution among each of the decades. You might say that there was a spike in the 2000s, but 0.75% of the total words is still very small relatively speaking. It might also be due to the confounding effect of Usher’s song “Yeah” where he says the word 73 times in a matter of four minutes.
We thoroughly enjoyed working on this project, and for further analysis, we might drill down by the artist or the genre of the songs. We also recognized that we did not have equal coverage for all the decades, as we only had 5 years from the 1960s and the 2010s. It will be interesting to see how, and if, the results will change by 2020. As culture continues to change, we would expect to see the changes in music. Whether there will be more love, less dancing, or an entire new theme all together, we would hope that further analysis in text mining will be able to capture continuously changing culture and way of expression.
Peace, love, and rock on!
Columnists: Madison Smith, Kelly O’ Donnell, Nimish Garg, Andrew Procter, Kasey Jones