Interesting methodology here, but the examples cited are ridiculous. First of all Billboard Top 100 in recent years contains the worst quality modern pop music out there. So you are starting out with a highly flawed data set.
#musictheory #cosomusic

//

Hit songs rely on increasing “harmonic surprise” to hook listeners, study finds | Ars Technica
https://arstechnica.com/gaming/2021/08/hit-songs-rely-on-increasing-harmonic-surprise-to-hook-listeners-study-finds/

^ Also, it is VERY difficult to hear the harmonic shifts in "This is America" during the verses (even for someone with my background). I assert that the only thing "surprising" about that song is the video. I never thought that song was anything special on its own; people are conflating it with the (excellent) video.

(If you think I'm just a #cosoclassical nerd looking down on hip-hop, you're mistaken. I've always been a big fan of artists like Wu-Tang Clan, The Roots, Chance the Rapper.)

The idea of harmonic "surprise" driving listener interest is not new at all, and has been studied before. There is plenty of music from the 1500s that still does this.

To be clear: there is a lot of GREAT pop music out there. I am just saying that most of Billboard is formulaic crap, so that anything that is even remotely "different" stands out in context.

Here is a much better example of what this article is discussing. This is a brilliantly written song.

https://strongsongspodcast.com/episode/single-ladies-by-beyonce

@voltronic

I'm trying to recall the time when most popular music wasn't formulaic crap, and not having great luck.

@mcfate
Are you discounting the many excellent bands and singer-songwriters from the 60s and 70s??? You can't honestly say that Carol King, Stevie Wonder, The Beatles, Led Zeppelin, Marvin Gaye, Jimi Hendrix, etc. are formulaic crap.

@mcfate
Queen, Black Sabbath, Rush, Ray Charles, Prince, Talking Heads, Siouxsie and the Banshees, The Beach Boys ... I could go on.

@voltronic

I'm not at all sure how to communicate that you can't "prove" there was no bad popular music by appealing to groups that didn't happen to MAKE any.

@mcfate
You're completely missing the point. That isn't even remotely what I'm doing.

You said that you can't recall a time where most pop music wasn't formulaic crap. I just gave you examples of artists that were highly popular in their time, but were also highly creative and wrote excellent music.

There is obviously bad music in any era. There is no need for strawmen here.