While this is considered a feature (by Canonical) and is specially designed for those who love to minimize the app and listen to the music in the background, some of you (including me) might find this inconsistent behavior annoying, not to mention that it doesn’t come with an option to disable it. For those who are frustrated with this issue, here is the workaround fix:

  1. Add the PPA:
  2. Update the system and install the Rhythmbox plugin: “rhythmbox-plugin-close-on-hide”
  3. Open Rhythmbox and go to “Tools -> Plugins.” Select “Close on Hide.”

Now, when you click the Close button in Rhythmbox, regardless of whether or not the music is playing, it will terminate and exit. That’s it.