This is simply the old artdisplay plugin from Rhythmbox <2.96.

It was removed from newer Rhythmbox versions in favor of the small cover display in the song title bar, but it still works. So if you prefer the old style, just get it.

Installation:

1.) Copy the folder 'ArtDisplay' to '~/.local/share/Rhythmbox/plugins'.

2.) Activate the plugin in Rhythmbox.