Plex is fine as a whole, it just handles my music library kind of clunkily and doesn't have much support for organization or dynamic playlists. It's obviously meant more for movies and shows and that's why Plexamp exists (which I don't use).
Whenever I posted on /r/Plex on Reddit, people would comment that I should use another player, but that place is a cesspool with dedicated Plex haters; it's so weird. Plex does function as a music player, it's just a bit unfocused (design-wise) and bloated.
When I don't want to boot up Plex I use mocp, a terminal-based music player, so I'm not in need of a fancy player like Deadbeef, Strawberry, Audacious, MediaMonkey, Musicbee, etc. but they do offer more to the user than Plex does.
I don't know anything about Jellyfin as a user but I've heard from others Jellyfin and Emby are ready to compete with Plex on a level playing field. Not sure if it's an exaggerated hype or not. :P