Remove the update manager? Remove the bootloader and all kernels if you want to - you might if you're preparing a container image, it won't stop you. Remove glibc and init? Fine, if that's what you want - might have no need for those if you're prepping it up for embedded.
The price of having a computer that does exactly what it's told is that you have to know what to tell it. But that's well worth while.
Speaking as someone who has to use Google Cloud Platform most days; they also do some horrific stuff, and their documentation and support are miles behind the AWS and Azure equivalents. And having worse documentation than Azure is quite the achievement.