Use all, or none - it's up to you
The bigger questions:
-
why do you want to remove them?
If it's a RAM usage issue, then you may want to look here:
Superdirt lazy samples loading -
how you plan to remove them -
So long as disk space is not an issue, I recommend doing it via the supercollider startup.scd - commenting out the default samples line.
If you just delete the files, you'll likely run into issues when you want to upgrade the superdirt quark down the track