I’m looking into this myself right now. From everything I’ve read and watched (have spent way to much time on this) I think for people who’ve gone this route it is 50/50 if they think the down puffy under a quiet shell is too loud. That seems to depend on two things, the puffy itself (some are louder than others) and the hunters sensitivity to noise. Some hunters will not tolerate even next to no noise. I get it, especially when you put in the effort required to shoot big mature whitetails, you leave as little as possible to chance. How much noise is acceptable is situational, subjective, and experience based. If a puffy makes a crinkling noise when moving a quiet shell probably won’t help. If it is mostly noisy when rubbing against something then a quit shell likely will help.
If you are very sensitive to noise I would just fold and abandon the down puffy route. Then be prepared to pony up for something like a Sitka Fanatic or a FirstLite Thermic. If you think about it, a heavy fill puffy plus a quiet shell isn’t much less $$$, if any, than one of those options. And they are wind proof too. I don’t have an answer yet, sigh. Still figuring it out.