Mercury doesn't dissolve the gold fast enough to stop a Pewterarm alone from breaking out of whatever mercury container you have them in, not to mention the benefit of Compounding pewter would make it faster for them to escape. Or, he could just swallow his gold- and pewter-minds, then the mercury can't affect them as much while still giving him all of the benefits. He could even just tap his weight and break free of the container.
Another issue is getting the amount of mercury needed to keep someone like that contained and completely covered. Mercury is very hazardous, complete with gasses that it gives off at room temperature (another issue with keeping the hypothetical Rashek down). Most people would end up in bad shape after being around that much mercury for long, and F-gold with no Compounding wouldn't be enough to repair that damage unless you've literally been storing for decades.
I mean theoretically, if you could neutralize Rashek for long enough to shove him into a massive vat of mercury and keep him there, this would be a viable strategy, but good luck with that.