... I have virtually no NFB with the pot all the way up. At half way up, I have about 12% ...
You do realize that the way you have the NFB pot wired, as you turn it up for more negative feedback, you're also making the 470Ω resistor larger and increasing feedback 2 ways, right?
At the minimum setting, you have the full resistance of the 10k pot and the 47k resistor in series as one half of the feedback divider (the series resistance), with 470Ω to ground as the other half (the shunt resistance).
At maximum, you have 47k as the series resistance of the feedback divider, and 10k + 470Ω as the shunt part of the feedback divider. The 10,470Ω resistance to ground is also changing the resistance the long-tail is standing on, and may alter its operation somewhat.
So I hope you're not calculating assuming that the 470Ω to ground is a constant value, because it's not the way you have the schematic drawn (and presumably, wired).
You seem to be ignoring that the voltage present at the speaker terminals (which depends on output power and load impedance for the tap used) is the quantity being divided by the feedback resistor divider, and the resulting quantity of voltage determines the amount of feedback. Resistor ratios by themselves are meaningless without a knowledge of how much source voltage is being divided and fed back.
Anyway, all that said, if it works for you, then cool. Pick your best-sounding 3-4 arrangements, wire up your switch.