> with volumes full up... relationship between the frequency of the sine wave, the voltage at the input
You do not have to run "full up", you do not have to know the input voltage or the frequency.
What Iannone said. Measure the voltage. square it, divide by the load you are using.
It is customary to use a Sine, and for the output to be "sine-like". Tube amps always bend the wave, and for gitar it is reasonable to turn-up until the wave is clearly bent (by eye on 'scope or by by ear on a monitor speaker).
Audio amps are not normally rated on Square wave. In most audio this is very unrealistic. We never have a full power sine, and never-never have a full-power square which will sag the power supply worse than the full sine. In over-driven guitar, maybe it IS reasonable. However many meters are calibrated for sine-waves and give different numbers for other wave shapes.
You can vary the frequency. For a first-test use 400Hz, where guitar has a lot of power and where most guitar amps do best. Then check 100Hz to see if bass is strong or weak (either way is acceptable for different guitar styles). Then check 1,600 and 6KHz to be sure treble can be strong, but it is very acceptable for treble power to be half of midband power.
You can vary the load impedance, re-compute power, and see if other loading gives more power or prettier distortion.
Push-pull amps driven past flat-top sine but not to fully-square waves should be symmetrical. (Some P-P amps go assymetrical when over-over-driven, that's OK.) SE amps never make perfectly symmetric over-drive, but strong assymetry is a sign that you should change your idle current or load.
Check your meter against your signal generator. I have seen DMMs which fall-off badly above 400Hz.
> a very simple audio wattmeter
Too simple. 3db steps. You may know it is more than 25W but less than 50W; we usually want a more specific number. Anyway, we have meters.