How would I check the voltage in the wall?
You could make a "suicide plug", but that's dangerous, so we won't mention it again.
The wall voltage has to come into the amp at the cord or IEC receptacle (which your amp probably has). Measure from hot to neutral where the cord enters the amp.
Does your voltage and bias more or less stay the same?
Not really, and it doesn't really matter. Wall voltage drifts up and down depending on season, time of day and local usage/demand, among other things. Further, tubes drift. The plate current with a given bias voltage can and will meander up and down depending on the tube sample.
It is less obvious with cathode biasing, because the cathode resistor times the changing current produces a voltage drop (which is equal to tube bias in 99% of cases) that offsets the effect of the current change. You don't get the automatic compensation in a fixed-biased amp. That is, unless you use some kind of servo-bias arrangement, but the added complexity doesn't give much in return in a guitar amp.
I can't say that I've ever cared about the small drifts you cite. If the tube drifted from 10mA to 100mA, that would be a problem and you'd have to wonder if there is a problem in the bias circuit. In your case, the bias circuit could be rock-solid and it could just be a matter of simple tube drift.