I have never seen a voltage rating for a resistor.
All are rated in their datasheets. Most small MF Rs are OK for 200 or 250 V across them - your B+ may be much higher, what's important is what the resistor sees.
Would these be fine to use in an amp build?
Small MFs are perfect as grid leaks, loading Rs or small signal cathode loads. Plate loads are another story, I vastly prefer to use over-rated Rs there, typically 1 W for a stage with a max dissipation of 200 mW and 2 W above. The Rs will last longer, run cooler and noise will be lower.
Other the wattage and ohms, are there other rating one needs to be aware of?
Max voltage, max power, resistance and tolerances are the main factors.
Some types of carbon-based Rs have a negative temperature coefficient - their actual resistance decreases with temperature, metal films and MOXen show a positive one - R increases with T.
Carbon Rs drift significantly under load, most stabilize after "burn in".
Noise should also be considered for input stages and critical applications. Broadly similar Rs using the same technology can show vastly different results here, some are noisy, others quiet as a mouse.
Inductance can be a problem with some wirewounds. Capacitance exists but isn't significant in audio applications.