There are a few things you can do, depending upon exactly how your amp & cabinet are built. You're just gonna have to consider these and ponder how much work you're ready to do.
I am assuming the speaker mounts from the inside of the amp. You could change to front-mounting it. You may have to destroy your baffle board and remake it.
If the baffle board screws into the rest of the cab, you could get some sort of 1/8-1/4" spacers and move the baffle board forward a bit. Of all the possibilities, that would probably be the easiest. Of course if the Baffle board is glued in..... then no go.
The second easiest thing would be if your grill cloth is on a lightweight frame that velcros to the front of the amp. Then you enlarge the speaker hole and change to front mount, most likely can use the grill cloth frame as it. Fairly painless.
Elongating the speaker mounting holes and/or moving the screws, as Shooter suggested, is a possibility. There will be plenty of opportunities to trash the grille cloth in this operation unless you are very careful.
Worst case, the current baffle board is glued in and you have to hack out the baffle board, leave stubs along the sides, and make a new one with the speaker moved off-axis to a different location. That will probably cause problems with the new speaker mounted either too far forward or towards the rear. AND force you to make a new baffle board. Not pretty.