I am not allowed to have people come to my personal property for business as per the local homeowners association.
Yeah, I don't want to offend anyone, it's just a policy.
Problem is, people take it personally.
Another reason is I just don't want people coming to my personal property.
No offense taken here.

Where I live, it's an apartment, so in addition to the rules of the complex, AND of the city, I'm not supposed to have customers come over for business purposes AND my office cannot have a direct door to the outside either. So, that leaves phone, Internet, their place, or even McDonald's (LOL... buy them and myself a drink... business expense... as long as we conduct business.) to conduct business. Now, friends they don't have rules on, other than that any complex rule they break, I'm responsible for.
So, as far as visitors go, well, it's your property as long as the taxes are kept up (otherwise the Government decides it's THEIR property instead,) you have every right to say John Q Public NOT welcome here. Yes people don't understand, and take things personally and get hurt. That's life. Better they know up-front in many cases, but that isn't always possible.
You do what you have to do to keep the HOA gestapo from getting mad at you also. Personally, I'll avoid HOA controlled property when I finally get around to buying, as if it were the plague (if at all possible) because as I see it: I bought the place, I should be able to paint it florescent yellow with hot-pink polka-dots on one half, and all primary-color plaid on the other half if I have (lost my mind AND have) the urge to do so. I should be able to add a pool, run power tools, park my car in the drive instead of the garage, etc. if I want to without some HOA committee getting their panties in a bunch over it.