From my understanding, someone can have the virus and spread up to two weeks before showing any symptoms. If you are in a risk group (over 60 or underlying health issues) then it might not be a good idea.
Maybe discuss with the housekeepers about their risk and yours, maybe there can be a compromise?