Yeah that only works for a little while... You have to show him who is the parent and who is boss.
Kids learn associate there surroundings, and know what they can do in some places vs others. Even though you don't have him full time, if you teach/train him, he will remember.