In the World of Saṃsāra there are a rare class of portals that only work when walked into from a certain direction. These can be two ways so if someone turns around and walks back, they might return to the World of Saṃsāra from which they came. These Fae Portals are normally invisible to all know forms of sight. They are normally in out-of-the-way places but if they are in busy areas people will sometimes put fences around them to keep the unaware from disappearing. This can have unattended consequences when people in a local area use them to get rid of those they do not want around.
Note: not all realms require a portal to be accessible. They might just be a local area, or an area marked on a map.