Yes, this can happen if enemies push you out of the borders, it's kind of a problem with unity itself. I will write this bug down and try to find a solution to this. The most easiest would be just to spawn the player back in the scene when falling out of the map.