FYI, EVERY time I went to the fishfolk, I managed to avoid being tagged on the way there, but couldn't avoid it on the way back. Guess I'll have to try again now with my archer, but I think there's something that prevents you from going back the exact same way you came.