One of the pics shows them on a raft during a storm. Maybe they were blown off-course and washed up on the north shore because of that? Or they've got a pretty poor sense of direction - they set out to go back south, but ended up north! Since Isola is a magic island, compasses probably don't work.