I have a builder that has been building since I started the game yesterday and still have not master that skill, but I have 2 villagers that have master in science that started later then the builder.

Builders and farmers take longer to reach master than scientists, generally. A builder has to walk long distances to pick up a piece of wood or boulder then back to the construction site. When he gets there he may not always succeed. It is similar with the farmers; they have to walk back and forth to the ocean or the coconuts (the crops aren't so far, of course). A scientist, on the other hand, just has to stand at the table, with occasional forays into the scientists' hut or to move along the table.

Doctors are hard to get to master level as well; and they seem to tend to like to enjoy the pond.