It's not your imagination. That's actually been around since the original release of Village Sim. It has to do with the way that villagers' positions are determined when the game is launched, and is affected by the current occupation or status of each villager. There are several locations that seem to "attract" many, if not most, of the villagers at startup: the research table, the well, and the "sickroom" (to name a few).