I think the saying is thus because mothers are more of the 'nurturing' type. That mode of thought is a bit outdated, but still it's more often true than not, even today, that mothers are more often in that role than fathers.
Outside of the scope of a joke, yes, a father had dang well better love his child, if he expects to deserve the title of 'Father'.