15 Let me add that our monks are not to be deterred from their resolution by you with your viper's tongue and savage bite. Your argument respecting them runs thus: If all men were to seclude themselves and live in solitude, who is there to frequent the churches? Who will remain to win those engaged in secular pursuits? Who will be able to urge sinners to virtuous conduct? Similarly, if all were as silly as you, who could be wise? And, to follow out your argument, virginity would not deserve our approbation.
For if all were virgins, we should have no marriages; the race would perish; infants would not cry in their cradles; midwives would lose their pay and turn beggars; and Dormitantius, all alone and shrivelled up with cold, would lie awake in his bed. The truth is, virtue is a rare thing and not eagerly sought after by the many. Would that all were as the few of whom it is said: “Many are called, few are chosen.” The prison would be empty. But, indeed, a monk's function is not to teach, but to lament; to mourn either for himself or for the world, and with terror to anticipate our Lord's advent.
Knowing his own weakness and the frailty of the vessel which he carries, he is afraid of stumbling, lest he strike against something, and it fall and be broken. Hence he shuns the sight of women, and particularly of young women, and so far chastens himself as to dread even what is safe.
Source: Against Vigilantius (New Advent)