As, then, this act of self-restraint, which in appearance is one and the same, is found in fact to be different in different persons, according to the principles and motives which lead to it; so in the same way with those who cannot allow in the worship of the Divine Being altars, or temples, or images. The Scythians, the Nomadic Libyans, the godless Seres, and the Persians, agree in this with the Christians and Jews, but they are actuated by very different principles. For none of these former abhor altars and images on the ground that they are afraid of degrading the worship of God, and reducing it to the worship of material things wrought by the hands of men. Neither do they object to them from a belief that the demons choose certain forms and places, whether because they are detained there by virtue of certain charms, or because for some other possible reason they have selected these haunts, where they may pursue their criminal pleasures, in partaking of the smoke of sacrificial victims.
But Christians and Jews have regard to this command, “You shall fear the Lord your God, and serve Him alone;” and this other, “You shall have no other gods before Me: you shall not make unto you any graven image, or any likeness of anything that is in heaven above, or that is in the earth beneath, or that is in the water under the earth: you shall not bow down yourself to them, nor serve them;” and again, “You shall worship the Lord your God, and Him only shall you serve.” It is in consideration of these and many other such commands, that they not only avoid temples, altars, and images, but are ready to suffer death when it is necessary, rather than debase by any such impiety the conception which they have of the Most High God.
Source: Against Celsus (New Advent)