5 But if any one wishes to know why the grace is given by water and not by a different element, let him take up the Divine Scriptures and he shall learn. For water is a grand thing, and the noblest of the four visible elements of the world. Heaven is the dwelling-place of Angels, but the heavens are from the waters: the earth is the place of men, but the earth is from the waters: and before the whole six days' formation of the things that were made, the Spirit of God moved upon the face of the water. The water was the beginning of the world, and Jordan the beginning of the Gospel tidings: for Israel deliverance from Pharaoh was through the sea, and for the world deliverance from sins by the washing of water with the word of God.
Where a covenant is made with any, there is water also. After the flood, a covenant was made with Noah: a covenant for Israel from Mount Sinai, but with water, and scarlet wool, and hyssop. Elias is taken up, but not apart from water: for first he crosses the Jordan, then in a chariot mounts the heaven. The high-priest is first washed, then offers incense; for Aaron first washed, then was made high-priest: for how could one who had not yet been purified by water pray for the rest? Also as a symbol of Baptism there was a laver set apart within the Tabernacle.
Source: Catechetical Lectures (New Advent)