The causes for which a pastor can be removed legitimately from his parish are especially the following: 1/ a manner of acting which brings grave detriment or disturbance to ecclesiastical communion; 2/ ineptitude or a permanent infirmity of mind or body which renders the pastor unable to fulfill his functions usefully; 3/ loss of a good reputation among upright and responsible parishioners or an aversion to the pastor which it appears will not cease in a brief time; 4/ grave neglect or violation of parochial duties which persists after a warning; 5/ poor administration of temporal affairs with grave damage to the Church whenever another remedy to this harm cannot be found.
Source: Code of Canon Law (Vatican.va)