Magnifica Humanitas §198

¹⁹⁸ Sometimes there is talk of “artificial moral agents,” as if machines were able to distinguish between right and wrong with greater consistency than a human being. Yet moral judgment cannot be reduced to calculation, for it involves conscience, personal responsibility and the recognition of the other as a person. Therefore, it is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems. No algorithm can make war morally acceptable. AI does not remove the intrinsic inhumanity of conflict; indeed it can only bring about conflict more quickly and render it more impersonal, lowering the threshold for resorting to violence, transforming defense into threat prediction and thus reducing victims to data. In this way, it will accustom us to the idea that violence is inevitable and needs only to be optimized. This does not diminish the importance of instilling, as far as possible, values and sound judgment into the artificial systems we build, so that they can contribute to a moral ecosystem in which humans are better able to listen to their own consciences, as well as allowing AI models to establish appropriate boundaries.