Background: AI has quietly sparked a reform in higher education, bringing convenience to teaching but also triggering a series of ethical challenges. Currently, most relevant research participants are students, university teachers have not been given sufficient attention. Objective: Our purpose was to engage in a systematic review of the literature to identify the ethical challenges of AI teaching assistants in higher education faced by university teachers and propose some corresponding solutions. Method: A comprehensive search was conducted on the Web of Science and Scopus databases to identify English articles related to AI teaching assistants and higher education. After systematic screening, a review was conducted on 24 articles. Then, we used the thematic induction method for analysis. Results and Conclusions: Analysis of these studies provides evidence of that AI teaching assistants brought benefits to higher education, but also found a series of ethical challenges. Through literature analysis, we have summarized these ethical challenges which most related to university teachers into five aspects: teacher-student relationship apathy, human teacher displacement, academic and education integrity, algorithmic bias, and digital divide. We found that in response to these ethical challenges, university teachers should continuously improve their relevant abilities and strengthen their supervision of themselves and students in using AI. Universities or higher education institutions should provide AI resources and training opportunities for teachers. Decision-makers and AI education product developers should also fulfill their ethical responsibilities.  All responsible parties should collaborate to creating a better educational environment to address the ethical challenges  of AI in higher education.