Baltic Way 2019 · Problem 20

No such numbers $x$ can ever exist. To see this, let us recall that when $x$ is an integer, the right-hand side of the equation is always $\equiv 0 \pmod{3}$ or $1 \pmod{3}$. We recall also that when $x$ and $y$ are integers, we have

$$P(x) \equiv P(y) \pmod{3}, \quad \text{whenever} \quad x \equiv y \pmod{3}.$$

Therefore

$$P(x) \equiv P(-3) \equiv -40 \equiv 2 \pmod{3} \quad \text{when} \quad x \equiv 0 \pmod{3},$$

and similarly

$$P(x) \equiv P(-5) \equiv -156 \equiv 0 \pmod{3} \quad \text{when} \quad x \equiv 1 \pmod{3},$$

and

$$P(x) \equiv P(-1) \equiv -4 \equiv 2 \pmod{3}, \quad \text{when} \quad x \equiv 2 \pmod{3}.$$

In particular, $P(x)$ is always $\equiv 0 \pmod{3}$ or $\equiv 2 \pmod{3}$, and $P(P(x))$ is always $\equiv 2 \pmod{3}$. Therefore we always have $P(P(x)) \not\equiv x^2 \pmod{3}$ for every integer $x$.