ARM: uniphier: replace <common.h> with <linux/delay.h> where possible

The <common.h> includes too many headers.  Actually, these files
needed to include it for udelay() declaration.  Now we can replace
it with <linux/delay.h> thanks to commit 5bc516ed ("delay:
collect {m, n, u}delay declarations to include/linux/delay.h").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>