Factor out (afi_clocks - 1) into a single place instead of having it
all around the function.

Signed-off-by: Marek Vasut <marex@denx.de>