This one last set of delay configuration registers was not properly
zeroed out originally, fix it and zero them out.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Dinh Nguyen <dinguyen@opensource.altera.com>
Cc: Chin Liang See <clsee@altera.com>