avr32: Fix cast warning in board.c

The frame buffer pointer in global_data is not a pointer, so we should
remove these casts.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andreas Bießmann <andreas.devel@googlemail.com>