gpio: ath79: Make the driver removable

As we now allow the driver to be built as a module it should be
removable.

Signed-off-by: Alban Bedel <albeu@free.fr>
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>