clk: core: remove powerpc special handling

Now that the powerpc clocks are properly marked as big endian, we can
remove the special handling for PowerPC.

Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
Signed-off-by: Stephen Boyd <sboyd@kernel.org>