This reverts commit 562f8df1.

Note: Even un-reverting this patch couldn't works as expected, based
on the latest testing from Heiko Schocher.

Signed-off-by: Jagannadha Sutradharudu Teki <jagannadh.teki@gmail.com>
Cc: Heiko Schocher <hs@denx.de>