Update inexact round bounds and computation
The current version of the specification only allows output zero points
to be subtracted as integers, so the FP32 implementation was updated to
reflect that.
The error bound was fixed to now use the output before the zero point is
added to compute the error scale.
Signed-off-by:
Ian Tayler Lessa <ian.taylerlessa@arm.com>
Change-Id: I5d2834b22180137880ffa905567ff1722d61c060
Loading
Please register or sign in to comment