Skip to content

MLBEDSW-10872: MLCE: Fix LUT table for RSQRT

Johan Alfvén requested to merge MLBEDSW-10872 into main
  • RSQRT operator caused an output mismatch
  • Missing branch for handling LUT index 0
  • Values near zero (index 0) should map to max output value, consistent with the reference implementation

Change-Id: Iba4cc6c49281fc749398b05c2edbdcd70caa34d4 Signed-off-by: Johan Alfvén johan.alfven@arm.com

Merge request reports