Fix up issues in existing python scripts.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Id4adab404560c3129c66f31c21ff0ce148283c73

Rename to tosa_verif_run_tests to match build_tests
Improve output and system under test support
Improve xunit support
Add results checker
Add utilities json2numpy and json2fbbin
Add set of python tests
Update README.md

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca

Split generator and runner scripts
Add package setup
Add py-dev-env.sh/.bash to allow editing source files during dev
Update README.md with installation info

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d

When testing rank mismatch tests, could be accessing beyond the end
a tensor's shape.

Change-Id: I304463e50579d574d629abd13098017a26a1406b
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I267fa624241f60f1fa613e27e4fe46fc43768f8a

Change-Id: I33237ebfd946b9ec91352c2b0dc6298cc113cd77
Signed-off-by: Les Bell <les.bell@arm.com>

REDUCE ops - test generation now limited to ranks 1-4 and so
             WrongRank tests created
SCATTER/GATHER ops - enable WrongRank testing

Change-Id: I0909573e0edabddbad8764e7542d654967a5c8b9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Jeremy Johnson authored Nov 16, 2021 and Eric Kunze committed Nov 18, 2021

TRANSPOSE - remove WrongRank testing as no limit in spec
RESIZE - make floating point test naming same format as int
       - fix MaxDimExceeeded by not restricting dims and
         fix max dim check to match spec
       - catch stride less than/equal zero and regen
PAD - remove zero point testing as not in spec

Change-Id: I4eb0b78d7f577469d202866d95833e7d5fd2f2a8
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Signed-off-by: Les Bell <les.bell@arm.com>
Change-Id: I68a13e1b337b1afc2ab5e0edcffda2b4b0cecdda

- update example .tosa

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I1eb9f60d33de494408963e6ac16c1566254200f1

- test like [1] + [2] = [1] should be treated as invalid test
- modify matchRankShape() function so it allows size 1 only on the source tensor but not target tensor

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I6bbb6a63dc1143712e7eef736a991cac419b009e

Kevin Cheng authored Nov 10, 2021 and Eric Kunze committed Nov 11, 2021

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I49d498dd3d4c069d8d1db07310f939268b9df4b7

Add RankMismatch and DimensionMismatch support for SELECT
Update RankMismatch ops to also support DimensionMismatch
Update POW op to have proper broadcast testing
A few other broadcastable ops missing Rank/Dimension testing

Change-Id: I6566f45a7a0db4f9f008456ea7a8e23d4192f4f9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I9ed3d8971a133b4cbb2cf7d827f4e69d55dee246

Matthew Haddon authored Oct 15, 2021 and Eric Kunze committed Nov 09, 2021

 * Transpose WrongRank tests now use ranks 7, 8
 * Concat ERROR_IF checks now test for inaccurate summation
of output shape tensor dimension

Change-Id: If32f43a4dbd872d0ef7625fa3d4969c863a11b8c
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Signed-off-by: Les Bell <les.bell@arm.com>

Matthew Haddon authored Oct 14, 2021 and Eric Kunze committed Nov 09, 2021

Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Signed-off-by: Les Bell <les.bell@arm.com>
Change-Id: Ie6c8c8653874f9eed6007a54a3ad526601a4a669

Matthew Haddon authored Oct 13, 2021 and Eric Kunze committed Nov 09, 2021

 * Operators implemented: sigmoid, tanh, arthmetic_right_shift,
mul, table, select, equal, greater, greater_equal, concat, reverse,
tile, scatter, gather, case
 * Note that over the course of implementation some specific ERROR_IF
checks have been added for some of the above operators

Change-Id: I80595e6eb9a3e5efd1cc6fd7aa28bbc2dd614980
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Signed-off-by: Les Bell <les.bell@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

- update thirdparty/serialization_lib
- avoid building uncessary targets in submodule

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I34698014b63e4d2d8b78011c001c85882014007c

Change-Id: I1de45a21779ec014a0fda2c971dbc8f91df45bdb
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

Update to the corresponding serialization library

Change-Id: I323bf306d409b51bbf53447927953edd0fd79983
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

- TRANSPOSE: move perm attribute check to compile-time checker
- TABLE: add output type checker

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I834a5f290fbc384ef339b624060e6e5c77072c36

- MATMUL: only check a_zp/b_zp valid when this->qinfo exists
- Fix typo in debug message

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I2cedcb25e4f57fcaec2caa1b850ea1232a023340

Allow random shift values rather than just 11
Check for values that cause unpredictable behaviour

Change-Id: I508c9d0982808ac03b8400d47ad9445d6afb1b79
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

- update serialization_lib hash
- PAD:
    1. make padding as an attribute instead of tensor.
    2. add pad_const_int (for non-float type) / pad_const_fp (for float type)
- TRANSPOSE: make perm as an attribute instead of tensor
- TABLE: make table as attribute instead of tensor
- update examples/ tests

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: Iddc446db4b356ba2f36ea4a79b7220b9cfc2aa4e

- update serialization_lib hash
- check model version against serializer version
- add "-v" to command line option to print out model version

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I554cf0fdead22e7e5f91d2e98831459286c40a2d

Jeremy Johnson authored Oct 20, 2021 and Eric Kunze committed Oct 25, 2021

Change-Id: Idb36b1f1c0d78ec101c168865a9c8d03221b4c84
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Kevin Cheng authored Oct 20, 2021 and Eric Kunze committed Oct 21, 2021

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I37fa4fedbeb64c1e147f2b43f45cf8e25854256c

Jeremy Johnson authored Oct 19, 2021 and Eric Kunze committed Oct 21, 2021

Change output size error_if check in ref model to match specification.
Remove size check from test generation as output shape is always
correctly calculated.

Change-Id: I5be64f31e6448b47e80fc0a4af11bb312f366a26
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: Ifd771171904d1e5a9db3ea1cae3ac9017e971c8c

- Also delay tensor allocation after operator being validated
  ERROR_IF can be caught first before 0 or negative dimension set the graph_status to UNPREDICTABLE
- Rescale, Argmax, FullyConnected, Matmul, Pad, Reshape, Slice, Transpose, Clamp, Concat, Equal, Greater, GreaterEqual, Table

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I4e1b3e5794fe195ce1a37e28443ae584645a3b91

Matthew Haddon authored Oct 11, 2021 and Eric Kunze committed Oct 18, 2021

Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Change-Id: I659337aadfd0498bf88a95737f69c51efec797cc

Matthew Haddon authored Oct 11, 2021 and Eric Kunze committed Oct 18, 2021

Change-Id: I75f2a4ab6790dcbdfaec064f42f601d8f44da70b
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>

Matthew Haddon authored Oct 08, 2021 and Eric Kunze committed Oct 18, 2021

 * Negative tests for rescale op added

Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Change-Id: I70aead1c6a67f159c7b7c9a05f7d5f0b92521584

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: Idd12aab4d52365f3137a403d362790abc16e66d0

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: I0947f136e768605f997fcaf74c6b9a7e62e748a4

Les Bell authored Oct 11, 2021 and Eric Kunze committed Oct 12, 2021

Change-Id: Id1c2d2a41bdfff6b3fe45f79bf99abd0b5beb704
Signed-off-by: Les Bell <les.bell@arm.com>

 * Negative tests for ERROR_IFs given in spec
 * Constrict dimension size of latter ranks if
rank is larger than 4

Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
Change-Id: Iffea1874e876dba83c8a7c63049283bf7b3ba74b

Matthew Haddon authored Sep 29, 2021 and Eric Kunze committed Oct 11, 2021

 * Negative testing support added for the following operators:
reduce_all, reduce_any, reduce_max, reduce_min, reduce_product, reduce_sum

Change-Id: Ife4f0307b681510e8910a23a39efc127c82ae120
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>

Jeremy Johnson authored Oct 07, 2021 and Eric Kunze committed Oct 11, 2021

Change-Id: Ic2aebe40b5cce61d4576a64f4f48ff87b36475c2
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

 * Added negative testing support for the following operators:
abs, bitwise_not, ceil, clz, exp, floor, log, logical_not, negate, reciprocal, rsqrt

Change-Id: Icc6f146c6407502520330678420951749ba2a9ef
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>