Add --test-version option to tosa_verif_conformance_generator to
select the version for tests to output.
Add --output-type to allow json, binary or both files to be created
during conformance generation.
Fix passing schema_path to test runner.
Add explicit verify lib path arg to test runner.

Change-Id: I5f1ad137d713fca408a98470ea77bddf8916c5f3
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Fixed PrimitiveGenerator starting point and added test sets 1-5.
Fixed verify_test reduce_product missing data_type.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Iaf080ce5c1adb5819f70d1a285d04baa36016092

Jeremy Johnson authored Oct 04, 2023 and Eric Kunze committed Oct 04, 2023

Add support for dot-product MatMul - test set 0

Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com>
Change-Id: Ifd15b42570014b634f59c94a1fd1cd56bac79ea4
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Jack Frankland authored Sep 20, 2023 and Eric Kunze committed Oct 03, 2023

* Add verifiers to validate the result of a reduce produce operation.
* Add test cases for the new validator.

Change-Id: I666d1a67f498e7893e0f224bc5408a4134f2ef6c
Signed-off-by: Jack Frankland <jack.frankland@arm.com>

Updated to conformance generator to not generate tests with results for
compliance tests.
Updated test runner to run compliance mode version (precise & abs mode)
of reference model to create test results to use against SUT results.
Updated reference model to enable abs_mode on correct desc.json flags.
Updated test checker to support compliance checking using verifier lib.
Seperated color printing from test checker.

Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Make compliance meta-data data-type required for all.
Add data-type checking for all verifier modes.
Add initial enum support for new ROUND compliance mode.
Improve print out information from library.
Use numpy ctypes.data_as to get f16 support compared to ctypes_lib.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7

Jack Frankland authored Sep 13, 2023 and Jeremy Johnson committed Oct 02, 2023

Add a verifier to check two results are correct within a certain ULP
tolerance for IEEE-754 32-bit floating point values.

Add a test to check the ULP verifier is correct.

Signed-off-by: Jack Frankland <jack.frankland@arm.com>
Change-Id: Iaf43069f300999479d998e7837746b247ca5177e

Jack Frankland authored Sep 15, 2023 and Eric Kunze committed Sep 28, 2023

* Ignore build-debug (this is a common sub-directory for debug builds).
* Ignore .cache (this is a clangd LSP directory
* Ignore compile_commands (can be generated by CMake used by LSP
  clients)
* Update copyright date.

Signed-off-by: Jack Frankland <jack.frankland@arm.com>
Change-Id: I74a5bc8c03f51e2d534d4870b58d477e830f6f42

Jack Frankland authored Sep 13, 2023 and Eric Kunze committed Sep 28, 2023

Add a verifier to check that tensor results match exactly.
Add a unit test to check the behavior of this new verifier.

Change-Id: I9b80a6d57640fec67c6be80a97b3af484aeb935e
Signed-off-by: Jack Frankland <jack.frankland@arm.com>

Jeremy Johnson authored Sep 27, 2023 and Eric Kunze committed Sep 28, 2023

tosa_verif_conformance_generator
- Move to using ref-model-path instead of ref-model-dir
- Add schema-path and flatc-path
- Add model_files module to locate default places for files
convert2conformance
- Remove default paths
verifier
- Switch to using exact path of verifier library
tosa_verif_run_tests
- Use conformance model_files to locate defaults

Change-Id: Ieca3b164670e2a7dcb047743667cc4e8317daa97
Signed-off-by: Josef Malmström <josef.malmstrom@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Won Jong Jeon authored Sep 18, 2023 and Eric Kunze committed Sep 28, 2023

Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Iff77091e4a57f487431ffbf7ac1c89301a153c8b

Josef Malmstrom authored Sep 20, 2023 and Eric Kunze committed Sep 28, 2023

By default, setuptools_scm will guess the next version
and supply that as the version number when the package
is installed.

For example, this meant that installing
when tag "v0.60.2" was checked out would give package
version "0.60.3.dev8+gf5a8283.d20230920". With this
change, the package version would instead be
"0.60.2.post1.dev8+gad80c03".

Change-Id: I0bca9a6abec38e82839b52fe2933d2c0d14b5a84
Signed-off-by: Josef Malmström <josef.malmstrom@arm.com>

Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com>
Change-Id: I4cd818af0db5e6e8a96641246cd3263ba4878f56

Attempt to avoid API getting out of sync.

Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: Ic7b72c3f906e4a38cb26159bb67e9b1c4e22ca96

Jeremy Johnson authored Sep 14, 2023 and Eric Kunze committed Sep 18, 2023

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I3620799a01784865b3de1407e70e02582966f6d1

MAX_SCALE for the EIGHTK level is 256 now.

Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: I6f3a3623d18ff238888d994c2a4129091f5a92fe

Tai Ly authored Sep 07, 2023 and Eric Kunze committed Sep 15, 2023

Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2

Include 0.90 draft serialization library

Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: I290360bfd1c147507e092868b5546c9be08682c2

Grant Watson authored Sep 12, 2023 and Eric Kunze committed Sep 14, 2023

- Updates TOSA specification to the latest version
- Updates generate_api.py to generate the operator API correctly
for ops with additional tensor inputs.
- Removes default arguments for func_debug and func_config to make
the API C compliant again.
- Updates model_runner_tests.cpp for operators that have changed.
- Adds a unit test for the Tile operator to check that generated code
for additional tensor inputs works correctly.

Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: I1e26065c6ed333b2ca4b3da39972d30f896fa6e5

Jeremy Johnson authored Sep 12, 2023 and Eric Kunze committed Sep 13, 2023

Build steps to create tosa_reference_verify_lib.so
New python class to handle loading and calling verifier with pytests.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Id42c71c398ed7364742b4a039977412ef3081c4a

Jeremy Johnson authored Sep 12, 2023 and Eric Kunze committed Sep 13, 2023

Increase size of tensor dimensions to meet compliance size needs.
Also improve warning output about this size requirement.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I9dc2b8dab05a1bac0083cdca9d99845c287d3ce9

TatWai Chong authored Aug 31, 2023 and Eric Kunze committed Sep 13, 2023

Align with the change in the spec.
Define idiv_floor to give equivalent behavior to the floating-point
floor function for image coordinate calculation.

Change-Id: Ice06d354d58b1bb0dfedab55c9cc9eac1598b50c
Signed-off-by: TatWai Chong <tatwai.chong@arm.com>

Incorrectly used "tag" in test.desc instead of "tags"
Incorrectly selected all tests rather than just all positive
tests depending on operator selection option.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib257bc57accf745beedfa4a3b2ece9539c0972c2

input_unsigned and output_unsigned were added to the specification.

Older TOSA files with uint data types are still supported.

Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: I125886ffc92975d99971e56e2075dd5d96bdbdc4

Jeremy Johnson authored Sep 05, 2023 and Eric Kunze committed Sep 07, 2023

Add initial support for compliance and lazy data-gen meta data
added to desc.json for MATMUL.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64

Jeremy Johnson authored Sep 04, 2023 and Eric Kunze committed Sep 07, 2023

Rename scripts utlities to tosa_* such as tosa_json2numpy

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie8d584e2afb189fb74cf96b39590c7c27444ba14

Georgios Pinitas authored Aug 22, 2023 and Eric Kunze committed Sep 07, 2023

Change verifier API to consume verification configuration in a JSON
format and enable appropriate validation to be performed within the
verifier code in the reference model.
Also update to latest spec changes for main compliance but not yet
including bias support.

Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I0ceaa1714dd041b00b5b29cd937c8f05e701bc4c

Serialization_lib change: https://eu-gerrit-1.euhpc.arm.com/c/mltech/mirrors/tosa/serialization_lib/+/542261



Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I59752a3ba7981c356634b911160cedf30fb2599e

Jack Frankland authored Aug 12, 2023 and Eric Kunze committed Sep 06, 2023

MacOS does not have the `sys/prctl.h` header. This patch conditionally
excludes `prctl.h` on MacOS and usage of its functionality in the same
way we currently do for windows.

Signed-off-by: Jack Frankland <jack.frankland@arm.com>
Change-Id: Ic56ec358552126b3a10827d9c52db388a8acc214

Grant Watson authored Aug 28, 2023 and Eric Kunze committed Sep 05, 2023

Updates the generate_api.py script and associated
templates to allow func_config and debug_config
to be passed when running individual operators
on the API.

This will allow us, for example, to set precise_mode
and abs_mode when running individual operators.

Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: Ia3e7ffc146f876daa307558433177c68285843b7

In tosa_test_gen.py, change output_shape for dim operators to [] instead of [1]

Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I8e7901e02831e9a467bce17e55e9379cb5f1b739

Jeremy Johnson authored Aug 22, 2023 and Eric Kunze committed Aug 30, 2023

Compress flatbuffer JSON files for conformance to allow while_loop
tests to fit in the 30MB file size limit.
Do not include new SHAPE type in usable DTypes so that selection of
ERROR_IF tests are not changed.

Also enable strict json mode for flatbuffers.

Change-Id: Id89a9963244c34769a4662f9951adde0aa3334d9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

remove unnecessary change of tensor shape array for SHAPE data type

Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I8e6ca98448188a1c30f9d29c22b3194d327b17ac

Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Id5820033bb3cc98b5e284237a91b22ffd54a80e3

Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: I9504fa05b93417182a76aa74743f779db3036594

Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Iea11ee5d3d98773e9c5e9b827593c05afb41ce3b

The double quotation mark is not always appear in the assembly output
of operations. Remove it to easier capture general printing pattern.

Change-Id: I59301daa07abd905de7259f8ae527a74b1ce5a6c
Signed-off-by: TatWai Chong <tatwai.chong@arm.com>

add support for bias broadcasting for operators:
  - conv2d
  - conv3d
  - depthwise_conv2d
  - transpose_conv2d
  - fully_connected

could not add framework test for this because tf.nn.bias_add requires
bias size to match channel dimension.

manually tested reference model evaluation on tosa mlir with bias size
of 1

Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I70d29d231a63fc03b10e3006cbd6b16b53cca1f2

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I17227adfca203f353e51ae9297f7e01f8d7edbe9

Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: I22939e75a08349816a9c103eb7312edcec53855e