This section adds a comment stating that running
operations outside the constraints of level=8k
may lead to undefined behaviour.

Change-Id: I5f6ea12ecce96ce6d8a1cb7a14de7e0b73c40bb9
Signed-off-by: Luke Hutton <luke.hutton@arm.com>

Luke Hutton authored Jul 10, 2025 and Jeremy Johnson committed Jul 14, 2025

This commit removes some ambiguity in the README by
merging duplicate sections.

Signed-off-by: Luke Hutton <luke.hutton@arm.com>
Change-Id: If5eb9e4352b0f98754a8fd64771c807791813ea4

Introduced a "Versioning" section describing the Calendar
Versioning (CalVer) scheme used by the project.

Change-Id: Ie7b5634c0bb4c39bde8fc9de53ba53501b7cbf0d
Signed-off-by: Peng Sun <peng.sun@arm.com>

Philip742 authored Jun 19, 2025 and Jeremy Johnson committed Jun 23, 2025

Note does not cover test commands

Signed-off-by: Philip Wilkinson <philip.wilkinson@arm.com>
Change-Id: Ib1cb340290cf5c663e00185aaaaad68388bb4477

Deeptanshu Sekhri authored Apr 15, 2025 and Jeremy Johnson committed Jun 02, 2025

Improved clarity and structure for the ModelRunner
Python bindings section. Added build instructions with usage example.

Signed-off-by: Deeptanshu Sekhri <deeptanshu.sekhri@arm.com>
Change-Id: I8176f83cc8a92260dea9b4cab1b5d5c6fa7ec0f0

Justin Ngo authored May 16, 2025 and Eric Kunze committed May 19, 2025

Unit test test suite for PyTorch utilizing FxImporter to lower PyTorch
graphs to Torch-MLIR then to TOSA IR. Current infrastructure is mostly
based on the existing TF/TFL frameworks test suite.

This is the initial work for the Torch unit test suite. The end goal
is to add more Torch unit tests for a more robust test suite.

Signed-off-by: Justin Ngo <justin.ngo@arm.com>
Change-Id: I829ddf708556488166bc2bfdd4b52a1cfb4fc34d

- update the version of tools and packages
- add high-level featue description
- create verif/README.md to introduce detailed CTS information

Change-Id: Id80810a700ddf8040954857813cce7464700f45b
Signed-off-by: TatWai Chong <tatwai.chong@arm.com>

Eirene Vlassi Pandi authored Apr 02, 2025 and Eric Kunze committed May 02, 2025

- Added documentation on load_npy and save_npy behavior.
- Included usage examples showing support for both string and DType
 inputs.

Signed-off-by: Eirene Vlassi Pandi <eirene.vlassipandi@arm.com>
Change-Id: I84ec2fd385109857633e8da3eaddea03acab9d16

Make tosa_verif_* tools assume they are in the ref_model repo with a
local build directory.
Update README with profile info, and updated commands to match
assumption of local build directory.
Add pytests for model file locations util.
Fix paths for non-build items with build_sub_dirs.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I70cbcb64423b6782650166507016500268a2496d

Remove exptected_return_code & expected_result_files from desc.json
Clean up readme about desc.json
Add "terminate_early" to desc.json schema and add override warning

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib7f4c5365ccf58b3c6d1c7e11c4af6817931e6a2

Michael Platings authored Apr 11, 2025 and Jeremy Johnson committed Apr 16, 2025

Updated to use new flatc build instruction and location

Signed-off-by: Michael Platings <michael.platings@arm.com>
Change-Id: I2958db32fec4070508dc61ee85decffee586be6f

Michael Platings authored Feb 20, 2025 and Eric Kunze committed Feb 20, 2025

Previously the documentation implied that a release build would be
created by default. The default is toolchain-specific, hence the
recommendation in the CMake documentation:
"Users should always explicitly specify the build type"

Signed-off-by: Michael Platings <michael.platings@arm.com>
Change-Id: I0b5b793600a548f7b7dd540d588f212f78aae5af

Eirene Vlassi Pandi authored Feb 13, 2025 and Eric Kunze committed Feb 19, 2025

Remove instructions related to handling fp8 from README as
this no longer an issue.

Signed-off-by: Eirene Vlassi Pandi <eirene.vlassipandi@arm.com>
Change-Id: I4a35133ea8712177f4c321edf42e875674d3095a

Won Jong Jeon authored Nov 04, 2024 and Won Jong Jeon committed Nov 06, 2024

Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: I50b67bcc338cbc9ba5a11e95ecb96fa94f9a3105

Victor Anaehobi authored Sep 16, 2024 and Eric Kunze committed Sep 25, 2024

Add pre-commit checks section

Change-Id: I3c613bd25428e1fc0e5e8c0ca0906f20c40aac8b

Peng Sun authored Sep 10, 2024 and Eric Kunze committed Sep 12, 2024

  * add cmd boolean option: --terminate_early
  * add JSON test descriptor field: terminate_early

Signed-off-by: Peng Sun <peng.sun@arm.com>
Change-Id: If3f09cdbf66eb450aaf494ed0d6b0dab2a2d9723

Kaushik Varadharajan authored Jun 26, 2024 and Eric Kunze committed Jul 31, 2024

Installable with `pip install .` in the repository's root directory.

The Python package is built as an .so, which setuptools copies into
py_package/ and installs.

Signed-off-by: Kaushik Varadharajan <kaushik.varadharajan@arm.com>
Change-Id: I20030c02bab6ca0357b55422fc7066ac3a6f6546

Justin Ngo authored Jul 16, 2024 and Eric Kunze committed Jul 16, 2024

- Update cmake requirement in README
- Add cmake version check around CMP0135 policy flag

Signed-off-by: Justin Ngo <justin.ngo@arm.com>
Change-Id: I6d575ad484c5ad95ef6a119661074845b10e7936

Jeremy Johnson authored May 01, 2024 and Eric Kunze committed May 13, 2024

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I651ef963b6c14da58dc4cecd8bf4da8fe1a38417

Jeremy Johnson authored Mar 27, 2024 and Eric Kunze committed Mar 28, 2024

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I8cd95ed0ae3260eea785e4e0275961534cd1f0c5

Add --test-version option to tosa_verif_conformance_generator to
select the version for tests to output.
Add --output-type to allow json, binary or both files to be created
during conformance generation.
Fix passing schema_path to test runner.
Add explicit verify lib path arg to test runner.

Change-Id: I5f1ad137d713fca408a98470ea77bddf8916c5f3
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Jeremy Johnson authored Sep 04, 2023 and Eric Kunze committed Sep 07, 2023

Rename scripts utlities to tosa_* such as tosa_json2numpy

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie8d584e2afb189fb74cf96b39590c7c27444ba14

Jeremy Johnson authored Aug 02, 2023 and Eric Kunze committed Aug 07, 2023

* Add deprecation notice for conformance_tests repo
* Add deprecation notice for framework tests in conformance

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I92b17f4d6d993e47f30d7996a18fc13b934c4cdf

Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I7015de6989940a535a12b737432e5efa8304d9a3

Brings up to date with commonly tested configuration

Change-Id: Id1791f506e6d822d69ee962a9403a7092afeaaf6
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

"tosa_run_tests" -> "tosa_verif_run_tests"

Signed-off-by: Luke Hutton <luke.hutton@arm.com>
Change-Id: Iff8c4874af1dc3e203e1f49563cfa0fd1f373811

* Added IModelRunner interface using pimpl idiom, which allows a user to
  initialize, configure and run the model.
* Added unit tests for IModelRunner.
* Added doctest as third-party submodule.
* Added user options to specify paths for dependencies.
* Moved general func_config functions to separate utility, which removes
  cxxopts dependency.

Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com>
Change-Id: If42f1f82cd6dadf18911a48dcd5fa579b719aff2

Change-Id: Ia681aa0eb1d42fa31a4658badb376f7b32b2e338
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

Describes the proper way to build debug.

Change-Id: Ie2f888fc71f84af5382e7e8107eb2d3d93a3b056
Signed-off-by: Eric Kunze <eric.kunze@arm.com>

Change-Id: I5cb16ea1f47ee454f03a1d5182827e3fd9f49128
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

And fixes in tosa_verif_run_tests:
* support for no-color printing
* stop double printing of error messages on verbose
* differentiate result code pass from results check

Change-Id: I26e957013a8d18f7d3d3691067dfb778008a1eea
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>

Script converts framework and reference model unit tests into a
suitable format for including into a conformance suite.

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ida1ec8a0a7ea31fd3a3f62c4cb52d7cc2bf0b439

Rename to tosa_verif_run_tests to match build_tests
Improve output and system under test support
Improve xunit support
Add results checker
Add utilities json2numpy and json2fbbin
Add set of python tests
Update README.md

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca

Split generator and runner scripts
Add package setup
Add py-dev-env.sh/.bash to allow editing source files during dev
Update README.md with installation info

Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d

- Updated example tests, added missing npy files
- Updated README file with fields in desc.json

Change-Id: I5651ed5260866a8381adf620fb6dd6405cb2833b

Change-Id: Iac786eff96183938d2fd11cde9313c6e8e1270a5

- Remove Usage and Format
- Run black on verif/*.py scripts

Signed-off-by: Kevin Cheng <kevin.cheng@arm.com>
Change-Id: Ie81515891eb0039540f614894f4b6b0e0e78ba74

Change-Id: I2f8e7fa63e2ae40203e57d2cc8814bde3b312cb6
Signed-off-by: Eric Kunze <eric.kunze@arm.com>