Github actions jobs have been plagued recently by failure to install
packages using apt due to flaky mirrors.

Make apt retry a number of times before giving up. Also add the official
Ubuntu mirrors as a fallback for azure ones.

Kajetan Puchalski authored Jan 11, 2023 and Douglas Raillard committed Jan 11, 2023

FIX

Adjust doc comments for df_tasks_states and related functions to clarify
that 'cpu' corresponds to the CPU on which the trace event was fired.
In the case of some events it does not necessarily correspond to where
the task was running previously.

Kajetan Puchalski authored Jan 10, 2023 and Douglas Raillard committed Jan 11, 2023

FEATURE

Latency-related functions such as df_latency_wakeup and
df_latency_preemption at the moment do not return any information about
which CPUs the latency ocurred on.

Add a 'cpu' column to df_latency_preemption along with 'cpu' &
'target_cpu' columns to df_latency_wakeup.

Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>

FIX

The recent trace-cmd decodes the function name even in raw output, so
adjust the dtype.

FEATURE

Update libtraceevent used by trace-cmd now that
https://bugzilla.kernel.org/show_bug.cgi?id=216896 has been solved.

FEATURE

Update to trace-cmd 3.1.5 so that it can handle trace.dat v7 format.

Update the year in man pages so that CI checks pass.

Since it's provided by Python >= 3.7 and we depend on that, there is no
reason to ship our own anymore.

Since Python 3.7, nullcontext is provided by contextlib directly so
there is no need to pull that from lisa.

Doc build currently generates warnings due to some dependencies
triggering warnings in other dependencies. In order to avoid making the
whole CI test fail because of that, remove -W from SPHINXOPTS.

FIX

Avoid pandas warning by forcing the current behavior.

FIX

Ignore some deprecation warnings inside some dependencies to avoid test
failure.

Dietmar Eggemann authored Dec 16, 2022 and Douglas Raillard committed Dec 19, 2022

UTIL_AVG_UNCHANGED is an internal util_est state bit and should not be
exposed via util_est.enqueued in the sched_util_est_se trace event.

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>

FIX

Recognize typing.Self as a legitimate annotation.

FIX

Avoid hitting this warning:

DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13

Add Python 3.11 to the tested versions

Add more debug logging to debug rt-app calibration issue.

FEATURE

Allow setting the --log option of rt-app.

Also set the level to "debug" while calibrating.

FIX

Fix CI by ignoring deprecation warnings triggered internally by
Jupyterlab/IPython themselves.

Dietmar Eggemann authored Nov 14, 2022 and Douglas Raillard committed Nov 14, 2022

Occasionally the load_tracking invariance tests
[load\|util]_[correctness\|behaviour] fail on rb5 little/medium CPUs
on low frequencies.
The root cause is wrong rt-app calibration for these CPUs/frequencies.
The reason for the rt-app calibration issue is not known at the moment.
Add an assert in _get_rtapp_profile() of class InvarianceItemBase to
catch the issue earlier.

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>

FIX

Do not error if a class does not define __init__.

FEATURE

Recognize PEP 673 Self return annotation on classmethods as being a
factory class method.

FIX

Older kernels do not have this folder and also do not suffer from the
broken sha1 in the /sys/kheaders.tar.xz headers, so ignore them.

Patrick Bellasi authored Nov 04, 2022 and Douglas Raillard committed Nov 04, 2022

FIX

A Systrace generated by the Perfetto's traceconv tool has records with this
format:
```
    <idle>-0     (-----) [000] .... 608397.919018: sched_wakeup: comm=traced pid=403293 prio=120 target_cpu=000
 ```
which timestamp gets parsed as `7.919018`, i.e. by ignoring all the digits but
one decimal.

The regexp also picks up all the spaces before the task name, by ending up with
duplicated names when the same task name is parsed also (without space) form an
event field (e.g. a `sched_switch`'s `next_comm`)

Fix the timestamp parsing by ensuring that, after the `__cpu` filed has been
parsed, we skip all and only the non digit chars following it.
Fix the command name parsing by ensuring we discard all the space before the
name start.

FIX

Headers found at /sys/kheaders.tar.xz generated when CONFIG_IKHEADERS=y
contain a broken version of the include/linux/atomic/*. headers. They
embed a sha1sum of the file on their last line, but
kernel/gen_kheaders.sh strips some multiline comments from the file,
changing the SHA1.

Work around the issue by updating the SHA1 in the file.

FIX

Disable Tabulator pagination as it is now fast enough to load all the
data. This allows scrolling when clicking on the plot to work correctly.

FIX

Fix scrolling in the table when clicking on the plot. Instead of making
the plot disappear, the corresponding row is selected in the table.

Note that the table will not change page automatically.

FIX

Ensure hv.extension() is done after all imports, otherwise plots are not
displayed in some circumstances:

https://github.com/holoviz/holoviews/issues/5491

FEATURE

Pass path to module and kernel path instead of relying on Kbuild's
$(src) and $(srctree) variables, as they seem to be broken on some
android's kernel.

FEATURE

Expose devlib's max_async Target parameter. It is used to limit the
max number of in-flight async commands at any point in time.

FIX

Update HRTxtTraceParser skeleton regex to allow for a TGID field, which
is part of recent systrace files.

Pierre Gondois authored Oct 14, 2022 and Douglas Raillard committed Oct 14, 2022

commit 487e1ac4 ("lisa.tests.scheduler.eas_behaviour:
Add big-task-duty-cycle platform parameter")
modified the prototype of _get_rtapp_profile(), but an implementation
was left with the initial prototype, triggering errors. Fix that.

Reported-by: Deepak Kumar Mishra <deepakkumar.mishra@arm.com>
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

Pierre Gondois authored Oct 07, 2022 and Douglas Raillard committed Oct 13, 2022

On a rb5, the first OPPs of the little CPUs are less energy
efficient than the first OPPs of the big CPUs:
little CPUs:
freq=300000  power=28864  capa=59  power/capa=489.22
freq=403200  power=38774  capa=79  power/capa=490.81
freq=480000  power=46183  capa=95  power/capa=486.13
freq=576000  power=55419  capa=114 power/capa=486.13
freq=652800  power=62732  capa=129 power/capa=486.29
freq=748800  power=71968  capa=148 power/capa=486.27
freq=825600  power=82719  capa=163 power/capa=507.47 <- [3]
freq=902400  power=94168  capa=178 power/capa=529.03
freq=979200  power=113400 capa=194 power/capa=584.53
freq=1056000 power=114700 capa=209 power/capa=548.80
freq=1132800 power=126190 capa=224 power/capa=563.34
[...]
BIG CPUs:
freq=825600 power=133030  capa=301 power/capa=441.96
freq=902400 power=151283  capa=329 power/capa=459.82
freq=979200 power=168490  capa=357 power/capa=471.96 <- [1]
freq=1056000 power=188799 capa=385 power/capa=490.38 <- [2]
freq=1209600 power=224387 capa=441 power/capa=508.81
freq=1286400 power=244608 capa=469 power/capa=521.55
freq=1363200 power=275303 capa=497 power/capa=553.92
freq=1459200 power=305249 capa=533 power/capa=572.69
freq=1536000 power=340314 capa=561 power/capa=606.62
freq=1612800 power=381776 capa=589 power/capa=648.17
[...]

When running the TwoBigThreeSmall test, the big task has a
utilization such as the task cannot fit on the little CPUs.
util~=290 so that 290 * 1.2 = 348. The capacity of the little CPUs
is 350. The small task will run at ~50% of the maximum capacity of
the little CPUs, so util~=174.
The big CPUs will run at the OPP [1], which is more energy efficient
than all the little CPUs' OPPs. Now if the big CPU utilization jumps
to [2], the little CPUs' OPPs below [3] become more energy efficient.

Thus, it appears that for the rb5, the utilization chosen for the
big tasks makes the big CPUs either attractive or repulsive for the
small tasks, and this for a range of 20 util. This makes the
TwoBigThreeSmall test for rb5 unstable.

Add a platform specific 'big-task-duty-cycle' parameter to handle
this specific test case. This parameter allows to choose a higher
utilization for the big tasks, making the big CPUs less attractive
for small tasks.

Suggested-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>

FIX

Using a trace window can remove all activations for a given task. That
leads to <NA> values in dataframes, that bokeh chokes on. Avoid creating
such <NA> values in the first place fixes that.