Docs › Porting devices › Porting the modules › Testing your PAL port

Testing your PAL port

Two sets of tests are provided for verification and testing of a new PAL port:

PAL porting tests.
Device Management end-to-end tests.

PAL porting tests provide module-level verification to allow porting in a modular way.

When the porting tests pass and mbed-cloud-client-example can connect to Device Management, you can verify further functionality by running the Device Management E2E tests. The tests verify features such as CoAP GET, PUT, POST and Firmware update.

Detailed information on the E2E tests and their execution is available in the E2E tests README document.

This document focuses on the PAL porting tests.

PAL test overview

PAL provides a test suite to test the service API, which is used by the services running on top of PAL.

The tests are written in Unity format. All modules have their own Unity tests. Each test is documented in the Doxygen style; the documentation explains the target and the scenario of the test.

To ensure that a PAL port is working correctly, all PAL tests must pass on the target platform.

PAL test structure

The PAL test source code is located in mbed-cloud-client/mbed-client-pal/Test:

Test cases are split into module folders and are located in mbed-cloud-client/mbed-client-pal/Test/PAL_Modules/.
Unity test runners and main function for the tests are in mbed-cloud-client/mbed-client-pal/Test/TESTS/ directory.
Target OS and platform initialization code is located under the mbed-cloud-client-example/source/platform folder.

Note: The unit tests themselves do not need modification to run on a new platform and should not be modified. It should be sufficient to port the required drivers and platform initialization code.

Compiling PAL tests

To compile the tests you need to first import the mbed-cloud-client-example parent project with mbed import mbed-cloud-client-example. This import command will also automatically clone the mbed-cloud-client sub project that contains the test code:

Mbed OS

PAL has its own fork of Unity framework, and therefore, you need to disable the Mbed OS Unity framework with following command:

cd mbed-cloud-client-example
echo "mbed-os/features/frameworks/unity/*" >>.mbedignore

Compile the Mbed OS test with:

cd mbed-cloud-client-example
mbed test -t GCC_ARM -m K64F --compile \
	-n 'mbed-cloud-client-mbed-client-pal-*' \
	--app-config=mbed_app.json \
	--source=./ \
	--source=./source/platform/mbed-os \
	--source=./mbed-cloud-client/mbed-client-pal/Test/Unity \
	--source=./mbed-cloud-client/mbed-client-pal/Test/PAL_Modules \
	-DPAL_PLATFORM_DEFINED_CONFIGURATION=\"mbedOS_SST.h\" \
	-DRBP_TESTING \
	-DMBED_ENTRYPOINT \
	-DMBED_CLOUD_APPLICATION_NONSTANDARD_ENTRYPOINT

Note: If the device under test (DUT) is configured to use internal storage only, add the -DPAL_USE_FILESYSTEM=0 flag to the compilation command.

Linux

PAL test Linux compilation is otherwise identical to the example application compilation, but the mbedCloudClientExample.elf make target is left out. This will compile the tests and the example application.

cd mbed-cloud-client-example
python pal-platform/pal-platform.py deploy --target x86_x64_NativeLinux_mbedtls generate
cd __x86_x64_NativeLinux_mbedtls
cmake  -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=./../pal-platform/Toolchain/GCC/GCC.cmake -DEXTERNAL_DEFINE_FILE=./../define.txt
make

FreeRTOS

Similarly in FreeRTOS, the make command is issued without arguments to compile everything.

cd mbed-cloud-client-example
python pal-platform/pal-platform.py deploy --target K64F_FreeRTOS_mbedtls generate
cd __K64F_FreeRTOS_mbedtls
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=./../pal-platform/Toolchain/ARMGCC/ARMGCC.cmake -DEXTERNAL_DEFINE_FILE=./../define.txt
make

Running PAL tests

To run the full suite succesfully, you need:

An internet connection for the network tests.
Writable storage available for the storage tests.

To run the full PAL test suite:

Mbed OS

cd mbed-cloud-client-example
python TESTS/PAL/test.py mbed BUILD/tests/K64F/GCC_ARM/
icetea --reset --suite "TESTS/PAL/cache/dynamic.json" --tcdir "TESTS/PAL/testcases"

To run a single PAL test, flash the binary of interest shown by the TESTS/PAL/test.py script and monitor the progress from serial output.

Linux

cd mbed-cloud-client-example
python TESTS/PAL/test.py Linux __x86_x64_NativeLinux_mbedtls/Debug
icetea --suite "TESTS/PAL/cache/dynamic.json" --tcdir "TESTS/PAL/testcases"

To run a single PAL test, execute the relevant binary shown in the TESTS/PAL/test.py script and monitor the progress from standard output.

FreeRTOS

cd mbed-cloud-client-example
python TESTS/PAL/test.py mbed BUILD/tests/K64F/GCC_ARM/
icetea --reset --suite "TESTS/PAL/cache/dynamic.json" --tcdir "TESTS/PAL/testcases"

To run a single PAL test, flash the relevant binary shown in the TESTS/PAL/test.py script and monitor the progress from serial output.

Example PAL test failure

An example output of test output for test error:

mbed-cloud-client/mbed-client-pal/Test/PAL_Modules/Networking/pal_socket_test.c:664:TEST(pal_socket, ServerSocketScenario):===!!!===> <***UnityResult***>FAIL</***UnityResult***> <===!!!===: Expected 0x00000000 Was 0xFFFF0006

In this example, the socket test for the service scenario failed because the server failed to bind to a socket. This happened because the socket was not released by the OS after a previous run and returned the error PAL_ERR_SOCKET_ADDRESS_IN_USE instead of success.

For the full list of PAL error codes and their meaning, please refer to Source/PAL-Impl/Services-API/pal_errors.h.

Tip: A test error lists the expected and actual return values (Expected X Was Y) and the line number (:<line number>:TEST) in the test file where the error occurred, making it easy to understand which error caused the test to fail.

Troubleshooting

Ubuntu: First tests pass and then you get ResourceInitError: Required amount of devices not available.
- The error means that the Icetea cannot find DUT to run the test on.
- This is most likely because the board did not auto-mount after re-flashing. The PAL tests require the board to be flashed and reset several times during the test suite.
- If there is no automount option, a quick workaround can be to call udisksctl mountin a loop
  - while true; do sleep 1; findmnt /dev/sdc --noheadings --output TARGET || udisksctl mount -b /dev/sdc; done

Documentation