Testing and CI

Developers using the PDDB will find that the PDDB native API is accurate and fully functional on all of the standard modes for Xous: running on real hardware, Renode emulation, and Hosted mode emulation.

In hosted mode, the PDDB will create a smaller version of itself (just 4MiB in length) by creating a file called tools/pddb-images/hosted.bin. In Renode, the PDDB creates a full-length PDDB (98MiB) at tools/pddb-images/renode.bin. The passwords in Renode are hard-wired to a for both the boot PIN and the update password, and the silicon DNA is 0x0.

The following features are available to assist with testing the PDDB in hosted mode:

  • "deterministic" -- forces a deterministic seed to the RNG in hosted mode
  • "ci" -- adds auto-running self-tests to the build
  • "pddbtest" -- adds shellchat test commands using the native API to the build; also works on hardware
  • "autobasis" -- patches over Basis enumeration commands to automatically create a test set of Bases.
  • "test-rekey" -- changes the DNA in hosted mode so re-key migrations can be tested

By default, "pddbtest" is enabled in the cargo xtask run configuration, enabling some extra commands in the shellchat PDDB menu.

The "ci" Feature

The "ci" test automatically kicks off a set of scripted tests that run inside the PDDB itself; so this does not use the API calls, but instead it calls methods normally only accessible internally within the server. The test does a series of stress-tests, creating and deleting keys of random sizes, with a specified naming scheme and data contents so that they can be automatically analyzed for correctness. The analysis is enabled by calling dbg_dump() periodically through the test -- an API call that is only made available in hosted mode. This call creates a new file that is a snapshot of the PDDB, along with a dump of the system keys, so that an analysis script can decrypt and analyze the PDDB's structure.

The correctness analysis of "ci" runs is done by the tools/ppdbdbg.py script. This script takes the keys provided, and decrypts the PDDB binary image. It scans the page table, checks the make-before-break buffer, free space tables, dictionaries and keys for consistency. It expects to see a certain pre-defined pattern of data in the dictionaries -- if you run this on a dump not generated by "ci", it will throw CI errors.

The "ci" script is not a "true" CI in the sense that it is not currently run automatically on every build. Part of the complication is that there isn't a good way in hosted mode to "quit" the emulation based on an internal trigger.

A set of true "CI" scripts are also being developed that will run for every build on the hardware-in-loop testing.

The "autobasis" Feature

This feature will automatically create and enumerate a couple dozen Bases in the PDDB, according to an algorithm to derive names and passwords. This is meant to accelerate stress testing of scenarios covering resource exhausting in the presence of secret Bases.