SystemView Post-Mortem Mode Recording
Post-mortem analysis is similar to single-shot recording, with one difference: SystemView events are continuously recorded. When the target buffer is filled older events are overwritten and reading the buffer provides the latest recorded events. Post-mortem analysis can provide information of long-time system tests and helps analyzing system crashes.
Post-Mortem Mode displays the latest system activity
Post-Mortem Mode and Single-Shot Recording
While Single-Shot recording allows analyzing system behaviour from a defined starting point until the SystemView buffer is filled with events, Post-Mortem Mode displays the latest system activity when reading out the data.
Both modes can be useful for different purposes.
Post-mortem mode can be useful when a system runs for a long time and suddenly crashes and for long-time tests of a system, to check and verify the system behaviour even after running for several hours.
Single-Shot recording can be used to record the startup of a system and to verify a specific part of the system behaves as expected, i.e. an interrupt triggers the correct task activation.
When a system which makes use of SystemView’s post-mortem mode crashes, the SystemView recording can provide important information about the system’s behaviour and what might have caused the crash.
This information may help to directly identify the problem or to create a reproduction scenario for further analysis.
A house-control system, controlling the heating system and connected to the internet, receives commands from a control server. The commands are handled by the controller task. The house-control works well, but some units crash once in a while. When debugging the system the crash does not occur. There is no hint what might go wrong.
SystemView’s post-mortem mode can help in this case and provide information about what happened prior to the crash, i.e. which task was running and crashed.
The SystemView module is included in the application and conﬁgured for post-mortem mode. The system can be used as usual.
Once the system crashed, a J-Link is attached to the system. With the J-Link the target is halted and the SystemView buﬀer read from the target into a ﬁle.
Analyzing the recording with SystemView shows that the the last activity has been in the monitoring task, which handles sensor data.
Going back in time displays the last system activity ﬂow. The controller task received a command to power down the heating system. While the controller task was active a timer interrupt triggered the monitoring task which preempts the controller task and tries to read the sensor data.
A closer look at the source code reveals that accessing the sensor can cause a crash when the heating system has been powered down.The problem could be solved by locking access with a resource semaphore.
Long-Time Behaviour Issues
SystemView’s post-mortem mode does not only help to analyse system crashes. It can be used to analyse apparently fine running systems, too, i.e. to verify that a system behaves as expected after free running for a longer period of time.
Analysis of free running systems might reveal issues in the system, like time differences or memory leaks, which cannot be found when debugging the system for a fraction of time but might affect released products.
Among other information a System displays its run time on the display.
In long time tests, after running for several hours, the time was off by 1 minute, compared to a an external stopwatch.
Normal debugging of the system does not show any issue. The system time is retrieved and displayed correctly. The issue seems to occur only when the system is running for a longer period.
Running a system with a debugger connected can influence the system behaviour. SystemView’s post-mortem mode can help in such cases. The SystemView module is included in the application and configured for post-mortem mode. Now the system is left running stand-alone.
In a new long time test, when the wrong time is displayed, a J-Link is attached to the system. With the J-Link the target is halted and the SystemView buﬀer read from the target into a ﬁle.
Loading the saved ﬁle into the SystemView App reveals that a timer interrupt handler sometimes runs for more than one millisecond and a SysTick interrupt gets lost, which results in the system time to be more and more off.
NOTE: Post-mortem analysis requires the debugger or debug probe to be able to connect to the target system without resetting it or modifying the RAM
To get as much useful data for analysis as possible it is recommended to use a large buffer for SystemView, 8 kByte or more. External RAM can be used for the SystemView buffer.
To configure the target system for post-mortem mode, the following configuration needs to be done: Define SEGGER_SYSVIEW_POST_MORTEM_MODE as 1
Define SEGGER_SYSVIEW_SYNC_PERIOD_SHIFT to make sure at least 1 sync is in the buffer.
Recommended sync frequency: Buffer size / 16. I.e. 4 kByte buffer = Sync every 256 packets = SEGGER_SYSVIEW_SYNC_PERIOD_SHIFT as 8.
To start recording in the system call SEGGER_SYSVIEW_Start(), i.e. in main() after calling SEGGER_SYSVIEW_Conf().
For more information refer to chapter "Target configuration" of the SystemView User Manual.
Get Post-Mortem Data From The System
To get the data which has been recorded in post-mortem mode, the SystemView buffer has to be read via SystemView or an external debugger.
- Configure and initialize SystemView from the application (SEGGER_SYSVIEW_Conf() or SEGGER_SYSVIEW_Init()).
- Start recording in the application from where it should be analyzed (SEGGER_SYSVIEW_Start()).
- Connect a debugger, load the target application, and let the system run.
With a J-Link the SystemView App can automatically read post-mortem data from the target.
- Start SystemView and select Target → Read Recorded Data
Without a J-Link or without the SystemView App the data can be read using following steps:
Since the SystemView buffer is a ring buffer, the data might have to be read in two chunks to start reading at the beginning and save as much data as possible.
- When the system crashed or all tests are done, attach with a debugger to the system and halt it.
- Get the SystemView RTT buffer (Usually _SEGGER_RTT.aUp.pBuffer).
- Save the data from pBuffer + WrOff until the end of the buffer to a file.
- Append the data from pBuffer until pBuffer + RdOff - 1 to the file.
- Save the file as *.SVdat or *.bin.
- Open the file with SystemView.