Difference between revisions of "Ctest"

(Using the continuous track)
(viewing the source changes)
Line 21: Line 21:
 
We also may be interested in which files changed and their diffs. Clicking on the ''12'' in the ''Update Files'' column will bring us to a page listing the modified files. Clicking on the individual files will bring up a diff of the change. In this way we can quickly browse any change set.
 
We also may be interested in which files changed and their diffs. Clicking on the ''12'' in the ''Update Files'' column will bring us to a page listing the modified files. Clicking on the individual files will bring up a diff of the change. In this way we can quickly browse any change set.
  
[[Image:Cdash-websvn-diff-500.png|none|frame|Diff of one of the changed files]]
+
{|
 +
| [[Image:Cdash-modified-files-sm.png|none|frame|Modified files]]
 +
| [[Image:Cdash-websvn-diff-500.png|none|frame|Diff of one of the changed files]]
 +
|}

Revision as of 21:28, 2 June 2014

This page shows some examples of using VisIt's CDash dashboard.

Organization

The dashboard is organized as a table with a row for each submitting system. The columns represent various stats and show deltas. Each of the table entries can be clicked to access more detailed information. The 3 tracks, Nightly, Continuous and Experimental, can also be grouped. We're using the trusted label to further organize submissions. Nightly submissions are run once per day using a dated checkout, ie 00:00:00. All submitters thus grab the save revision no matter when the nightly submission is executed. Continuous submissions probe the repository periodically for updates, grab them, build incrementally and run the regression suite. This gives developers quick feedback about their changes.

Submitting

Ctest support is integrated in to VisIt's build system and may be enabled by setting BUILD_TESTING=ON in your builds CMakeCache.txt. After building with this option enabled invoking the ctest command from the build directory will run the tests and generate a report. Such a run could even be uploaded to the dashboard by adding the track, for example ctest -D Experimental. Nightly and continuous submissions are best automated using a CMake script. Scripts and instructions are located here.

Using the continuous track

Continuous submissions

The figure above shows a snapshot of the continuous section of the dashboard. We'll use the highlighted entry to point out some of the useful features. For example we can see that there were 12 files changed which resulted in 70 new test failures. The first thing we might want to do is look at the new failures. The test report will contain image diffs and stderr output either of which may be enough information for us to amend the commit.

Drilling down

viewing the new failures.

If we click on the 80 in the Test Fail collumm we will see all the failures not just the new one. To isolate the new failures, we click on the +70. This will bring up a list of just the test that failed as a result of our commit. From there which we can click on the Failed status to see the test output, stderr, and an image difference.

New failures

viewing the source changes

We also may be interested in which files changed and their diffs. Clicking on the 12 in the Update Files column will bring us to a page listing the modified files. Clicking on the individual files will bring up a diff of the change. In this way we can quickly browse any change set.

Modified files
Diff of one of the changed files