Launch tests
status = test_run()
status = test_run(module)
status = test_run(module, test_name)
status = test_run(module, test_name, options, exportToFile)
:module A String array. This input argument must be
- the name of an internal Scilab module (“core”, “time”, ...) or a sub-module (e.g. “optimization|neldermead”).
- the name of an ATOMS module (“module_lycee”, “nisp”, ...). To be taken into account, the module must be loaded when test_run() is called.
- the absolute directory path of a module.
: :test_name A string array : :options A string array
:no_check_ref does not check if the .dia and .dia.ref are equal : :no_check_error_output The error output stream is not checked. This
option can be used when Scilab complains about the localization being not available.
- : :create_ref create the .dia.ref file and does not check if the .dia
- and .dia.ref are equal
- : :show_error If an error occurs, show the last 10 lines of the
- execution
- : :show_diff If a difference is found, show the result of the command
- diff -u
- : :list Does not perform the tests but displays a list of available
- tests
: :help display some examples about how to use this command : :mode_nw Add the “-nw” option to the launch : :mode_nwni Add the “-nwni” option to the launch : :nonreg_test runs only the non-regression tests, skipping unit tests : :unit_test Runs only the unit tests, skipping non-regression tests : :skip_tests Skip the tests : :enable_lt Enable long-time execution tests : :short_summary Does not display statistics nor execution time after
execution (only number of executed, passed, failed and skipped tests will be displayed on a single line).:
:
Search for .tst files in the unit test and non-regression test library execute them, and display a report about success of failures. The .tst files are searched in directories SCI+”/modules//tests/unit_tests” and SCI+”/modules//tests/nonreg_tests”. Whenever a test is executed, a .dia file is generated which contains the full list of commands executed along with message which appears in the console. When the script is done, the .dia file is compared with the .dia.ref file which is expected to be in the same directory as the .tst file. If the two file are different, the test fails.
Special tags may be inserted in the .tst file, which help to control the processing of the corresponding test. These tags are expected to be found in Scilab comments.
These are the available tags:
Each test is executed in a separated process, created with the “host” command. That enables the current command to continue, even if the test as created an unstable environment. It also enables the tests to be independent from one another.
It may happen that the output of a test depends on the platform on which it is executed. In this case, the .ref file cannot be correct for all platforms and unit tests may fail for some platform. In this case, we can create a default .ref and create additionnal .ref file for each platform.
The various platform-specific .ref files must have one of the following extensions.
The algorithm is the following. First, the .ref is considered. If this file does not exist, the platform-specific .ref file is examined depending on the current platform.
// Launch all tests
// =============================================
test_run();
test_run([]);
test_run([],[]);
// Test one or several module
// =============================================
// Test one module
test_run('time');
// Test several modules
test_run(['time','string']);
// Test a submodule
test_run('optimization|neldermead');
// Refer to a module by its path
test_run(SCI+'/modules/core');
// Launch a specific test
// =============================================
// One specific test
test_run('time','datenum');
// Several tests
test_run('time',['datenum';'calendar']);
// Skip some tests
// =============================================
test_run('time',['datenum';'calendar'],'skip_tests');
// Options
// =============================================
// does not check if the .dia and .dia.ref are equal
test_run('time','datenum','no_check_ref');
// Create the .dia.ref file and does not check if the .dia and .dia.ref are equal
test_run([],[],'create_ref');
// Does not perform the tests but displays a list of available tests
test_run([],[],'list');
// Display some examples about how to use this command
test_run([],[],'help');
// Runs only the non-regression tests, skipping unit tests
test_run([],[],'nonreg_test');
// Runs only the unit tests, skipping non-regression tests
test_run([],[],'unit_test');
// Do not check the error output (std err)
test_run('boolean','bug_2799','no_check_error_output');
// Combine several options
test_run([],[],['no_check_ref','mode_nw']);
// Run unitary tests of an external module (with his path)
test_run('SCI/contrib/toolbox_skeleton')
// Export to a XML Xunit file
test_run('boolean',[],[],TMPDIR+"/boolean_test_run.xml");
test_run('time','datenum',[],TMPDIR+"/time_datenum_test_run.xml");
The tests are performed in the temporary directory, not in the directory which originaly contain the tests files. The .tst file is copied into the temporary directory, the test is performed and the .dia.ref is copied back into the original location.
The .tst script is not run as is. Instead, a header and a footer are inserted at the beginning and at the end of the .tst at the time the script is copied into the temporary directory. The role of this modification is to redirect the output messages into the .dia file, so that the user can have a log file once the test is performed.
Version Description 5.4.0 test_run returns a status:
show_diff and show_error added as new options
CLI SHELL MODE tag is added. Replaces JVM NOT MANDATORY (still supported)
test_run can work on an external module.
Fourth argument added to export to a XML file