mirror of
https://github.com/QB64-Phoenix-Edition/QB64pe.git
synced 2024-09-20 06:44:44 +00:00
5c08642b80
- adds some useful info to the test docs - remove handmade CRC32 function - use new featured function(s) instead
70 lines
3.6 KiB
Markdown
70 lines
3.6 KiB
Markdown
QB64-PE Testing Framework
|
|
=========================
|
|
|
|
QB64-PE features two separate testing frameworks, one for QB64-based tests
|
|
(mainly for the compiler and language functionality), and one for C++-based
|
|
tests (for `libqb`).
|
|
|
|
QB64-based Tests
|
|
----------------
|
|
|
|
The QB64-based tests live in `./tests/compile_tests`. Each folder in that
|
|
directory represents a different category of tests, and within each of those
|
|
category folders are `*.bas` files. Accompanying each `*.bas` file is either an
|
|
`*.output` or `*.err` file with the same base name as a corresponding `*.bas`
|
|
file. For test with an `*.output`, the `*.output` file contains the text that
|
|
the program compiled from the `*.bas` file should produce. For `*.err` tests,
|
|
the `*.err` file contains the text of the error the QB64-PE compiler should
|
|
produce when attempting to compile that code.
|
|
|
|
To avoid unexpected test failures, you should write the `*.bas` in such a way
|
|
that it at best does not produce lines with trailing spaces. If this is not
|
|
possible e.g. by the use of formatted output, then make sure these trailing
|
|
spaces also exist in the `*.output` and/or `*.err` files respectively, those
|
|
spaces are often overseen or even get stripped by some Editors when saving the
|
|
file. Also the test **can not** produce trailing empty lines, you **must** make
|
|
sure the test output ends with the last meaningful line.
|
|
|
|
Tests can also have an optional `*.flags` file. The contents of this file will
|
|
be provide as command line arguments to QB64-PE when compiling the test source.
|
|
|
|
Tests are compiled from the directory the `.bas` file is in. That is, they are
|
|
compiled as though the user `cd`'d into the directory the `.bas` file is in,
|
|
and then ran the compiler from that starting location to compile the `.bas`. If
|
|
necessary this behavior can be changed by including a `*.compile-from-base`
|
|
file next to the associated `*.bas` test file. This will case the test script
|
|
to compile the `*.bas` file from the directory the compiler is located in
|
|
instead, and the full path to the `*.bas` file will be provided to `-x`.
|
|
|
|
These tests are run via the `./tests/compile_tests.sh` script. The script
|
|
searches the `./tests/compile_tests` directory for any `*.bas` files, and then
|
|
uses the provide QB64-PE compiler to test them.
|
|
|
|
C++-based Tests
|
|
---------------
|
|
|
|
The C++-based testing lives in `./tests/c/`. The tests themselves exist as
|
|
`.cpp` files that compile into individual programs (each has its own `main()`).
|
|
The tests make use of `test.h` to run their tests and give proper output, and
|
|
failure is reported via the exit code.
|
|
|
|
The C++ tests are compiled using the `./tests/build.mk` make build logic. It's
|
|
fairly straight forward, each test executable gets its own name and then its
|
|
own combination of source/object files necessary to produce it. If you're
|
|
testing functionality from `libqb`, then you need to link the proper object
|
|
files into your test. Additionally, any extra compiler flags can be defined
|
|
individually for each test.
|
|
|
|
`./tests/run_c_tests.sh` is a script that runs all of the C++ tests during the
|
|
build. Internally it has a list of the names of all the test executables (to
|
|
avoid having to use `make` to discover them). It expects the tests to have
|
|
already been built via a `make build-tests` call, and runs the executables one
|
|
at a time and checks their exit code for errors.
|
|
|
|
QBasic Testcases
|
|
----------------
|
|
|
|
The QBasic testcases are a selection of QBasic programs that live in
|
|
`./tests/qb64_testcases`. During the build process all of these programs are
|
|
tested to verify that QB64-PE is still capable of compiling them with no
|
|
errors. The behavior of the compiled programs is not verified.
|