2.5 KiB
QB64-PE Testing Framework
QB64-PE features two separate testing frameworks, one for QB64-based tests
(mainly for the compiler and language functionality), and one for C++-based
tests (for libqb
).
QB64-based Tests
The QB64-based tests live in ./tests/compile_tests
. Each folder in that
directory represents a different category of tests, and within each of those
category folders are *.bas
files. Accompanying each *.bas
file is either an
*.output
or *.err
file with the same base name as a coresponding .bas
file. For test with an *.output
, the *.output
file contains the text that
the program compiled from the *.bas
file should produce. For *.err
tests,
the *.err
file contains the text of the error the QB64-PE compiler should
produce when attempting to compile that code.
Tests can also have an optional *.flags
file. The contents of this file will
be provide as command line arguments to QB64-PE when compiling the test source.
These tests are run via the ./tests/compile_tests.sh
script. The script
searches the ./tests/compile_tests
directory for any *.bas
files, and then
uses the provide QB64-PE compiler to test them.
C++-based Tests
The C++-based testing lives in ./tests/c/
. The tests themselves exist as
.cpp
files that compile into individual programs (each has its own main()
).
The tests make use of test.h
to run their tests and give proper output, and
failure is reported via the exit code.
The C++ tests are compiled using the ./tests/build.mk
make build logic. It's
fairly straight forward, each test executable gets its own name and then its
own combination of source/object files necessary to produce it. If you're
testing functionality from libqb
, then you need to link the proper object
files into your test. Additionally, any extra compiler flags can be defined
individually for each test.
./tests/run_c_tests.sh
is a script that runs all of the C++ tests during the
build. Internally it has a list of the names of all the test executables (to
avoid having to use make
to discover them). It expects the tests to have
already been built via a make build-tests
call, and runs the executables one
at a time and checks their exit code for errors.
QBasic Testcases
The QBasic testcases are a selection of QBasic programs that live in
./tests/qb64_testcases
. During the build process all of these programs are
tested to verify that QB64-PE is still capable of compiling them with no
errors. The behavior of the compiled programs is not verified.