Content-Length: 8816 | pFad | http://gcc.gnu.org/testing/

GCC Testing Efforts

This page describes regular efforts to test GCC thoroughly, plus ideas for additional testing.

For information about running the GCC testsuites, see Installing GCC: Testing. For information about testsuite organization and adding new tests, see Test Suites in the GCC Internals manual and the README files in the testsuite directories.

Current efforts

• H.J. Lu has several automated Linux/ia32 and Linux/Intel64 testers which build GCC and run the GCC testsuite as well as SPEC CPU 2K/2006 with various optimization options. All test results are sent to the gcc-testresults mailing list and any regressions are sent to the gcc-regression mailing list.
• Several people perform regular builds and regression test runs and send their test results to the gcc-testresults mailing list.
• Jan-Benedict Glaw is running a build robot that tries to build various cross-targets (stage1 only) on some machines.

Ideas for further testing

• Perform regular builds and testing of current GCC sources that are not already being reported regularly; see Installing GCC: Testing for instructions on submitting test results.
• Build cross compilers and test with simulators as described in How to test GCC on a simulator.
• If your system is beefy enough, do regular builds and test runs with RTL consistency checks enabled. This slows the compiler down by an order of magnitude but has found plenty of bugs in the past.
• Set up an autobuilder to nag people in e-mail when they submit patches that break builds. Ideally we would have one of these for at least each of the primary evaluation platforms listed in the current release criteria, but the more the merrier. Kaveh Ghazi < ghazi@caip.rutgers.edu> suggests that the autobuilders should keep track of regressions in the number of warnings and nag patchers until the new warnings are fixed, just as for testsuite regressions. It's important to have the autobuilders coordinate with each other, to avoid flooding contributors with mail.
• Build and test some or all of the following applications, some of which are part of the GCC release criteria. Links for downloads and for information about the packages are available in the build and test guides. If the instructions are incomplete for your target, update them. Additions to this list (accompanied with build and test guides) are welcome.

  Name	Language	Build and test guide
  Blitz++	C++	testing-blitz
  Boost	C++	testing-boost
  FTensor	C++	testing-ftensor
  LAPACK	Fortran 77	testing-lapack
  POOMA	C++	testing-pooma
  Qt	C++	testing-qt

• If the operating system kernel you use is normally compiled with GCC, try building it with the current sources. Make sure it boots. If you're building with relatively stable GCC sources such as a release branch, use the newly built kernel for running further GCC tests (keeping in mind the NO WARRANTY section of the GPL).
• Build and test applications that are important to you; investigate and report any problems you find.
• Build and test packages that are normally available on your platform and for which you have access to source.
• Run benchmarks regularly and report performance regressions.
• Extend the build robot to also do local builds, run the testsuite, visualize test result differences and probably use something like BuildBot. Some of the Compile Farm machines could also be used.