Prep for 10.2 M2

Performance fix of systemd daemon result-viewer

AUTHORS

@@ -1,4 +1,4 @@
-Copyright 2008 - 2019 by Phoronix Media.
+Copyright 2008 - 2020 by Phoronix Media.
 
 Lead Architects:
 

ChangeLog

@@ -1,5 +1,253 @@
 PHORONIX TEST SUITE CHANGE-LOG
 
+Phoronix Test Suite 10.2.0 Milestone 2
+6 December 2020
+
+pts-core: Transition to new JSON-based data format for managing test install metadata while maintaining backwards compatibility with prior XML format
+pts-core: Begin tracking test run-times per-hash (all configured test options) for more detailed time tracking and in turn time estimates on subsequent runs
+pts-core: Increase compression factor when uploading benchmark results
+pts-core: Last minute fixes for PHP 8.x
+phodevi: Report if CPUFreq Boost enabled/disabled
+turbostat: Add TurboStat module to easily generate turbostat dumps on a per-test basis to log files
+phoromatic: Much faster web viewer performance especially with large databases
+
+Phoronix Test Suite 10.2.0 Milestone 1
+21 November 2020
+
+pts-core: Log lock_stat, config.gz, dmidecode to system logs
+pts-core: Only show Apple options when running on macOS
+pts-core: Add extra safeguard around FORCE_TIMES_TO_RUN for disabling dynamic run count
+ob_auto_compare: Intelligently choose more relevant reference results
+phodevi: macOS CPU temperature monitoring support
+phodevi: Support for reading WattsUp Pro power meter data directly without utility using WATTS_UP_METER=<USB TTY>
+phodevi: Report Google Compute Engine machine type as part of motherboard string
+phodevi: Add dumpe2fs to system logs when running on EXT4 and root
+phodevi: Add block size reporting
+phodevi: CPU power reporting for Ampere Altra
+phodevi: Various Ampere Altra hardware detection/reporting improvements
+phodevi: Support reading CPU package power consumption on macOS using powermetrics
+
+Phoronix Test Suite 10.0.1
+21 October 2020
+
+pts-core: Fix for older versions of PHP such as in RHEL7
+pts-core: Add Thermald version reporting to system table when active
+phodevi: Detection improvements for new CPUs
+phodevi: Avoid "11th Gen Intel" CPU string
+phodevi: Update for NVMe drive reporting with KIOXIA, Toshiba sometimes not reporting their brand first
+
+Phoronix Test Suite 10.0.0
+13 October 2020
+
+pts-core: Ensure any post-run script is triggered still under halt-testing/skip-test conditions
+pts-core: Allow batch processing of performance per dollar metrics
+phodevi: Add Intel AMX presence detection
+phodevi: Add detection for Intel Alder Lake, Jasper Lake, Sapphire Rapids, and others
+phoromatic: Ensure Windows tests get displayed even if running the Phoromatic Server on Linux
+flush_caches: Add module to flush system caches between runs
+test_timeout: Add module to quit a test run if it exceeds a predefined time limit or 'auto' determination
+
+Phoronix Test Suite 10.0.0 Milestone 3
+1 October 2020
+
+pts-core: Don't show Vulkan options if Vulkan support is known to be unavailable/broken
+pts-core: Add OptionSelect tag support within test suite definitions
+pts-core: Allow * wildcard match in FileCheck handling within external dependency XML
+pts-core: Add "OS" TestType
+pts-core: Various external dependency updates
+pts-core: Continued tweaking to the analytics engine
+phodevi: Add Neoverse N2, Cortex-X1 detection and other Arm CPU updates
+pts_Graph: Various improvements
+
+Phoronix Test Suite 10.0.0 Milestone 2
+12 September 2020
+
+pts-core: Honor NO_COLOR environment variable for disabling terminal coloring
+pts-core: Allow launching just the result viewer from systemd (phoronix-result-server.service)
+pts-core: Result viewer UI tweaks
+pts-core: Minor fixes
+pts-core: Add RequiresRootInstall test profile option
+pts-core: Result file text output tweaks
+pts-core: More robust test result parsing in certain time formats
+
+Phoronix Test Suite 10.0.0 Milestone 1
+19 August 2020
+
+pts-core: Add keep-results-in-result-file sub-command
+pts-core: Split the dynamic result viewer into its own embed class for result file handling to allow for easy re-use by OpenBenchmarking, Phoromatic, etc
+pts-core: Various changes for supporting the new OpenBenchmarking.org infrastructure
+pts_Graph: Add histogram type
+phodevi: CPU core name / family detection to friendly strings (OpenBenchmarking.org driven)
+phodevi: Report core name via system-info command
+phodevi: Various storage HDD/SSD detection reporting improvements
+phodevi: Ampere Altra detection
+phodevi: macOS/Apple hardware detection improvements
+
+Phoronix Test Suite 9.8.0
+9 July 2020
+
+pts-core: Handling of cases where all options are unsupported (such as CUDA and OpenCL tests with no valid backends found) to avoid prompts
+pts-core: Be more liberal in allowing CUDA/OpenCL GPU compute tests to run fine without display server active
+pts-core: Fix for not filling test suites of contained tests in some select cases (sub-suites listing in results viewer)
+pts-core: Allow creating new virtual test suites that are not prefixed by a OB repository
+phodevi: NVIDIA GPU detection within WSL2
+docker: Rewritten phoronix/pts Docker implementation based on Ubuntu 20.04 LTS
+
+Phoronix Test Suite 9.8.0 Milestone 2
+24 June 2020
+
+pts-core: Add client side plumbing for estimated test install time from OpenBenchmarking.org
+pts-core: Add estimate-install-time sub-command, show estimated install time when installing a test
+pts-core: Improve web browser selection logic for result viewer
+pts-core: Add search field to the result viewer areas for limiting based on title/argument matching
+pts-core: Various minor fixes
+pts-core: Don't show OpenCL test options if no OpenCL drivers/adapters appear present
+phodevi: Fix for incorrect monitor count when parsing X.Org log
+phodevi: Warning fixes for latest WSL
+
+Phoronix Test Suite 9.8.0 Milestone 1
+3 June 2020
+
+pts-core: When an install fails, try a newer minor version update to that test profile if available (fixes broken download mirrors, etc)
+pts-core: When a newer minor version of a test profile is installed and a requested (older) one is not, use the newer compatible version
+pts-core: Rewrite virtual test suite implementation
+pts-core: Add handler for ensuring reported system time is accurate, otherwise calculate current date/time using offsets
+pts-core: When scanning test suites on disk, ensure the latest test suite version is used
+pts-core: Add test-to-suite-map sub-command
+pts-core: Fix for loading the result viewer JavaScript / CSS when running on a Windows host
+pts-core: For tests requiring root but not run initially as root and using monitoring data, drop the first run to ensure sudo promot doesn't skew time
+pts-core: Fix PHP8 error in the 'info' sub-command
+pts-core: Other PHP deprecation code tweaks
+phodevi: Add amd_energy Linux driver support
+
+Phoronix Test Suite 9.6.1
+25 May 2020
+
+pts-core: Show unit/scale in CSV export
+pts-core: When limiting results to show in result viewer, keep child objects that match the desired type for the parent
+pts-core: Add rebuild-test-suite option for regenerating suite XML all cleaned up
+pts-core: Allow exporting result files to XML test suites via the result viewer
+linux_perf: Reporting many more events
+pts-core: External dependency updates
+pts_Graph: graph code tweaks
+phodevi: Avoid macOS reporting X-Core string before Intel CPU string
+phodevi: Sway compositor version detection
+phodevi: Detection of newer Arm CPU cores
+
+Phoronix Test Suite 9.6.0
+21 April 2020
+
+pts-core: Allow building virtual suites off external dependencies
+pts-core: Various result viewer improvements
+pts-core: Allow saving results under stress-run
+pts-core: Responsive/mobile result viewer improvements
+pts-core: Show local tests within the build-suite sub-command
+pts-core: Show local suites within the list-installed-suites sub-command
+pts_Graph: Restore capability for drawing mini graphs for all results in a result file overview
+
+Phoronix Test Suite 9.6.0 Milestone 3
+12 April 2020
+
+pts-core: Allow automated per-thread/core/clock/RAM graphs within the result viewer
+pts-core: Result viewer add "Limit displaying results to tests within:"
+pts-core: From the result viewer allow "Show Geometric Means Per-Suite/Category"
+pts-core: Add workload-topology sub-command
+pts-core: Allow adding deprecated tests to build-suite
+pts-core: CSV export improvements
+pts-core: Add "Do Not Show Results With Incomplete Data" to the result viewer
+pts-core: Allow test suite "Status" tag
+pts-core: Don't report OB updates on deprecated tests or suites
+
+Phoronix Test Suite 9.6.0 Milestone 2
+5 April 2020
+
+pts-core: Documentation updates
+pts-core: External dependency updates
+pts-core: Allow listing test profiles within the result viewer
+pts-core: Allow listing test suites and test suite data within the result viewer
+pts-core: Allow showing test profile information within the result viewer
+pts-core: Various other updates to the result viewer
+pts-core: Add vaapi as possible external dependency
+update_checker: Add message of the day support
+
+Phoronix Test Suite 9.6.0 Milestone 1
+24 March 2020
+
+pts-core: Add new analyze-run-times sub-command
+pts-core: Allow hiding results from within the result viewer
+pts-core: Allow generating performance-per-dollar metrics from within the result file
+pts-core: Allow showing system logs from the result viewer
+pts-core: Allow showing test description within the result viewer
+pts-core: Fixes for running the result viewer from a website sub directory in correctly loading assets
+pts-core: Add result viewer option "hide noisy results"
+pts-core: Add result viewer option "hide results with little change/spread"
+pts-core: Add result viewer option "only show results where X faster than Y"
+pts-core: Add XML export to result viewer export menu
+pts-core: Support deleting runs from the result viewer
+pts-core: Improve the print view from the result viewer
+phodevi: BSD tweaks/fixes
+
+Phoronix Test Suite 9.4.1
+4 March 2020
+
+Various minor bug fixes and OpenBenchmarking.org cache update
+
+Phoronix Test Suite 9.4.0-Vestby
+25 February 2020
+
+pts-core: Add option in the result viewer to allow compacting multi-option test results into single graphs
+pts-core: Add option in the result viewer to allow compacting multi-version test results into single graphs
+pts-core: Add Parent tag to result file schema for specifying relation to other result objects
+pts-core: Report test profile support errors when intended
+pts-core: Fix showing NVIDIA test options on non-Linux NVIDIA platforms
+pts-core: Don't show delete/annotation options for dynamically generated result objects where those options are not relevant
+pts-core: Allow showing performance-per-suite option within result viewer
+pts_Graph: When going to PDF with GD library, just use the OB rather than temporary file
+pts_Graph: Restore support for error bars on vertical bar graph rendering
+pts_Graph: Support for simpler view / rendering without header/footer
+
+Phoronix Test Suite 9.4.0 Milestone 3
+18 February 2020
+
+pts-core: Add PDF bookmarks to result graphs on PDF result file generation
+pts-core: Add automated executive summary of result files via executive-summary sub-command and PDF result export
+pts-core: Add remove-result-from-result-file sub-command (slightly different from remove-results-from-result-file)
+pts-core: Set SaveTestLogs = TRUE by default
+pts-core: Set SaveInstallationLogs = TRUE by default
+pts-core: Refactor test-logs (SaveTestLogs) handling
+pts-core: Refactor test installation logs handling
+pts-core: More verbose reporting of system/external dependency solving status
+pts-core: Get the modern result viewer working on Windows
+pts-core: Support for showing test logs from the result viewer
+pts-core: Support for showing install logs from the result viewer
+pts-core: PDF output improvements
+pts-core: Fix for some suites not showing up on list-suites sub-command
+pts-core: Report "Test Suites Containing This Test" in the info sub-command on tests
+pts-core: On PDF export allow showing geometric means of tests based on their test suite connections
+pts-core: Add intersect sub-command for showing tests common between multiple results/suites
+
+Phoronix Test Suite 9.4.0 Milestone 2
+5 February 2020
+
+pts-core: Add config options allowing to modify or delete result files from the result viewer
+pts-core: Allow deleting results from the result viewer
+pts-core: Allow editing result file information from result viewer
+pts-core: Allow deleting results (graphs) from result file in result viewer
+pts-core: Add ability to annotate (comment) on result graphs within the result viewer
+pts-core: Support converting graphs to PNG/PDF with Inkscape or svg-convert when PHP GD is not installed/enabled
+pts-core: Add Persistent Memory disk targets for disk test
+
+Phoronix Test Suite 9.4.0 Milestone 1
+13 January 2020
+
+pts-core: Allow test profile test options to expose more helper information strings
+pts-core: External dependency updates
+pts-core: When a test fails to install due to linker error, try to run that missing library through the external dependency handler to hint at the name
+pts-core: When a test fails to install due missing header file, try to run that missing library through the external dependency handler to hint at the name
+pts_Graph: Pass/fail graph formatting improvements
+phodevi: Convert large RAM capacity basic strings into GB rather than large MB
+phodevi: Add support for Linux 5.6+ drivetemp kernel module for ATA drive temperatures
+
 Phoronix Test Suite 9.2.1
 11 December 2019
 
@@ -11,6 +259,7 @@ Phoronix Test Suite 9.2.0-Hurdal
 
 pts-core: Fix for sensor-driven result outputs (e.g. power)
 pts-core: Clean up precision handling on graphs
+pts-core: Add run-subset sub-command
 pts-core: Add Clear Linux external dependencies handling for when not relying upon the Clear bundles that include all known dependencies
 phodevi: More robust ACPI power_meter handling
 phodevi: Support reading the NVMe drive temperature via HWMON on Linux 5.5+

README.md

@@ -1,4 +1,4 @@
-# Phoronix Test Suite 9.2.1
+# Phoronix Test Suite 10.2.0m2
 https://www.phoronix-test-suite.com/
 
 The **Phoronix Test Suite** is the most comprehensive testing and benchmarking

deploy/docker/ubuntu-pts-docker-build.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+
+export TESTS_TO_PRECACHE=""
+
+# Ensure Docker on system
+DIR_NAME=phoronix-pts-docker
+mkdir -p ./$DIR_NAME/base/
+OS_OUTER=`pwd`
+cd $DIR_NAME/base
+OS_ROOT_PATH=`pwd`
+
+rm -rf phoronix-test-suite/
+git clone https://github.com/phoronix-test-suite/phoronix-test-suite.git
+cd phoronix-test-suite
+rm -rf .git
+
+# cache OpenBenchmarking.org metadata
+export PTS_USER_PATH_OVERRIDE=$OS_ROOT_PATH/var/lib/phoronix-test-suite/
+rm -f $PTS_USER_PATH_OVERRIDE
+mkdir -p $PTS_USER_PATH_OVERRIDE
+./phoronix-test-suite make-openbenchmarking-cache lean
+
+# cache select tests
+export PTS_DOWNLOAD_CACHE_OVERRIDE=$OS_ROOT_PATH/var/cache/phoronix-test-suite/download-cache/
+mkdir -p $PTS_DOWNLOAD_CACHE_OVERRIDE
+export PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT=1
+# ./phoronix-test-suite make-download-cache $TESTS_TO_PRECACHE
+# ./phoronix-test-suite info 1809091-PTS-CLEARLIN01
+
+rm -f $PTS_USER_PATH_OVERRIDE/core.pt2so
+
+# cleanup
+cd $OS_OUTER/$DIR_NAME
+
+tar -C base -cf base.tar .
+rm -f base.tar.xz
+xz -v -T0 base.tar
+
+cat > Dockerfile << EOF
+FROM ubuntu:20.04
+MAINTAINER Phoronix Media <commercial@phoronix-test-suite.com>
+ADD base.tar.xz /
+ARG DEBIAN_FRONTEND=noninteractive
+RUN apt update
+RUN apt install -y unzip php-cli apt-utils mesa-utils php-xml git-core apt-file
+RUN apt-file update
+CMD ["/phoronix-test-suite/phoronix-test-suite", "shell"]
+EOF
+
+docker build -t $DIR_NAME .
+
+# docker run -it phoronix-pts-docker
+
+# docker tag phoronix-pts-docker phoronix/pts
+# docker push phoronix/pts
+

deploy/phoromatic-systemd/phoronix-result-server.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Phoronix Test Suite Result viewer
+After=syslog.target network.target
+Documentation=man:phoronix-test-suite(1)
+
+[Service]
+TimeoutStartSec=300
+ExecStart=/usr/bin/phoronix-test-suite start-result-viewer daemon
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target
+
+

documentation/TODO

@@ -1,41 +0,0 @@
-PHORONIX TEST SUITE PUBLIC TODO LIST:
-
-### PTS-CORE ###
-
-- Move External Dependencies into OpenBenchmarking.org cloud
-- Reporting to OpenBenchmarking.org on broken download URLs for test profiles.
-- Reporting of install times to OpenBenchmarking.org and also exposing this information back to clients.
-- Support for multiple languages / internationalization.
-- BLKID fall-back support for determining file-system [e.g. /sbin/blkid -s TYPE -o value /dev/sda1].
-- Possibly restore "weighted suites" [http://phorogit.com/index.php?p=phoronix-test-suite.git&a=commitdiff&h=548e5c073cc90d92ac3f382ca4933d5f556e78c6&hb=444441f9be866f51920e0ad8ca3909951b4d90c5]
-- Integrate finish-run command into the run command, auto-detect if the result file was a partial save and then ask if to perform run recovery'
-- Multi-threaded downloading of test files
-- Optimus / DRI_PRIME=1 / xrandr --listproviders better GPU detection
-- NVIDIA make use of UsedDedicatedGPUMemory as new GPU sensor
-- Clean up all the WebSocket code and make it fit to spec
-- Incremental/streaming result uploads support
-- Check with Phodevi's parsers what other tools support JSON outputs - e.g. http://karelzak.blogspot.com/2015/06/json-output-for-basic-tools.html for cleaner parsing
-- Improve titles on tables like in: http://openbenchmarking.org/result/1603172-GA-AMDTONGA973
-
-### PTS-GRAPH ###
-
-- Rotated text alignment for some PNG graphics is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
-
-### OPENBENCHMARKING.ORG ###
-
-- Shorten long change-logs on test profile pages - e.g. http://openbenchmarking.org/test/pts/hpcc
-
-### WEB SOCKET / HTML5 USER INTERFACE WORK ###
-
-- Implement WebSocket security - add the password prompts from ~/.phoronix-test-suite/user-config.xml, etc
-- HTTPS support
-- Ensuring only one session at a time, at least for benchmarking
-- Handle test installations and external dependencies (prompting for root password handling?)
-- Porting more of the web-interface code to the new JavaScript + WebSocket design rather than original PHP design [Originally with the 5.0 UI design it wasn't settled for mostly JS + WebSocket but some of the original code is with a traditional PHP-based + web server design... so slowly move over to newer design.]
-
-### PHOROMATIC ####
-
-- Implement the complementary WebSocket communication to HTTP pulls
-- Ensure sane validating of more user input / resource uploads
-- Hook up TriggerTarget for allowing event.php to optionally specify a sub-set of systems/groups to target for a given trigger
-- Clean up the code and modularize more of the code

documentation/man-pages/phoronix-test-suite.1

@@ -1,4 +1,4 @@
-.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "9.2.1"
+.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "10.2.0m2"
 .SH NAME
 phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.
 .SH SYNOPSIS
@@ -48,6 +48,9 @@ This option will permanently remove a installed test by the Phoronix Test Suite.
 .B benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.
 .TP
+.B estimate-install-time [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will provide estimates for test install/setup time length.
+.TP
 .B estimate-run-time [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will provide estimates for test run-time / length.
 .TP
@@ -60,7 +63,10 @@ This option will run the selected test(s).
 .B run-random-tests
 This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.
 .TP
-.B run-tests-in-suite
+.B run-subset [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the selected test(s) but prompt the user when passing any test suites or result files what subset / test(s) contained within there to run rather than running all passed tests/suites/results.
+.TP
+.B run-tests-in-suite [Suite]
 This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.
 .TP
 .B stress-batch-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
@@ -134,13 +140,16 @@ This option is used for uploading a test result to OpenBenchmarking.org.
 .B upload-test-profile
 This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.
 .TP
-.B upload-test-suite
+.B upload-test-suite [Suite]
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 .TP
 .SH INFORMATION
 .B info [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
 .TP
+.B intersect [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.
+.TP
 .B list-all-tests
 This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
 .TP
@@ -186,6 +195,9 @@ This option will list all available test profiles that are available from the en
 .B search
 This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..
 .TP
+.B test-to-suite-map
+This option will list all test profiles and any test suites each test belongs to.
+.TP
 .SH ASSET CREATION
 .B build-suite
 This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.
@@ -211,6 +223,9 @@ This will download the selected test file(s) to the Phoronix Test Suite download
 .B inspect-test-profile [Test]
 This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
 .TP
+.B rebuild-test-suite [Suite]
+This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.
+.TP
 .B result-file-to-suite [Test Result]
 This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
 .TP
@@ -220,12 +235,12 @@ This option can be used for validating a Phoronix Test Suite result file as bein
 .B validate-test-profile [Test]
 This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.
 .TP
-.B validate-test-suite
+.B validate-test-suite [Suite]
 This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.
 .TP
 .SH RESULT MANAGEMENT
-.B analyze-all-runs [Test Result]
-This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+.B analyze-run-times [Test Result]
+This option will read a saved test results file and print the statistics about how long the testing took to complete.
 .TP
 .B auto-sort-result-file [Test Result]
 This option is used if you wish to automatically attempt to sort the results by their result identifier string.
@@ -242,6 +257,9 @@ This option is used if you wish to edit the title and description of an existing
 .B extract-from-result-file [Test Result]
 This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.
 .TP
+.B keep-results-in-result-file [Test Result]
+This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.
+.TP
 .B merge-results [Test Result]  ...
 This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.
 .TP
@@ -251,9 +269,12 @@ This option will re-render and save all result graphs within a saved file. This
 .B remove-result [Test Result]
 This option will permanently remove the saved file set that is set as the first argument.
 .TP
-.B remove-results-from-result-file [Test Result]
+.B remove-result-from-result-file [Test Result]
 This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.
 .TP
+.B remove-results-from-result-file [Test Result]
+This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to provide a string to search for in removing those results from that given result file.
+.TP
 .B remove-run-from-result-file [Test Result]
 This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.
 .TP
@@ -266,15 +287,9 @@ This option is used if you wish to change the name of the saved name of a result
 .B reorder-result-file [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 .TP
-.B result-file-confidence [Test Result]
-This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.
-.TP
 .B result-file-raw-to-csv [Test Result]
 This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
 .TP
-.B result-file-stats [Test Result]
-This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
-.TP
 .B result-file-to-csv [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 .TP
@@ -290,8 +305,8 @@ This option will read a saved test results file and output the system hardware a
 .B show-result [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 .TP
-.B wins-and-losses [Test Result]
-This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
+.B workload-topology [Test Result]
+This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.
 .TP
 .SH OTHER
 .B commands
@@ -312,6 +327,22 @@ This option will display a list of available Phoronix Test Suite commands and po
 .B version
 This option will display the Phoronix Test Suite client version.
 .TP
+.SH RESULT ANALYSIS
+.B analyze-all-runs [Test Result]
+This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+.TP
+.B executive-summary [Test Result]
+This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.
+.TP
+.B result-file-confidence [Test Result]
+This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.
+.TP
+.B result-file-stats [Test Result]
+This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
+.TP
+.B wins-and-losses [Test Result]
+This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
+.TP
 .SH MODULES
 .B auto-load-module
 This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
@@ -368,5 +399,5 @@ https://www.phoronix.com/
 .br
 https://www.phoronix.com/forums/
 .SH AUTHORS
-Copyright 2008 - 2019 by Phoronix Media, Michael Larabel.
+Copyright 2008 - 2020 by Phoronix Media, Michael Larabel.
 .TP

documentation/phoronix-test-suite.html

@@ -209,42 +209,10 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
 		<h1>Phoronix Test Suite</h1>
 		</div>
 		<div id="pts_banner_nav"></div>
-		<div id="pts_template_body"><ol><li><a href="#%0AGettingStarted%0A">
-Getting Started
-</a></li><li><a href="#%0A%0AUserOptions%0A">
-
-User Options
-</a></li><li><a href="#%0A%0AModuleOptions%0A">
-
-Module Options
-</a></li><li><a href="#%0AInstallationInstructions%0A">
-Installation Instructions
-</a></li><li><a href="#%0A%0AExternalDependencies%0A">
-
-External Dependencies
-</a></li><li><a href="#%0AConfiguration%0A">
-Configuration
-</a></li><li><a href="#%0AGeneralInformation%0A">
-General Information
-</a></li><li><a href="#%0A%0AVirtualTestSuites%0A">
-
-Virtual Test Suites
-</a></li><li><a href="#%0AComponentTesting%0A">
-Component Testing
-</a></li><li><a href="#%0APhoromatic%0A">
-Phoromatic
-</a></li><li><a href="#%0AOfflineImprovements+ConfidentialTesting/AvoidingAccidentalResultUploads%0A">
-Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads
-</a></li><li><a href="#%0ACreatingTestProfiles%0A">
-Creating Test Profiles
-</a></li><li><a href="#%0ADevelopmentCredits%0A">
-Development Credits
-</a></li></ol><hr style="height: 50px; border: 0;"><a name="GettingStarted%0A"></a><h1>
-Getting Started
-</h1>
+		<div id="pts_template_body"><ol><li><a href="#GettingStarted">Getting Started</a></li><li><a href="#UserOptions">User Options</a></li><li><a href="#ModuleOptions">Module Options</a></li><li><a href="#InstallationInstructions">Installation Instructions</a></li><li><a href="#ExternalDependencies">External Dependencies</a></li><li><a href="#Configuration">Configuration</a></li><li><a href="#GeneralInformation">General Information</a></li><li><a href="#VirtualTestSuites">Virtual Test Suites</a></li><li><a href="#ComponentTesting">Component Testing</a></li><li><a href="#Phoromatic">Phoromatic</a></li><li><a href="#OfflineImprovements+ConfidentialTesting/AvoidingAccidentalResultUploads">Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads</a></li><li><a href="#DevelopmentCredits">Development Credits</a></li></ol><hr style="height: 50px; border: 0;"><a name="GettingStarted"></a><h1>Getting Started</h1>
 <h1>Overview</h1>
 <p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
-<p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
+<p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 400 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
 <p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
 <h1>OpenBenchmarking.org</h1>
 <p>OpenBenchmarking.org is an open, collaborative testing platform that makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite test client. OpenBenchmarking.org serves as a repository for storing test profiles, test suites, and result data. Test profiles and suites are stored in the OpenBenchmarking.org cloud to allow for new/updated tests to be seamlessly obtained via the Phoronix Test Suite without needing to manually update the Phoronix Test Suite client. OpenBenchmarking.org also makes it easy to facilitate side-by-side comparisons with any other results stored in the OpenBenchmarking.org cloud. Any Phoronix Test Suite user is permitted to upload their test results, test profiles, and suites to OpenBenchmarking.org.</p>
@@ -253,10 +221,7 @@ Getting Started
 <h1>Phoromatic</h1>
 <p>Phoromatic is a remote management system for the Phoronix Test Suite that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. Phoromatic can also interface with revision control systems to offer support for issuing new tests on a context-basis, such as whenever a Git commit has been pushed or new daily image available. The test results are then available from this central, secure location.</p>
 <p>Phoromatic is an add-on to the Phoronix Test Suite that's primarily intended for enterprise users when facilitating tests across a wide-spectrum of hardware within a test lab or when needing to carry out tests on a routine basis.</p>
-<hr style="height: 50px; border: 0;"><a name="UserOptions%0A"></a><h1>
-
-User Options
-</h1>
+<hr style="height: 50px; border: 0;"><a name="UserOptions"></a><h1>User Options</h1>
 <p>The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running <em>phoronix-test-suite help.</em></p><hr>
 <h1>System</h1>
 <h3>diagnostics<em>  </em>
@@ -297,6 +262,9 @@ User Options
 <h3>benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.</p>
+<h3>estimate-install-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
+</h3>
+<p>This option will provide estimates for test install/setup time length.</p>
 <h3>estimate-run-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will provide estimates for test run-time / length.</p>
@@ -309,7 +277,10 @@ User Options
 <h3>run-random-tests<em>  </em>
 </h3>
 <p>This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.</p>
-<h3>run-tests-in-suite<em>  </em>
+<h3>run-subset<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option will run the selected test(s) but prompt the user when passing any test suites or result files what subset / test(s) contained within there to run rather than running all passed tests/suites/results.</p>
+<h3>run-tests-in-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.</p>
 <h3>stress-batch-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
@@ -383,13 +354,16 @@ User Options
 <h3>upload-test-profile<em>  </em>
 </h3>
 <p>This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.</p>
-<h3>upload-test-suite<em>  </em>
+<h3>upload-test-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.</p>
 <h1>Information</h1>
 <h3>info<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will show details about the supplied test, suite, virtual suite, or result file.</p>
+<h3>intersect<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.</p>
 <h3>list-all-tests<em>  </em>
 </h3>
 <p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
@@ -435,6 +409,9 @@ User Options
 <h3>search<em>  </em>
 </h3>
 <p>This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..</p>
+<h3>test-to-suite-map<em>  </em>
+</h3>
+<p>This option will list all test profiles and any test suites each test belongs to.</p>
 <h1>Asset Creation</h1>
 <h3>build-suite<em>  </em>
 </h3>
@@ -460,6 +437,9 @@ User Options
 <h3>inspect-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.</p>
+<h3>rebuild-test-suite<em>  [Suite]</em>
+</h3>
+<p>This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.</p>
 <h3>result-file-to-suite<em>  [Test Result]</em>
 </h3>
 <p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
@@ -469,13 +449,13 @@ User Options
 <h3>validate-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.</p>
-<h3>validate-test-suite<em>  </em>
+<h3>validate-test-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.</p>
 <h1>Result Management</h1>
-<h3>analyze-all-runs<em>  [Test Result]</em>
+<h3>analyze-run-times<em>  [Test Result]</em>
 </h3>
-<p>This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.</p>
+<p>This option will read a saved test results file and print the statistics about how long the testing took to complete.</p>
 <h3>auto-sort-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to automatically attempt to sort the results by their result identifier string.</p>
@@ -491,6 +471,9 @@ User Options
 <h3>extract-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.</p>
+<h3>keep-results-in-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.</p>
 <h3>merge-results<em>  [Test Result]  ...</em>
 </h3>
 <p>This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.</p>
@@ -500,9 +483,12 @@ User Options
 <h3>remove-result<em>  [Test Result]</em>
 </h3>
 <p>This option will permanently remove the saved file set that is set as the first argument.</p>
-<h3>remove-results-from-result-file<em>  [Test Result]</em>
+<h3>remove-result-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.</p>
+<h3>remove-results-from-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to provide a string to search for in removing those results from that given result file.</p>
 <h3>remove-run-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.</p>
@@ -515,15 +501,9 @@ User Options
 <h3>reorder-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
-<h3>result-file-confidence<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
 <h3>result-file-raw-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
-<h3>result-file-stats<em>  [Test Result]</em>
-</h3>
-<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
 <h3>result-file-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.</p>
@@ -539,9 +519,9 @@ User Options
 <h3>show-result<em>  [Test Result]</em>
 </h3>
 <p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
-<h3>wins-and-losses<em>  [Test Result]</em>
+<h3>workload-topology<em>  [Test Result]</em>
 </h3>
-<p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
+<p>This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.</p>
 <h1>Other</h1>
 <h3>commands<em>  </em>
 </h3>
@@ -561,6 +541,22 @@ User Options
 <h3>version<em>  </em>
 </h3>
 <p>This option will display the Phoronix Test Suite client version.</p>
+<h1>Result Analysis</h1>
+<h3>analyze-all-runs<em>  [Test Result]</em>
+</h3>
+<p>This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.</p>
+<h3>executive-summary<em>  [Test Result]</em>
+</h3>
+<p>This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.</p>
+<h3>result-file-confidence<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
+<h3>result-file-stats<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
+<h3>wins-and-losses<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
 <h1>Modules</h1>
 <h3>auto-load-module<em>  </em>
 </h3>
@@ -604,10 +600,7 @@ User Options
 <h3>start-result-viewer<em>  </em>
 </h3>
 <p>Start the web-based result viewer.</p>
-<hr style="height: 50px; border: 0;"><a name="ModuleOptions%0A"></a><h1>
-
-Module Options
-</h1>
+<hr style="height: 50px; border: 0;"><a name="ModuleOptions"></a><h1>Module Options</h1>
 <p>The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the AutoLoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running <em>phoronix-test-suite list-modules.</em></p><hr>
 <h2>Backup Creation + Restore</h2>
 <p>This is a module for creating backups of the Phoronix Test Suite / Phoromatic and allows for restoring of created backups. The backup will be in ZIP or TAR format. If only a path is specified, the file-name will be auto-generated with a current time-stamp.</p>
@@ -616,9 +609,13 @@ Module Options
 <h2>Dummy Module</h2>
 <p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
 <p>phoronix-test-suite dummy_module.dummy-command</p>
+<p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
 <h2>Generate Perf FlameGraphs For Tests</h2>
 <p>Setting FLAME_GRAPH_PATH=&lt;path to flamegraph path&gt; will auto-load and enable this Phoronix Test Suite module. The module will generate a Linux perf FlameGraph for each test run during the benchmarking process. Details on FlameGraph @ https://github.com/brendangregg/FlameGraph</p>
 <p>This module utilizes the following environmental variables: FLAME_GRAPH_PATH.</p>
+<h2>Flush Caches</h2>
+<p>Loading this module will ensure caches (page cache, swap, etc) automatically get flushed prior to running any test.</p>
+<p>This module utilizes the following environmental variables: PTS_FLUSH_CACHES.</p>
 <h2>Graphics Override</h2>
 <p>This module allows you to override some graphics rendering settings for the ATI and NVIDIA drivers while running the Phoronix Test Suite.</p>
 <p>This module utilizes the following environmental variables: FORCE_AA, FORCE_AF.</p>
@@ -630,6 +627,7 @@ Module Options
 <p>This module utilizes the following environmental variables: LINUX_PERF.</p>
 <h2>Dynamic Result Viewer</h2>
 <p>This module pre-loads the HTTP dynamic result viewer for Phoronix Test Suite data.</p>
+<p>phoronix-test-suite load_dynamic_result_viewer.start</p>
 <h2>Log Exporter</h2>
 <p>This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.</p>
 <p>This module utilizes the following environmental variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.</p>
@@ -648,6 +646,7 @@ Module Options
 <p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler and powersave governor checks.</p>
 <p>phoronix-test-suite perf_tips.show</p>
 <p>This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.</p>
+<p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler: it gives a warning if BFQ is being used with an incorrect configuration in a disk benchmark, and suggests the right configuration to use. For the moment it only works for existing, throughput-based tests. It will need to be extended for responsiveness and soft real-time-latency tests.</p>
 <h2>Benchmarking Compiler PGO Impact</h2>
 <p>This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.</p>
 <p>phoronix-test-suite pgo.benchmark</p>
@@ -660,6 +659,7 @@ Module Options
 <p>phoronix-test-suite phoromatic.list-results</p>
 <p>phoronix-test-suite phoromatic.clone</p>
 <p>phoronix-test-suite phoromatic.export-results-for-account-schedules</p>
+<p>The Phoromatic module contains the client support for interacting with Phoromatic and Phoromatic Tracker services.</p>
 <h2>Pushover.net</h2>
 <p>Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.</p>
 <p>This module utilizes the following environmental variables: PUSHOVER_NET_USER.</p>
@@ -674,23 +674,105 @@ Module Options
 <h2>System Monitor</h2>
 <p>This module contains sensor monitoring support.</p>
 <p>This module utilizes the following environmental variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.</p>
+<p>Monitoring these sensors is as easy as running your normal Phoronix Test Suite commands but at the beginning of the command add: MONITOR=&lt;selected sensors&gt;.  For example, this will monitor the CPU temperature and voltage during tests: 
+
+    MONITOR=cpu.temp,cpu.voltage phoronix-test-suite benchmark universe
+
+For some of the sensors there is an ability to monitor specific device, e.g. cpu.usage.cpu0 or hdd.read-speed.sda. If the PERFORMANCE_PER_WATT environment variable is set, a performance per Watt graph will also be added, assuming the system's power consumption can be monitored. PERFORMANCE_PER_SENSOR= will allow similar behavior but for arbitrary sensors. Below are all of the sensors supported by this version of the Phoronix Test Suite.
+
+Supported Options:
+
+  - all
+  - all.ambient
+  - ambient.temp
+  - all.cgroup
+  - cgroup.cpu-usage
+  - all.cpu
+  - cpu.fan-speed
+  - cpu.freq
+  - all.cpu.freq
+  - cpu.freq.cpu0
+  - cpu.freq.cpu1
+  - cpu.freq.cpu2
+  - cpu.freq.cpu3
+  - cpu.freq.cpu4
+  - cpu.freq.cpu5
+  - cpu.freq.cpu6
+  - cpu.freq.cpu7
+  - cpu.peak-freq
+  - cpu.power
+  - cpu.temp
+  - cpu.usage
+  - all.cpu.usage
+  - cpu.usage.cpu0
+  - cpu.usage.cpu1
+  - cpu.usage.cpu2
+  - cpu.usage.cpu3
+  - cpu.usage.cpu4
+  - cpu.usage.cpu5
+  - cpu.usage.cpu6
+  - cpu.usage.cpu7
+  - cpu.usage.summary
+  - cpu.voltage
+  - all.gpu
+  - gpu.fan-speed
+  - gpu.freq
+  - gpu.memory-usage
+  - gpu.power
+  - gpu.temp
+  - gpu.usage
+  - gpu.voltage
+  - all.hdd
+  - hdd.read-speed
+  - all.hdd.read-speed
+  - hdd.read-speed.sda
+  - hdd.read-speed.sdb
+  - hdd.read-speed.nvme0n1
+  - hdd.temp
+  - all.hdd.temp
+  - hdd.temp.sda
+  - hdd.temp.sdb
+  - hdd.temp.nvme0n1
+  - hdd.write-speed
+  - all.hdd.write-speed
+  - hdd.write-speed.sda
+  - hdd.write-speed.sdb
+  - hdd.write-speed.nvme0n1
+  - all.memory
+  - memory.temp
+  - memory.usage
+  - all.swap
+  - swap.usage
+  - all.sys
+  - sys.fan-speed
+  - sys.iowait
+  - sys.power
+  - sys.temp
+  - sys.voltage
+  - all.sys.voltage
+
+NOTE: Use the "system-sensors" command to see what sensors are available for monitoring on the system.</p>
+<h2>Test Timeout</h2>
+<p>This module allows killing a test if it exceeds a defined threshold, such as if the test is hung, etc. TEST_TIMEOUT_AFTER= environment variable can be used for controlling the behavior. When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit on its own for whatever reason. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run.</p>
+<p>This module utilizes the following environmental variables: TEST_TIMEOUT_AFTER.</p>
 <h2>Timed Screenshot</h2>
 <p>This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.</p>
 <p>This module utilizes the following environmental variables: SCREENSHOT_INTERVAL.</p>
 <h2>Toggle Screensaver</h2>
 <p>This module toggles the system's screensaver while the Phoronix Test Suite is running. At this time, the GNOME and KDE screensavers are supported.</p>
 <p>This module utilizes the following environmental variables: HALT_SCREENSAVER.</p>
+<h2>Linux Turbostat Dumper</h2>
+<p>Setting TURBOSTAT_LOG_DIR=_DIR_ will auto-load and enable this Phoronix Test Suite module. The module will -- if turbostat is installed on the system and the user is root -- allow dumping of the TurboStat data to the specified directly on a per-test basis. This allows easily collecting of turbostat logs for each test being run.</p>
+<p>This module utilizes the following environmental variables: TURBOSTAT_LOG_DIR.</p>
 <h2>Update Checker</h2>
-<p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.</p>
+<p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date plus also handles message of the day information.</p>
 <h2>Utilize Wine On Linux Benchmarking</h2>
 <p>This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.</p>
 <p>This module utilizes the following environmental variables: USE_WINE.</p>
 <h2>System Event Watchdog</h2>
 <p>This module has support for stopping/interrupting tests if various system issues occur, like a temperature sensor exceeds a defined threshold.</p>
 <p>This module utilizes the following environmental variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD.</p>
-<hr style="height: 50px; border: 0;"><a name="InstallationInstructions%0A"></a><h1>
-Installation Instructions
-</h1>
+<hr style="height: 50px; border: 0;"><a name="InstallationInstructions"></a><h1>Installation Instructions</h1>
 <h1>Setup Overview</h1>
 <p>The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
 <p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
@@ -698,7 +780,7 @@ Installation Instructions
 <h1>Dependencies</h1>
 <p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.</p>
 
-<p>For installing PHP on Windows, the <a href="https://www.microsoft.com/web/platform/phponwindows.aspx">Microsoft Web Platform Installer</a> provides an automated alternative that will install PHP into the directory expected by Phoronix Test Suite. Users opting to install PHP manually must extract the files to <em>C:\Program Files (x86)\PHP\</em> or a subdirectory.</p>
+<p>The <em>phoronix-test-suite.bat</em> Windows launcher for the Phoronix Test Suite will automatically download and setup PHP on the local system if PHP is not present already.</p>
 <p>As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:</p>
 <ul>
 <li><strong>PHP DOM</strong> is needed for XML operations and must be installed for the Phoronix Test Suite to function.</li>
@@ -727,33 +809,14 @@ Installation Instructions
 <p>Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the <em>phoronix-test-suite</em> executable in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in <em>/usr/share/doc/phoronix-test-suite/</em>.</p>
 <h2>Fedora / Red Hat Installation</h2>
 <p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.</p>
-<h2>Gentoo Installation</h2>
-<p>Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the <a href="http://bugs.gentoo.org/show_bug.cgi?id=216656">Gentoo BugZilla</a>.</p>
-<h2>Mandriva Installation</h2>
-<p>Phoronix Test Suite packages are available through Mandriva Cooker starting with the Mandriva Linux 2009 release.</p>
-<h2>OpenSolaris Installation</h2>
-<p>The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris platforms, but some features may not fully function. Additionally, the PTS External Dependencies feature is supported on OpenSolaris using IPS (the Image Packaging System). However, not all test profiles are supported on Solaris / OpenSolaris, and it may be in violation of your end-user agreement to perform such benchmarks on the Sun Solaris operating system. Using the OpenSolaris Image Packaging System, installing <em>SUNWphp524</em> will provide PHP, PHP GD, and related dependencies. The Phoronix Test Suite has also been tested on Oracle Solaris 11 Express, OpenIndiana, StormOS, and Nexenta Core Platform.</p>
 <h2>BSD Installation</h2>
-<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
-<p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. With DragonFlyBSD using dports, the command is <em>pkg install php5 php5-dom php5-zip php5-json php5-simplexml</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
+<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but should run well on the likes of FreeBSD and DragonFlyBSD.</p>
 <h2>MacOS Installation</h2>
 <p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
-<h2>Windows Installation</h2>
-<p>The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
-<p>At this time the Phoronix Test Suite client is dependent upon the <em>php.exe</em> executable being installed to <em>C:\Program Files (x86)\PHP\ </em> or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run <em>phoronix-test-suite.bat</em></p>
-<p>NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.</p>
-<h2>Facebook HipHop</h2>
-<p>The Phoronix Test Suite can use Facebook's <a href="https://github.com/facebook/hiphop-php">HipHop HHVM</a> virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the <em>PHP_BIN</em> environment variable can be set to point to an alternative <em>hhvm</em> binary.</p>
-<p>The Phoronix Test Suite also supports the older version of Facebook HipHop that serves as a PHP to C++ translator and compiler with GCC. This support though is primitive and not officially supported. To compile the Phoronix Test Suite using HipHop/GCC, run <em>find . -name "*.php" &gt; files.list &amp;&amp; hphp --input-list=files.list -k 1 --log=3 --include-path="." --cluster-count=50 -v "AllDynamic=true" -v "AllVolatile=true"</em> from the root <em>phoronix-test-suite/</em> directory. It can then be executed in the form of <em>/tmp/hphp_XXX/program -f pts-core/phoronix-test-suite.php system-info</em>.</p>
-<hr style="height: 50px; border: 0;"><a name="ExternalDependencies%0A"></a><h1>
-
-External Dependencies
-</h1>
+<hr style="height: 50px; border: 0;"><a name="ExternalDependencies"></a><h1>External Dependencies</h1>
 <p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
-<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.</p><hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li>
-<hr style="height: 50px; border: 0;"><a name="Configuration%0A"></a><h1>
-Configuration
-</h1>
+<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.</p><hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li>
+<hr style="height: 50px; border: 0;"><a name="Configuration"></a><h1>Configuration</h1>
 <h1>User Files &amp; Folders</h1>
 <p><strong>~/.phoronix-test-suite/user-config.xml</strong></p>
 <p>This is a per-user configuration file. Among the information stored here is the test options, locations for storing files, and batch mode options. This file is formatted in XML.</p>
@@ -772,6 +835,8 @@ Configuration
 <p><strong>~/.phoronix-test-suite/test-suites/</strong></p>
 <p>This is the directory where test suites are stored.</p>
 <h1>Environment Variables</h1>
+<p><strong>TEST_TIMEOUT_AFTER</strong></p>
+<p>When this variable is set, the value will can be set to <em>auto</em> or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit. If the value is <em>auto</em>, it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run. In the future, auto might be enabled by default in a future PTS release.</p>
 <p><strong>TEST_RESULTS_NAME</strong></p>
 <p>When this variable is set, the value will be used as the name for automatically saving the test results.</p>
 <p><strong>TEST_RESULTS_IDENTIFIER</strong></p>
@@ -838,9 +903,7 @@ Configuration
 <p>If Phodevi fails to detect the system's monitor standard / default resolution, the mode can be specified in this variable. Example: <em>DEFAULT_VIDEO_MODE=1680x1050 phoronix-test-suite benchmark nexuiz</em>.</p>
 <p><strong>SKIP_EXTERNAL_DEPENDENCIES</strong></p>
 <p>To skip the Phoronix Test Suite external dependency checking/installation when installing a test, set this environment variable to <em>1</em>. If wishing to skip only certain external dependencies, set this variable's value to the name of the external dependencies (the generic dependency names used by the Phoronix Test Suite) to not install. Multiple dependencies to skip can be delimited by a comma.</p>
-<hr style="height: 50px; border: 0;"><a name="GeneralInformation%0A"></a><h1>
-General Information
-</h1>
+<hr style="height: 50px; border: 0;"><a name="GeneralInformation"></a><h1>General Information</h1>
 <h1>Frequently Asked Questions</h1>
 <p><strong><em>Q:</em> May I use the Phoronix Test Suite when running benchmarks for my own publication or blog? Are there any publishing restrictions?</strong></p>
 <p><strong>A:</strong> Anyone is more than welcome to use the Phoronix Test Suite for their own publication or purpose. While the Phoronix Test Suite came out of our internal test tools for carrying out Linux hardware reviews at <a href="http://www.phoronix.com/">Phoronix.com</a>, we invite other hardware review web-sites, technology journals, and independent publications to use our software too. While not required, we would just kindly ask that you mention in your review/article that the <em>Phoronix Test Suite</em> was used for carrying out your testing, and ideally to link to <a href="http://www.phoronix-test-suite.com/">www.phoronix-test-suite.com</a> so that your readers will know where to obtain the software if they are interested in running the tests. You are also more than welcome to upload your results to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> so that others may compare their results against yours in an easy manner.</p>
@@ -886,72 +949,114 @@ In the event that a test profile fails to install or run, the following general
 <p>- The Phoronix Test Suite recursively determines tests/suites and allows a suite to call another suite.</p>
 <h2>Module Writing</h2>
 <p>- By writing a Phoronix Test Suite module, you can easily extend the functionality of the Phoronix Test Suite. Some examples are being able to email the results upon completion, controlling the system's screensaver, updating a text LCD panel with the current test status, etc.</p>
-<hr style="height: 50px; border: 0;"><a name="VirtualTestSuites%0A"></a><h1>
-
-Virtual Test Suites
-</h1>
+<hr style="height: 50px; border: 0;"><a name="VirtualTestSuites"></a><h1>Virtual Test Suites</h1>
 <p>Virtual test suites are not like a traditional test suite defined by the XML suite specification. Virtual test suites are dynamically generated in real-time by the Phoronix Test Suite client based upon the specified test critera. Virtual test suites can automatically consist of all test profiles that are compatible with a particular operating system or test profiles that meet other critera. When running a virtual suite, the OpenBenchmarking.org repository of the test profiles to use for generating the dynamic suite must be prefixed. </p>
 <p>Virtual test suites can be installed and run just like a normal XML test suite and shares nearly all of the same capabilities. However, when running a virtual suite, the user will be prompted to input any user-configuration options for needed test profiles just as they would need to do if running the test individually. When running a virtual suite, the user also has the ability to select individual tests within the suite to run or to run all of the contained test profiles. Virtual test suites are also only supported for an OpenBenchmarking.org repository if there is no test profile or test suite of the same name in the repository. Below is a list of common virtual test suites for the main Phoronix Test Suite repository, but the dynamic list of available virtual test suites based upon the enabled repositories is available by running <em>phoronix-test-suite list-available-virtual-suites.</em></p><hr>
-<h3>All PTS Tests<em>  pts/all</em>
+<h3>All Tests In Pts<em>  pts/all</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Application Tests<em>  pts/application</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  pts/benchmark</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  pts/blas</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
+<h3>Boost Development Libraries Tests<em>  pts/boost</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
+<h3>Bsd Operating System Tests<em>  pts/bsd</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Disk Subsystem Tests<em>  pts/disk</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
+<h3>Everything In Pts<em>  pts/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
+<h3>Fortran Tests<em>  pts/fortran</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Fortran</p>
+<h3>Game Tests<em>  pts/game</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
+<h3>Go Language Tests<em>  pts/golang</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Go Language</p>
+<h3>Graphics Subsystem Tests<em>  pts/graphics</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
 <h3>Installed Tests<em>  pts/installed</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every PTS Test<em>  pts/everything</em>
+<h3>Java Tests<em>  pts/java</em>
 </h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing java.</p>
+<h3>LAPACK (Linear Algebra Pack) Tests<em>  pts/lapack</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on LAPACK (Linear Algebra Pack)</p>
 <h3>Linux Operating System Tests<em>  pts/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  pts/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  pts/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  pts/bsd</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
+<h3>Memory Subsystem Tests<em>  pts/memory</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  pts/macosx</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the memory sub-system.</p>
+<h3>Network Subsystem Tests<em>  pts/network</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  pts/windows</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
+<h3>Node.js + NPM Tests<em>  pts/node-npm</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  pts/hurd</em>
+<p>This is a collection of test profiles having an external dependency on Node.js + NPM</p>
+<h3>OpenCV Tests<em>  pts/opencv</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  pts/system</em>
+<p>This is a collection of test profiles having an external dependency on OpenCV</p>
+<h3>Openmpi Tests<em>  pts/openmpi</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.</p>
+<h3>Os Subsystem Tests<em>  pts/os</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the os sub-system.</p>
 <h3>Processor Subsystem Tests<em>  pts/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  pts/graphics</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Python Tests<em>  pts/python</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Other Subsystem Tests<em>  pts/other</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.</p>
+<h3>Ruby Tests<em>  pts/ruby</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  pts/utility</em>
+<p>This is a collection of test profiles having an external dependency on Ruby</p>
+<h3>Rust Tests<em>  pts/rust</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  pts/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
+<p>This is a collection of test profiles having an external dependency on Rust</p>
 <h3>Scientific Tests<em>  pts/scientific</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  pts/benchmark</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
+<h3>Simulator Tests<em>  pts/simulator</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  pts/application</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a simulator software test.</p>
+<h3>Solaris Operating System Tests<em>  pts/solaris</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  pts/game</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
+<h3>System Subsystem Tests<em>  pts/system</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
+<h3>Utility Tests<em>  pts/utility</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
+<h3>Windows Operating System Tests<em>  pts/windows</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
 <h3>Smp Tests<em>  pts/smp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
+<h3>Opencl Tests<em>  pts/opencl</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
 <h3>Cuda Tests<em>  pts/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
@@ -967,9 +1072,6 @@ Virtual Test Suites
 <h3>Docker Tests<em>  pts/docker</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing docker.</p>
-<h3>Python Tests<em>  pts/python</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.</p>
 <h3>Go Tests<em>  pts/go</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.</p>
@@ -982,219 +1084,112 @@ Virtual Test Suites
 <h3>Responsiveness Tests<em>  pts/responsiveness</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.</p>
-<h3>Openmpi Tests<em>  pts/openmpi</em>
+<h3>All Tests In System<em>  system/all</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.</p>
-<h3>All SYSTEM Tests<em>  system/all</em>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Application Tests<em>  system/application</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  system/benchmark</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  system/blas</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
+<h3>Boost Development Libraries Tests<em>  system/boost</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
+<h3>Bsd Operating System Tests<em>  system/bsd</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Disk Subsystem Tests<em>  system/disk</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
+<h3>Everything In System<em>  system/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
+<h3>Game Tests<em>  system/game</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
+<h3>Graphics Subsystem Tests<em>  system/graphics</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
 <h3>Installed Tests<em>  system/installed</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every SYSTEM Test<em>  system/everything</em>
-</h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
 <h3>Linux Operating System Tests<em>  system/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  system/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  system/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  system/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  system/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  system/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  system/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  system/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
-<h3>Processor Subsystem Tests<em>  system/processor</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  system/disk</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  system/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  system/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
 <h3>Network Subsystem Tests<em>  system/network</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  system/other</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
+<h3>OpenCV Tests<em>  system/opencv</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  system/utility</em>
+<p>This is a collection of test profiles having an external dependency on OpenCV</p>
+<h3>OpenMPI Tests<em>  system/openmpi</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  system/simulator</em>
+<p>This is a collection of test profiles having an external dependency on OpenMPI</p>
+<h3>Processor Subsystem Tests<em>  system/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Python Tests<em>  system/python</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Python</p>
 <h3>Scientific Tests<em>  system/scientific</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  system/benchmark</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
+<h3>Solaris Operating System Tests<em>  system/solaris</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  system/application</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
+<h3>System Subsystem Tests<em>  system/system</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  system/game</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
+<h3>Utility Tests<em>  system/utility</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
+<h3>Windows Operating System Tests<em>  system/windows</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
 <h3>Opencl Tests<em>  system/opencl</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
 <h3>Cuda Tests<em>  system/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
-<h3>All GIT Tests<em>  git/all</em>
+<h3>Smp Tests<em>  system/smp</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>Installed Tests<em>  git/installed</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
+<h3>All Tests In Git<em>  git/all</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every GIT Test<em>  git/everything</em>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Bsd Operating System Tests<em>  git/bsd</em>
 </h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Everything In Git<em>  git/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
 <h3>Linux Operating System Tests<em>  git/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  git/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  git/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  git/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  git/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  git/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  git/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  git/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
 <h3>Processor Subsystem Tests<em>  git/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  git/disk</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Rust Tests<em>  git/rust</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  git/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  git/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
-<h3>Network Subsystem Tests<em>  git/network</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  git/other</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
+<p>This is a collection of test profiles having an external dependency on Rust</p>
 <h3>Utility Tests<em>  git/utility</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  git/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
-<h3>Scientific Tests<em>  git/scientific</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  git/benchmark</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  git/application</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  git/game</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
 <h3>Smp Tests<em>  git/smp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
-<h3>All WINDOWS Tests<em>  windows/all</em>
-</h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>Installed Tests<em>  windows/installed</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every WINDOWS Test<em>  windows/everything</em>
-</h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
-<h3>Linux Operating System Tests<em>  windows/linux</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  windows/solaris</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  windows/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  windows/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  windows/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  windows/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  windows/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
-<h3>Processor Subsystem Tests<em>  windows/processor</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  windows/disk</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  windows/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  windows/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
-<h3>Network Subsystem Tests<em>  windows/network</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  windows/other</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  windows/utility</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  windows/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
-<h3>Scientific Tests<em>  windows/scientific</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  windows/benchmark</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  windows/application</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  windows/game</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
-<hr style="height: 50px; border: 0;"><a name="ComponentTesting%0A"></a><h1>
-Component Testing
-</h1>
+<hr style="height: 50px; border: 0;"><a name="ComponentTesting"></a><h1>Component Testing</h1>
 <h1>Compiler Testing &amp; Masking</h1>
 <p>A majority of the test profiles provided by OpenBenchmarking.org to the Phoronix Test Suite are source-based tests. Relying upon the upstream source-code for each program under test allows for the tests to be easily brought to new platforms and architectures, avoids any out-of-tree / non-default packaging differences by different distributions and operating systems, and to allow the Phoronix Test Suite user to easily test new compilers and/or compiler options. For the source-based tests, the Phoronix Test Suite relies upon a compiler (e.g. GCC, LLVM/Clang, Sun Studio, Open64, et al) being present on the system under test. The Phoronix Test Suite does respect <em>CC/CXX</em> environment variables and test profiles are expected to honor <em>CFLAGS/CXXFLAGS</em> and other compiler settings.</p>
 <p>As of Phoronix Test Suite 3.8, a compiler masking method is utilized for logging compiler options and other settings with each test profile installation. Prior to installing an open-source test, the Phoronix Test Suite determines the intended compiler to be used based upon the pre-set environment variables or the pre-set compiler(s) within the <em>PATH</em> environment variable. The Phoronix Test Suite then masks the compiler to ensure that any options/flags submitted to the compiler are first passed through pts-core so that they can be logged for later use, then is linked to the original, intended compiler. Additionally, other compiler binary names of the same type are blacklisted to prevent their un-intended use (i.e. if a test profile has hard-coded <em>gcc</em> in its build script, but <em>clang</em> is set as the compiler via <em>CC</em>, a sym-link will automatically be created from <em>gcc</em> to the masked <em>clang</em> for the duration of its test installation).</p>
@@ -1204,14 +1199,10 @@ Component Testing
 <h1>Disk / File-System Testing</h1>
 <p>By default tests are installed to <em>~/.phoronix-test-suite/installed-tests/</em>. However, the location can be updated from <em>~/.phoronix-test-suite/user-config.xml</em> or dynamically via the <em>PTS_TEST_INSTALL_ROOT_PATH</em> environment variable.</p>
 <p>When any disk tests are executed, the Phoronix Test Suite will attempt to log the mount options and scheduler of the disk/partition being used for testing. This information is subsequently displayed within the system information table. If the scheduler / mount options are maintained the same throughout all test runs, only a single line is displayed otherwise the options for each test run will be automatically displayed. The file-system in use is always captured and shown in the system information table.</p>
-<hr style="height: 50px; border: 0;"><a name="Phoromatic%0A"></a><h1>
-Phoromatic
-</h1>
+<hr style="height: 50px; border: 0;"><a name="Phoromatic"></a><h1>Phoromatic</h1>
 <h1>Overview</h1>
 <p>The Phoronix Test Suite now features its own built-in Phoromatic server and client implementations. See the <em>phoromatic.pdf</em> / <em>phoromatic.html</em> documentation for more information on this advanced capability.</p>
-<hr style="height: 50px; border: 0;"><a name="OfflineImprovements+ConfidentialTesting/AvoidingAccidentalResultUploads%0A"></a><h1>
-Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads
-</h1>
+<hr style="height: 50px; border: 0;"><a name="OfflineImprovements+ConfidentialTesting/AvoidingAccidentalResultUploads"></a><h1>Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads</h1>
 <h1>Offline Enhancement Via Local Cache</h1>
 <p>Beginning with Phoronix Test Suite 9.0, there are improvements to improve the out-of-the-box experience if running the Phoronix Test Suite in a strictly offline environment / behind-the-firewall without access to OpenBenchmarking.org for being able to obtain test profiles / test suites. From Phoronix Test Suite 3.0 when OpenBenchmarking.org was introduced until Phoronix Test Suite 9.0, Internet connectivity was initially required for obtaining the test profiles/suites as the cloud/repository. OpenBenchmarking.org allows for tests to be updated independently of the Phoronix Test Suite releases as well as allowing new tests to be introduced on-demand. Aside from when new tests require explicit new PTS features, this allows tests/suites to be seamlessly used by older versions of the Phoronix Test Suite without any upgrade process required, assuming Internet connectivity is available.</p>
 <p>Beginning with Phoronix Test Suite 9.0, a static snapshot of the official tests/suites is included as part of the Phoronix Test Suite package. The intention with this is to provide a static snapshot with all tests/suites as of release time, similar to the behavior with pre-3.0 releases. The benefit to including this static snapshot is helping those that are running strictly offline/isolated to be able to have at least recent tests/suites available without first needing to query OpenBenchmarking.org for this data. But Internet support is certainly desired in order to be able to obtain updated and new test profiles.</p>
@@ -1223,79 +1214,9 @@ Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads
 <p>Removal of OpenBenchmarking.org upload support can be done by deleting <em>phoronix-test-suite/pts-core/objects/pts_openbenchmarking_upload.php</em>. If that file is removed, the Phoronix Test Suite should respond gracefully and not prompt users about any upload and within that file is the only logic for actually uploading the results to Openbenchmarking. So simply by removing that file you should be covered from any accidental uploading of results. Removal/disabling of this file also prevents any anonymous usage reporting.</p>
 <p>For those without the ability to remove that file from their Phoronix Test Suite installation or as a secondary safeguard, from the Phoronix Test Suite user configuration file (<em>/etc/phoronix-test-suite.xml<em> as root or <em>~/.phoronix-test-suite/user-config.xml<em> for most users) is a "<em>AllowResultUploadsToOpenBenchmarking</em>" option. If setting that value to <em>FALSE</em>, it should apply the same behavior as if deleting the pts_openbenchmarking_upload file.</em></em></em></em></p>
 <p>If distributing a customized/local copy of the Phoronix Test Suite, the default behavior of the configuration file (in addition to deleting the pts_openbenchmarking_upload file) can be done via the user configuration defaults defined within <em>pts-core/static/user-config-defaults.xml</em>.</p>
-<hr style="height: 50px; border: 0;"><a name="CreatingTestProfiles%0A"></a><h1>
-Creating Test Profiles
-</h1>
-<h1>Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile</h1>
-<p>A test profile itself consists of a main XML file containing the test's meta-data (<em>test-definition.xml</em>), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes (<em>downloads.xml</em>), and the <em>install.sh</em> script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like <em>pre.sh</em>, <em>interim.sh</em>, and <em>post.sh</em> for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. <em>_linux</em> or <em>_windows</em> if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.</p>
-<p>Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like <em>c-ray</em>, <em>tesseract</em> or the hundreds of others that are publicly available. Simply run for example <em>phoronix-test-suite benchmark tesseract</em> and then look at the test profile's contents either via the default locations in <em>~/.phoronix-test-suite/test-profiles</em> or <em>/var/lib/phoronix-test-suite/test-profiles/</em> if running as root on non-Windows platforms.</p>
-<h2>Versioning Mandate</h2>
-<p>The Phoronix Test Suite / OpenBenchmarking.org philosophy <em>mandates that with any change, a new version of the test profile be tagged</em>. This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a <strong>test-X.Y.Z</strong> format where <em>X</em> or <em>Y</em> are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The <em>Z</em> is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. <em>phoronix-test-suite benchmark scimark2-1.2.1</em> instead of <em>phoronix-test-suite benchmark scimark2</em>, which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.</p>
-<h2>Writing A Sample Program</h2>
-<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.</p>
-<p>The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.</p>
-<p>For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.</p>
-<blockquote>#include &lt;iostream&gt;<br>
-#include &lt;math.h&gt;
-<p>int main()<br>
-{<br>
-double pi = 0;</p>
-<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
-pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
-<p> pi *= 4;<br>
-std::cout &lt;&lt; "Done Calculating Pi..." &lt;&lt; endl;<br>
-return 0;<br>
-}</p></blockquote>
-<p>The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> and the file-name would be <em>sample-program/test-definition.xml</em>. Our (very basic) profile is showcased below.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.</p>
-<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The <em>install.sh</em> file (in our instance) is to be placed inside <em>test-profiles/sample-program</em>. Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
-<blockquote>#!/bin/sh<br><br>
-tar -xjf sample-pi-program-1.tar.bz2<br>
-g++ sample-pi-program.cpp -o sample-pi-program<br>
-echo "#!/bin/sh<br>
-./sample-pi-program 2&gt;&amp;1<br>
-" &gt; sample-program<br>
-chmod +x sample-program</blockquote>
-<p>This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for <em>sample-program</em> that covers all of this.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or <em>$LOG_FILE</em>. In the case of a test profile just measuring how long it takes to run, it is as simple as a <em>results-definition.xml</em> looking like:</p>
-<blockquote>&lt;?xml version="1.0"?&gt;<br>
-&lt;PhoronixTestSuite&gt;<br>
-  &lt;SystemMonitor&gt;<br>
-    &lt;Sensor&gt;sys.time&lt;/Sensor&gt;<br>
-  &lt;/SystemMonitor&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via <em>phoronix-test-suite benchmark sample-program</em> and then by looking at the test profile source via <em>~/.phoronix-test-suite/test-profiles/pts/sample-program*</em> or within <em>/var/lib/phoronix-test-suite/test-profiles/pts/</em> if running as root.</p>
-<hr style="height: 50px; border: 0;"><a name="DevelopmentCredits%0A"></a><h1>
-Development Credits
-</h1>
+<hr style="height: 50px; border: 0;"><a name="DevelopmentCredits"></a><h1>Development Credits</h1>
 <p>The Phoronix Test Suite is based upon the extensive testing and internal tools developed by <a href="http://www.phoronix.com/">Phoronix.com</a> since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are <a href="http://www.michaellarabel.com/">Michael Larabel</a> and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</p>
-<p style="padding-top: 30px; text-align: center;">Copyright &copy; 2008 - 2019 by Phoronix Media.</p></div>
+<p style="padding-top: 30px; text-align: center;">Copyright &copy; 2008 - 2020 by Phoronix Media.</p></div>
 		<div id="pts_footer">
 			<div id="pts_footer_logo"><a href="http://www.phoronix-test-suite.com/"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" viewbox="0 0 76 41" width="76" height="41" preserveaspectratio="xMinYMin meet">
   <path d="m74 22v9m-5-16v16m-5-28v28m-23-2h12.5c2.485281 0 4.5-2.014719 4.5-4.5s-2.014719-4.5-4.5-4.5h-8c-2.485281 0-4.5-2.014719-4.5-4.5s2.014719-4.5 4.5-4.5h12.5m-21 5h-11m11 13h-2c-4.970563 0-9-4.029437-9-9v-20m-24 40v-20c0-4.970563 4.0294373-9 9-9 4.970563 0 9 4.029437 9 9s-4.029437 9-9 9h-9" stroke="#c8d905" stroke-width="4" fill="none"></path>

documentation/phoronix-test-suite.md

@@ -4,7 +4,7 @@
 ## Overview
 The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.
 
-The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via [OpenBenchmarking.org](http://openbenchmarking.org/) integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.
+The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 400 tests available by default, which are transparently available via [OpenBenchmarking.org](http://openbenchmarking.org/) integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.
 
 Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as *phoronix-test-suite benchmark c-ray* , which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.
 
@@ -70,6 +70,9 @@ This option will permanently remove a installed test by the Phoronix Test Suite.
 #### benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.
 
+#### estimate-install-time  [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will provide estimates for test install/setup time length.
+
 #### estimate-run-time  [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will provide estimates for test run-time / length.
 
@@ -82,7 +85,10 @@ This option will run the selected test(s).
 #### run-random-tests
 This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.
 
-#### run-tests-in-suite
+#### run-subset  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the selected test(s) but prompt the user when passing any test suites or result files what subset / test(s) contained within there to run rather than running all passed tests/suites/results.
+
+#### run-tests-in-suite  [Suite]
 This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.
 
 #### stress-batch-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
@@ -158,7 +164,7 @@ This option is used for uploading a test result to OpenBenchmarking.org.
 #### upload-test-profile
 This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.
 
-#### upload-test-suite
+#### upload-test-suite  [Suite]
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 
 
@@ -166,6 +172,9 @@ This option can be used for uploading a test suite to your account on OpenBenchm
 #### info  [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
 
+#### intersect  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.
+
 #### list-all-tests
 This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
 
@@ -211,6 +220,9 @@ This option will list all available test profiles that are available from the en
 #### search
 This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..
 
+#### test-to-suite-map
+This option will list all test profiles and any test suites each test belongs to.
+
 
 ## Asset Creation
 #### build-suite
@@ -237,6 +249,9 @@ This will download the selected test file(s) to the Phoronix Test Suite download
 #### inspect-test-profile  [Test]
 This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
 
+#### rebuild-test-suite  [Suite]
+This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.
+
 #### result-file-to-suite  [Test Result]
 This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
 
@@ -246,13 +261,13 @@ This option can be used for validating a Phoronix Test Suite result file as bein
 #### validate-test-profile  [Test]
 This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.
 
-#### validate-test-suite
+#### validate-test-suite  [Suite]
 This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.
 
 
 ## Result Management
-#### analyze-all-runs  [Test Result]
-This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+#### analyze-run-times  [Test Result]
+This option will read a saved test results file and print the statistics about how long the testing took to complete.
 
 #### auto-sort-result-file  [Test Result]
 This option is used if you wish to automatically attempt to sort the results by their result identifier string.
@@ -269,6 +284,9 @@ This option is used if you wish to edit the title and description of an existing
 #### extract-from-result-file  [Test Result]
 This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.
 
+#### keep-results-in-result-file  [Test Result]
+This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.
+
 #### merge-results  [Test Result]  ...
 This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.
 
@@ -278,9 +296,12 @@ This option will re-render and save all result graphs within a saved file. This
 #### remove-result  [Test Result]
 This option will permanently remove the saved file set that is set as the first argument.
 
-#### remove-results-from-result-file  [Test Result]
+#### remove-result-from-result-file  [Test Result]
 This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.
 
+#### remove-results-from-result-file  [Test Result]
+This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to provide a string to search for in removing those results from that given result file.
+
 #### remove-run-from-result-file  [Test Result]
 This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.
 
@@ -293,15 +314,9 @@ This option is used if you wish to change the name of the saved name of a result
 #### reorder-result-file  [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 
-#### result-file-confidence  [Test Result]
-This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.
-
 #### result-file-raw-to-csv  [Test Result]
 This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
 
-#### result-file-stats  [Test Result]
-This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
-
 #### result-file-to-csv  [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 
@@ -317,8 +332,8 @@ This option will read a saved test results file and output the system hardware a
 #### show-result  [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 
-#### wins-and-losses  [Test Result]
-This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
+#### workload-topology  [Test Result]
+This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.
 
 
 ## Other
@@ -341,6 +356,23 @@ This option will display a list of available Phoronix Test Suite commands and po
 This option will display the Phoronix Test Suite client version.
 
 
+## Result Analysis
+#### analyze-all-runs  [Test Result]
+This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+
+#### executive-summary  [Test Result]
+This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.
+
+#### result-file-confidence  [Test Result]
+This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.
+
+#### result-file-stats  [Test Result]
+This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
+
+#### wins-and-losses  [Test Result]
+This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
+
+
 ## Modules
 #### auto-load-module
 This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
@@ -407,6 +439,8 @@ This is a simple module intended for developers to just demonstrate some of the
 
 phoronix-test-suite dummy_module.dummy-command
 
+This is a simple module intended for developers to just demonstrate some of the module functions.
+
 
 ### Generate Perf FlameGraphs For Tests
 Setting FLAME_GRAPH_PATH=<path to flamegraph path> will auto-load and enable this Phoronix Test Suite module. The module will generate a Linux perf FlameGraph for each test run during the benchmarking process. Details on FlameGraph @ https://github.com/brendangregg/FlameGraph
@@ -414,6 +448,12 @@ Setting FLAME_GRAPH_PATH=<path to flamegraph path> will auto-load and enable thi
 This module utilizes the following environmental variables: FLAME_GRAPH_PATH.
 
 
+### Flush Caches
+Loading this module will ensure caches (page cache, swap, etc) automatically get flushed prior to running any test.
+
+This module utilizes the following environmental variables: PTS_FLUSH_CACHES.
+
+
 ### Graphics Override
 This module allows you to override some graphics rendering settings for the ATI and NVIDIA drivers while running the Phoronix Test Suite.
 
@@ -435,6 +475,8 @@ This module utilizes the following environmental variables: LINUX_PERF.
 ### Dynamic Result Viewer
 This module pre-loads the HTTP dynamic result viewer for Phoronix Test Suite data.
 
+phoronix-test-suite load_dynamic_result_viewer.start
+
 
 ### Log Exporter
 This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.
@@ -471,6 +513,8 @@ phoronix-test-suite perf_tips.show
 
 This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.
 
+This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler: it gives a warning if BFQ is being used with an incorrect configuration in a disk benchmark, and suggests the right configuration to use. For the moment it only works for existing, throughput-based tests. It will need to be extended for responsiveness and soft real-time-latency tests.
+
 
 ### Benchmarking Compiler PGO Impact
 This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.
@@ -495,6 +539,8 @@ phoronix-test-suite phoromatic.clone
 
 phoronix-test-suite phoromatic.export-results-for-account-schedules
 
+The Phoromatic module contains the client support for interacting with Phoromatic and Phoromatic Tracker services.
+
 
 ### Pushover.net
 Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.
@@ -523,6 +569,91 @@ This module contains sensor monitoring support.
 
 This module utilizes the following environmental variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.
 
+Monitoring these sensors is as easy as running your normal Phoronix Test Suite commands but at the beginning of the command add: MONITOR=<selected sensors>.  For example, this will monitor the CPU temperature and voltage during tests: 
+
+    MONITOR=cpu.temp,cpu.voltage phoronix-test-suite benchmark universe
+
+For some of the sensors there is an ability to monitor specific device, e.g. cpu.usage.cpu0 or hdd.read-speed.sda. If the PERFORMANCE_PER_WATT environment variable is set, a performance per Watt graph will also be added, assuming the system's power consumption can be monitored. PERFORMANCE_PER_SENSOR= will allow similar behavior but for arbitrary sensors. Below are all of the sensors supported by this version of the Phoronix Test Suite.
+
+Supported Options:
+
+  - all
+  - all.ambient
+  - ambient.temp
+  - all.cgroup
+  - cgroup.cpu-usage
+  - all.cpu
+  - cpu.fan-speed
+  - cpu.freq
+  - all.cpu.freq
+  - cpu.freq.cpu0
+  - cpu.freq.cpu1
+  - cpu.freq.cpu2
+  - cpu.freq.cpu3
+  - cpu.freq.cpu4
+  - cpu.freq.cpu5
+  - cpu.freq.cpu6
+  - cpu.freq.cpu7
+  - cpu.peak-freq
+  - cpu.power
+  - cpu.temp
+  - cpu.usage
+  - all.cpu.usage
+  - cpu.usage.cpu0
+  - cpu.usage.cpu1
+  - cpu.usage.cpu2
+  - cpu.usage.cpu3
+  - cpu.usage.cpu4
+  - cpu.usage.cpu5
+  - cpu.usage.cpu6
+  - cpu.usage.cpu7
+  - cpu.usage.summary
+  - cpu.voltage
+  - all.gpu
+  - gpu.fan-speed
+  - gpu.freq
+  - gpu.memory-usage
+  - gpu.power
+  - gpu.temp
+  - gpu.usage
+  - gpu.voltage
+  - all.hdd
+  - hdd.read-speed
+  - all.hdd.read-speed
+  - hdd.read-speed.sda
+  - hdd.read-speed.sdb
+  - hdd.read-speed.nvme0n1
+  - hdd.temp
+  - all.hdd.temp
+  - hdd.temp.sda
+  - hdd.temp.sdb
+  - hdd.temp.nvme0n1
+  - hdd.write-speed
+  - all.hdd.write-speed
+  - hdd.write-speed.sda
+  - hdd.write-speed.sdb
+  - hdd.write-speed.nvme0n1
+  - all.memory
+  - memory.temp
+  - memory.usage
+  - all.swap
+  - swap.usage
+  - all.sys
+  - sys.fan-speed
+  - sys.iowait
+  - sys.power
+  - sys.temp
+  - sys.voltage
+  - all.sys.voltage
+
+NOTE: Use the "system-sensors" command to see what sensors are available for monitoring on the system.
+
+
+### Test Timeout
+This module allows killing a test if it exceeds a defined threshold, such as if the test is hung, etc. TEST_TIMEOUT_AFTER= environment variable can be used for controlling the behavior. When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit on its own for whatever reason. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run.
+
+This module utilizes the following environmental variables: TEST_TIMEOUT_AFTER.
+
 
 ### Timed Screenshot
 This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.
@@ -536,8 +667,14 @@ This module toggles the system's screensaver while the Phoronix Test Suite is ru
 This module utilizes the following environmental variables: HALT_SCREENSAVER.
 
 
+### Linux Turbostat Dumper
+Setting TURBOSTAT_LOG_DIR=_DIR_ will auto-load and enable this Phoronix Test Suite module. The module will -- if turbostat is installed on the system and the user is root -- allow dumping of the TurboStat data to the specified directly on a per-test basis. This allows easily collecting of turbostat logs for each test being run.
+
+This module utilizes the following environmental variables: TURBOSTAT_LOG_DIR.
+
+
 ### Update Checker
-This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.
+This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date plus also handles message of the day information.
 
 
 ### Utilize Wine On Linux Benchmarking
@@ -565,7 +702,7 @@ Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian
 ## Dependencies
 The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called *php5-cli* or *php-cli* or *php7* or *php* . It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at [www.php.net](http://www.php.net/) . PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.
 
-For installing PHP on Windows, the [Microsoft Web Platform Installer](https://www.microsoft.com/web/platform/phponwindows.aspx) provides an automated alternative that will install PHP into the directory expected by Phoronix Test Suite. Users opting to install PHP manually must extract the files to *C:\Program Files (x86)\PHP\* or a subdirectory.
+The *phoronix-test-suite.bat* Windows launcher for the Phoronix Test Suite will automatically download and setup PHP on the local system if PHP is not present already.
 
 As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:
 
@@ -623,42 +760,14 @@ Debian/Ubuntu users are able to follow the Generic Installation instructions or
 The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a *phoronix-test-suite* package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.
 
 
-### Gentoo Installation
-Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the [Gentoo BugZilla](http://bugs.gentoo.org/show_bug.cgi?id=216656) .
-
-
-### Mandriva Installation
-Phoronix Test Suite packages are available through Mandriva Cooker starting with the Mandriva Linux 2009 release.
-
-
-### OpenSolaris Installation
-The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris platforms, but some features may not fully function. Additionally, the PTS External Dependencies feature is supported on OpenSolaris using IPS (the Image Packaging System). However, not all test profiles are supported on Solaris / OpenSolaris, and it may be in violation of your end-user agreement to perform such benchmarks on the Sun Solaris operating system. Using the OpenSolaris Image Packaging System, installing *SUNWphp524* will provide PHP, PHP GD, and related dependencies. The Phoronix Test Suite has also been tested on Oracle Solaris 11 Express, OpenIndiana, StormOS, and Nexenta Core Platform.
-
-
 ### BSD Installation
-The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.
-
-For DragonFlyBSD, it's a matter of just using *pkg_radd* with php5, php5-json, php5-dom, and php5-zip or *pkgsrc* . With DragonFlyBSD using dports, the command is *pkg install php5 php5-dom php5-zip php5-json php5-simplexml* . Under FreeBSD/PC-BSD, PHP can be obtained from the *php5* and *php5-extensions* Ports collection.
+The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but should run well on the likes of FreeBSD and DragonFlyBSD.
 
 
 ### MacOS Installation
 The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.
 
 
-### Windows Installation
-The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.
-
-At this time the Phoronix Test Suite client is dependent upon the *php.exe* executable being installed to *C:\Program Files (x86)\PHP\ * or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run *phoronix-test-suite.bat*
-
-NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.
-
-
-### Facebook HipHop
-The Phoronix Test Suite can use Facebook's [HipHop HHVM](https://github.com/facebook/hiphop-php) virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the *PHP_BIN* environment variable can be set to point to an alternative *hhvm* binary.
-
-The Phoronix Test Suite also supports the older version of Facebook HipHop that serves as a PHP to C++ translator and compiler with GCC. This support though is primitive and not officially supported. To compile the Phoronix Test Suite using HipHop/GCC, run *find . -name "*.php" > files.list && hphp --input-list=files.list -k 1 --log=3 --include-path="." --cluster-count=50 -v "AllDynamic=true" -v "AllVolatile=true"* from the root *phoronix-test-suite/* directory. It can then be executed in the form of */tmp/hphp_XXX/program -f pts-core/phoronix-test-suite.php system-info* .
-
-
 # External Dependencies
 The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.
 
@@ -668,7 +777,7 @@ Below are a list of the operating systems that currently have external dependenc
 ---
 
 
-Alpine LinuxAmazonAngstromArch LinuxCentOSClear LinuxClearOSClearOS Core ServerDebianDragonFlyBSDFedoraFluxbuntuGNU KFreeBSDGentooGoobuntuHPJoli CloudLinaroLinux Embedded Development EnvironmentLinux MintMEPISMac OS XMacPortsMageiaMandrivaMeeGoMicrosoft WindowsMidnightBSDMoblinMythbuntuNetBSDNexenta CoreOLPCOpenIndianaOpenMandrivaOpenMandrivaLinuxOpenSolarisOpenSuSEOptwareOracle ServerPCLinuxOSPTS Desktop LivePTS Linux LivePalmPardus LinuxRed Hat EnterpriseRed Hat Enterprise ServerSUSESUSE LinuxScientificScientificSLSolusSolus LinuxTermuxUbuntuVoid LinuxZenwalkgNewSensemacOS Brew
+Alpine LinuxAmazonAngstromArch LinuxClear LinuxClearOSClearOS Core ServerDebianDragonFlyBSDFedoraFluxbuntuGNU KFreeBSDGentooGoobuntuHPJoli CloudLinaroLinux Embedded Development EnvironmentLinux MintMEPISMac OS XMacPortsMageiaMandrivaMeeGoMicrosoft WindowsMidnightBSDMoblinMythbuntuNetBSDNexenta CoreOLPCOpenIndianaOpenMandrivaOpenMandrivaLinuxOpenSolarisOpenSuSEOptwareOracle ServerPCLinuxOSPalmPardus LinuxRed Hat EnterpriseRed Hat Enterprise ServerSUSESUSE LinuxScientificScientificSLSolusSolus LinuxTermuxUbuntuVoid LinuxZenwalkgNewSensemacOS Brew
 # Configuration
 
 ## User Files & Folders
@@ -706,6 +815,10 @@ This is the directory where test suites are stored.
 
 
 ## Environment Variables
+**TEST_TIMEOUT_AFTER**
+
+When this variable is set, the value will can be set to *auto* or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit. If the value is *auto* , it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run. In the future, auto might be enabled by default in a future PTS release.
+
 **TEST_RESULTS_NAME**
 
 When this variable is set, the value will be used as the name for automatically saving the test results.
@@ -932,66 +1045,111 @@ Virtual test suites can be installed and run just like a normal XML test suite a
 
 
 ---
-#### All PTS Tests  pts/all
-This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### All Tests In Pts  pts/all
+This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.
+
+#### Application Tests  pts/application
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.
+
+#### Benchmark Tests  pts/benchmark
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.
+
+#### BLAS (Basic Linear Algebra Sub-Routine) Library Tests  pts/blas
+This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library
+
+#### Boost Development Libraries Tests  pts/boost
+This is a collection of test profiles having an external dependency on Boost Development Libraries
+
+#### Bsd Operating System Tests  pts/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.
+
+#### Disk Subsystem Tests  pts/disk
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.
+
+#### Everything In Pts  pts/everything
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.
+
+#### Fortran Tests  pts/fortran
+This is a collection of test profiles having an external dependency on Fortran
+
+#### Game Tests  pts/game
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.
+
+#### Go Language Tests  pts/golang
+This is a collection of test profiles having an external dependency on Go Language
+
+#### Graphics Subsystem Tests  pts/graphics
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.
 
 #### Installed Tests  pts/installed
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
 
-#### Every PTS Test  pts/everything
-This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.
+#### Java Tests  pts/java
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing java.
+
+#### LAPACK (Linear Algebra Pack) Tests  pts/lapack
+This is a collection of test profiles having an external dependency on LAPACK (Linear Algebra Pack)
 
 #### Linux Operating System Tests  pts/linux
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.
 
-#### Solaris Operating System Tests  pts/solaris
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
+#### Macosx Operating System Tests  pts/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.
 
-#### BSD Operating System Tests  pts/bsd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
+#### Memory Subsystem Tests  pts/memory
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the memory sub-system.
 
-#### MacOSX Operating System Tests  pts/macosx
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
+#### Network Subsystem Tests  pts/network
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.
 
-#### Windows Operating System Tests  pts/windows
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
+#### Node.js + NPM Tests  pts/node-npm
+This is a collection of test profiles having an external dependency on Node.js + NPM
 
-#### Hurd Operating System Tests  pts/hurd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
+#### OpenCV Tests  pts/opencv
+This is a collection of test profiles having an external dependency on OpenCV
 
-#### System Subsystem Tests  pts/system
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
+#### Openmpi Tests  pts/openmpi
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.
+
+#### Os Subsystem Tests  pts/os
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the os sub-system.
 
 #### Processor Subsystem Tests  pts/processor
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.
 
-#### Graphics Subsystem Tests  pts/graphics
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
+#### Python Tests  pts/python
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.
 
-#### Other Subsystem Tests  pts/other
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### Ruby Tests  pts/ruby
+This is a collection of test profiles having an external dependency on Ruby
 
-#### Utility Tests  pts/utility
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
-
-#### Simulator Tests  pts/simulator
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
+#### Rust Tests  pts/rust
+This is a collection of test profiles having an external dependency on Rust
 
 #### Scientific Tests  pts/scientific
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.
 
-#### Benchmark Tests  pts/benchmark
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
+#### Simulator Tests  pts/simulator
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a simulator software test.
 
-#### Application Tests  pts/application
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
+#### Solaris Operating System Tests  pts/solaris
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.
 
-#### Game Tests  pts/game
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+#### System Subsystem Tests  pts/system
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.
+
+#### Utility Tests  pts/utility
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.
+
+#### Windows Operating System Tests  pts/windows
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.
 
 #### Smp Tests  pts/smp
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
 
+#### Opencl Tests  pts/opencl
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.
+
 #### Cuda Tests  pts/cuda
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
 
@@ -1007,9 +1165,6 @@ This is a collection of test profiles found within the specified OpenBenchmarkin
 #### Docker Tests  pts/docker
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing docker.
 
-#### Python Tests  pts/python
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.
-
 #### Go Tests  pts/go
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.
 
@@ -1022,74 +1177,74 @@ This is a collection of test profiles found within the specified OpenBenchmarkin
 #### Responsiveness Tests  pts/responsiveness
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.
 
-#### Openmpi Tests  pts/openmpi
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.
+#### All Tests In System  system/all
+This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.
 
-#### All SYSTEM Tests  system/all
-This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### Application Tests  system/application
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.
+
+#### Benchmark Tests  system/benchmark
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.
+
+#### BLAS (Basic Linear Algebra Sub-Routine) Library Tests  system/blas
+This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library
+
+#### Boost Development Libraries Tests  system/boost
+This is a collection of test profiles having an external dependency on Boost Development Libraries
+
+#### Bsd Operating System Tests  system/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.
+
+#### Disk Subsystem Tests  system/disk
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.
+
+#### Everything In System  system/everything
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.
+
+#### Game Tests  system/game
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.
+
+#### Graphics Subsystem Tests  system/graphics
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.
 
 #### Installed Tests  system/installed
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
 
-#### Every SYSTEM Test  system/everything
-This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.
-
 #### Linux Operating System Tests  system/linux
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.
 
-#### Solaris Operating System Tests  system/solaris
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
-
-#### BSD Operating System Tests  system/bsd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
-
-#### MacOSX Operating System Tests  system/macosx
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
-
-#### Windows Operating System Tests  system/windows
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
-
-#### Hurd Operating System Tests  system/hurd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
-
-#### System Subsystem Tests  system/system
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
-
-#### Processor Subsystem Tests  system/processor
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
-
-#### Disk Subsystem Tests  system/disk
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.
-
-#### Graphics Subsystem Tests  system/graphics
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
-
-#### Memory Subsystem Tests  system/memory
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.
+#### Macosx Operating System Tests  system/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.
 
 #### Network Subsystem Tests  system/network
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.
 
-#### Other Subsystem Tests  system/other
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### OpenCV Tests  system/opencv
+This is a collection of test profiles having an external dependency on OpenCV
 
-#### Utility Tests  system/utility
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
+#### OpenMPI Tests  system/openmpi
+This is a collection of test profiles having an external dependency on OpenMPI
 
-#### Simulator Tests  system/simulator
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
+#### Processor Subsystem Tests  system/processor
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.
+
+#### Python Tests  system/python
+This is a collection of test profiles having an external dependency on Python
 
 #### Scientific Tests  system/scientific
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.
 
-#### Benchmark Tests  system/benchmark
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
+#### Solaris Operating System Tests  system/solaris
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.
 
-#### Application Tests  system/application
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
+#### System Subsystem Tests  system/system
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.
 
-#### Game Tests  system/game
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+#### Utility Tests  system/utility
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.
+
+#### Windows Operating System Tests  system/windows
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.
 
 #### Opencl Tests  system/opencl
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.
@@ -1097,141 +1252,36 @@ This is a collection of test profiles found within the specified OpenBenchmarkin
 #### Cuda Tests  system/cuda
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
 
-#### All GIT Tests  git/all
-This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### Smp Tests  system/smp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
 
-#### Installed Tests  git/installed
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
+#### All Tests In Git  git/all
+This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.
 
-#### Every GIT Test  git/everything
-This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.
+#### Bsd Operating System Tests  git/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.
+
+#### Everything In Git  git/everything
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.
 
 #### Linux Operating System Tests  git/linux
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.
 
-#### Solaris Operating System Tests  git/solaris
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
-
-#### BSD Operating System Tests  git/bsd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
-
-#### MacOSX Operating System Tests  git/macosx
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
-
-#### Windows Operating System Tests  git/windows
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
-
-#### Hurd Operating System Tests  git/hurd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
-
-#### System Subsystem Tests  git/system
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
+#### Macosx Operating System Tests  git/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.
 
 #### Processor Subsystem Tests  git/processor
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.
 
-#### Disk Subsystem Tests  git/disk
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.
-
-#### Graphics Subsystem Tests  git/graphics
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
-
-#### Memory Subsystem Tests  git/memory
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.
-
-#### Network Subsystem Tests  git/network
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.
-
-#### Other Subsystem Tests  git/other
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### Rust Tests  git/rust
+This is a collection of test profiles having an external dependency on Rust
 
 #### Utility Tests  git/utility
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
-
-#### Simulator Tests  git/simulator
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
-
-#### Scientific Tests  git/scientific
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
-
-#### Benchmark Tests  git/benchmark
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
-
-#### Application Tests  git/application
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
-
-#### Game Tests  git/game
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.
 
 #### Smp Tests  git/smp
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
 
-#### All WINDOWS Tests  windows/all
-This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
-
-#### Installed Tests  windows/installed
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
-
-#### Every WINDOWS Test  windows/everything
-This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.
-
-#### Linux Operating System Tests  windows/linux
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
-
-#### Solaris Operating System Tests  windows/solaris
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
-
-#### BSD Operating System Tests  windows/bsd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
-
-#### MacOSX Operating System Tests  windows/macosx
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
-
-#### Windows Operating System Tests  windows/windows
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
-
-#### Hurd Operating System Tests  windows/hurd
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
-
-#### System Subsystem Tests  windows/system
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
-
-#### Processor Subsystem Tests  windows/processor
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
-
-#### Disk Subsystem Tests  windows/disk
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.
-
-#### Graphics Subsystem Tests  windows/graphics
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
-
-#### Memory Subsystem Tests  windows/memory
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.
-
-#### Network Subsystem Tests  windows/network
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.
-
-#### Other Subsystem Tests  windows/other
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
-
-#### Utility Tests  windows/utility
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
-
-#### Simulator Tests  windows/simulator
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
-
-#### Scientific Tests  windows/scientific
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
-
-#### Benchmark Tests  windows/benchmark
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
-
-#### Application Tests  windows/application
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
-
-#### Game Tests  windows/game
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
-
 
 # Component Testing
 
@@ -1281,98 +1331,6 @@ For those without the ability to remove that file from their Phoronix Test Suite
 If distributing a customized/local copy of the Phoronix Test Suite, the default behavior of the configuration file (in addition to deleting the pts_openbenchmarking_upload file) can be done via the user configuration defaults defined within *pts-core/static/user-config-defaults.xml* .
 
 
-# Creating Test Profiles
-
-## Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile
-A test profile itself consists of a main XML file containing the test's meta-data ( *test-definition.xml* ), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes ( *downloads.xml* ), and the *install.sh* script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like *pre.sh* , *interim.sh* , and *post.sh* for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. *_linux* or *_windows* if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.
-
-Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like *c-ray* , *tesseract* or the hundreds of others that are publicly available. Simply run for example *phoronix-test-suite benchmark tesseract* and then look at the test profile's contents either via the default locations in *~/.phoronix-test-suite/test-profiles* or */var/lib/phoronix-test-suite/test-profiles/* if running as root on non-Windows platforms.
-
-
-### Versioning Mandate
-The Phoronix Test Suite / OpenBenchmarking.org philosophy *mandates that with any change, a new version of the test profile be tagged* . This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a **test-X.Y.Z** format where *X* or *Y* are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The *Z* is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. *phoronix-test-suite benchmark scimark2-1.2.1* instead of *phoronix-test-suite benchmark scimark2* , which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.
-
-
-### Writing A Sample Program
-Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.
-
-The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.
-
-For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.
-
-
-> #include &lt;iostream&gt;  
-> #include &lt;math.h&gt;  
-> int main()  
-> {  
-> double pi = 0;  
->  for(long int i = 1; i &lt;= 87654321; i++)  
-> pi += (double) pow(-1, i + 1) / (2 * i - 1);  
->  pi *= 4;  
-> std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;  
-> return 0;  
-> }
-
-The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it *sample-program* and the file-name would be *sample-program/test-definition.xml* . Our (very basic) profile is showcased below.
-
-
-> &lt;PhoronixTestSuite&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;  
-> &lt;/PhoronixTestSuite&gt;
-
-This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of *sample-program* , it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic *build-utilities* package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.
-
-The next step is to write the *install.sh* file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The *install.sh* file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The *install.sh* file (in our instance) is to be placed inside *test-profiles/sample-program* . Below is the *install.sh* for the *sample-program* .
-
-
-> #!/bin/sh  
-> tar -xjf sample-pi-program-1.tar.bz2  
-> g++ sample-pi-program.cpp -o sample-pi-program  
-> echo &quot;#!/bin/sh  
-> ./sample-pi-program 2&gt;&amp;1  
-> &quot; &gt; sample-program  
-> chmod +x sample-program
-
-This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the *downloads.xml* file for *sample-program* that covers all of this.
-
-
-> &lt;PhoronixTestSuite&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;  
-> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;  
-> &lt;/PhoronixTestSuite&gt;
-
-The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or *$LOG_FILE* . In the case of a test profile just measuring how long it takes to run, it is as simple as a *results-definition.xml* looking like:
-
-
-> &lt;?xml version=&quot;1.0&quot;?&gt;  
-> &lt;PhoronixTestSuite&gt;  
->   &lt;SystemMonitor&gt;  
->     &lt;Sensor&gt;sys.time&lt;/Sensor&gt;  
->   &lt;/SystemMonitor&gt;  
-> &lt;/PhoronixTestSuite&gt;
-
-After that, with all the files in their correct locations, just run: *phoronix-test-suite benchmark sample-program* . The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via *phoronix-test-suite benchmark sample-program* and then by looking at the test profile source via *~/.phoronix-test-suite/test-profiles/pts/sample-program** or within */var/lib/phoronix-test-suite/test-profiles/pts/* if running as root.
-
-
 # Development Credits
 The Phoronix Test Suite is based upon the extensive testing and internal tools developed by [Phoronix.com](http://www.phoronix.com/) since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are [Michael Larabel](http://www.michaellarabel.com/) and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by [Phoronix Media](http://www.phoronix-media.com/) .
 

documentation/stubs/00_overview.html

@@ -5,7 +5,7 @@
 <body>
 <h1>Overview</h1>
 <p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
-<p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
+<p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 400 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
 <p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
 <h1>OpenBenchmarking.org</h1>
 <p>OpenBenchmarking.org is an open, collaborative testing platform that makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite test client. OpenBenchmarking.org serves as a repository for storing test profiles, test suites, and result data. Test profiles and suites are stored in the OpenBenchmarking.org cloud to allow for new/updated tests to be seamlessly obtained via the Phoronix Test Suite without needing to manually update the Phoronix Test Suite client. OpenBenchmarking.org also makes it easy to facilitate side-by-side comparisons with any other results stored in the OpenBenchmarking.org cloud. Any Phoronix Test Suite user is permitted to upload their test results, test profiles, and suites to OpenBenchmarking.org.</p>

documentation/stubs/00_user_options.html

@@ -44,6 +44,9 @@
 <h3>benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.</p>
+<h3>estimate-install-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
+</h3>
+<p>This option will provide estimates for test install/setup time length.</p>
 <h3>estimate-run-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will provide estimates for test run-time / length.</p>
@@ -56,7 +59,10 @@
 <h3>run-random-tests<em>  </em>
 </h3>
 <p>This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.</p>
-<h3>run-tests-in-suite<em>  </em>
+<h3>run-subset<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option will run the selected test(s) but prompt the user when passing any test suites or result files what subset / test(s) contained within there to run rather than running all passed tests/suites/results.</p>
+<h3>run-tests-in-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.</p>
 <h3>stress-batch-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
@@ -130,13 +136,16 @@
 <h3>upload-test-profile<em>  </em>
 </h3>
 <p>This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.</p>
-<h3>upload-test-suite<em>  </em>
+<h3>upload-test-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.</p>
 <h1>Information</h1>
 <h3>info<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will show details about the supplied test, suite, virtual suite, or result file.</p>
+<h3>intersect<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.</p>
 <h3>list-all-tests<em>  </em>
 </h3>
 <p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
@@ -182,6 +191,9 @@
 <h3>search<em>  </em>
 </h3>
 <p>This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..</p>
+<h3>test-to-suite-map<em>  </em>
+</h3>
+<p>This option will list all test profiles and any test suites each test belongs to.</p>
 <h1>Asset Creation</h1>
 <h3>build-suite<em>  </em>
 </h3>
@@ -207,6 +219,9 @@
 <h3>inspect-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.</p>
+<h3>rebuild-test-suite<em>  [Suite]</em>
+</h3>
+<p>This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.</p>
 <h3>result-file-to-suite<em>  [Test Result]</em>
 </h3>
 <p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
@@ -216,13 +231,13 @@
 <h3>validate-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.</p>
-<h3>validate-test-suite<em>  </em>
+<h3>validate-test-suite<em>  [Suite]</em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.</p>
 <h1>Result Management</h1>
-<h3>analyze-all-runs<em>  [Test Result]</em>
+<h3>analyze-run-times<em>  [Test Result]</em>
 </h3>
-<p>This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.</p>
+<p>This option will read a saved test results file and print the statistics about how long the testing took to complete.</p>
 <h3>auto-sort-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to automatically attempt to sort the results by their result identifier string.</p>
@@ -238,6 +253,9 @@
 <h3>extract-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.</p>
+<h3>keep-results-in-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.</p>
 <h3>merge-results<em>  [Test Result]  ...</em>
 </h3>
 <p>This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.</p>
@@ -247,9 +265,12 @@
 <h3>remove-result<em>  [Test Result]</em>
 </h3>
 <p>This option will permanently remove the saved file set that is set as the first argument.</p>
-<h3>remove-results-from-result-file<em>  [Test Result]</em>
+<h3>remove-result-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.</p>
+<h3>remove-results-from-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to provide a string to search for in removing those results from that given result file.</p>
 <h3>remove-run-from-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.</p>
@@ -262,15 +283,9 @@
 <h3>reorder-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
-<h3>result-file-confidence<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
 <h3>result-file-raw-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
-<h3>result-file-stats<em>  [Test Result]</em>
-</h3>
-<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
 <h3>result-file-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.</p>
@@ -286,9 +301,9 @@
 <h3>show-result<em>  [Test Result]</em>
 </h3>
 <p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
-<h3>wins-and-losses<em>  [Test Result]</em>
+<h3>workload-topology<em>  [Test Result]</em>
 </h3>
-<p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
+<p>This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.</p>
 <h1>Other</h1>
 <h3>commands<em>  </em>
 </h3>
@@ -308,6 +323,22 @@
 <h3>version<em>  </em>
 </h3>
 <p>This option will display the Phoronix Test Suite client version.</p>
+<h1>Result Analysis</h1>
+<h3>analyze-all-runs<em>  [Test Result]</em>
+</h3>
+<p>This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.</p>
+<h3>executive-summary<em>  [Test Result]</em>
+</h3>
+<p>This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.</p>
+<h3>result-file-confidence<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
+<h3>result-file-stats<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
+<h3>wins-and-losses<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
 <h1>Modules</h1>
 <h3>auto-load-module<em>  </em>
 </h3>

documentation/stubs/00_zmodule_options.html

@@ -12,9 +12,13 @@
 <h2>Dummy Module</h2>
 <p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
 <p>phoronix-test-suite dummy_module.dummy-command</p>
+<p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
 <h2>Generate Perf FlameGraphs For Tests</h2>
 <p>Setting FLAME_GRAPH_PATH=&lt;path to flamegraph path&gt; will auto-load and enable this Phoronix Test Suite module. The module will generate a Linux perf FlameGraph for each test run during the benchmarking process. Details on FlameGraph @ https://github.com/brendangregg/FlameGraph</p>
 <p>This module utilizes the following environmental variables: FLAME_GRAPH_PATH.</p>
+<h2>Flush Caches</h2>
+<p>Loading this module will ensure caches (page cache, swap, etc) automatically get flushed prior to running any test.</p>
+<p>This module utilizes the following environmental variables: PTS_FLUSH_CACHES.</p>
 <h2>Graphics Override</h2>
 <p>This module allows you to override some graphics rendering settings for the ATI and NVIDIA drivers while running the Phoronix Test Suite.</p>
 <p>This module utilizes the following environmental variables: FORCE_AA, FORCE_AF.</p>
@@ -26,6 +30,7 @@
 <p>This module utilizes the following environmental variables: LINUX_PERF.</p>
 <h2>Dynamic Result Viewer</h2>
 <p>This module pre-loads the HTTP dynamic result viewer for Phoronix Test Suite data.</p>
+<p>phoronix-test-suite load_dynamic_result_viewer.start</p>
 <h2>Log Exporter</h2>
 <p>This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.</p>
 <p>This module utilizes the following environmental variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.</p>
@@ -44,6 +49,7 @@
 <p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler and powersave governor checks.</p>
 <p>phoronix-test-suite perf_tips.show</p>
 <p>This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.</p>
+<p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler: it gives a warning if BFQ is being used with an incorrect configuration in a disk benchmark, and suggests the right configuration to use. For the moment it only works for existing, throughput-based tests. It will need to be extended for responsiveness and soft real-time-latency tests.</p>
 <h2>Benchmarking Compiler PGO Impact</h2>
 <p>This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.</p>
 <p>phoronix-test-suite pgo.benchmark</p>
@@ -56,6 +62,7 @@
 <p>phoronix-test-suite phoromatic.list-results</p>
 <p>phoronix-test-suite phoromatic.clone</p>
 <p>phoronix-test-suite phoromatic.export-results-for-account-schedules</p>
+<p>The Phoromatic module contains the client support for interacting with Phoromatic and Phoromatic Tracker services.</p>
 <h2>Pushover.net</h2>
 <p>Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.</p>
 <p>This module utilizes the following environmental variables: PUSHOVER_NET_USER.</p>
@@ -70,14 +77,98 @@
 <h2>System Monitor</h2>
 <p>This module contains sensor monitoring support.</p>
 <p>This module utilizes the following environmental variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.</p>
+<p>Monitoring these sensors is as easy as running your normal Phoronix Test Suite commands but at the beginning of the command add: MONITOR=&lt;selected sensors&gt;.  For example, this will monitor the CPU temperature and voltage during tests: 
+
+    MONITOR=cpu.temp,cpu.voltage phoronix-test-suite benchmark universe
+
+For some of the sensors there is an ability to monitor specific device, e.g. cpu.usage.cpu0 or hdd.read-speed.sda. If the PERFORMANCE_PER_WATT environment variable is set, a performance per Watt graph will also be added, assuming the system's power consumption can be monitored. PERFORMANCE_PER_SENSOR= will allow similar behavior but for arbitrary sensors. Below are all of the sensors supported by this version of the Phoronix Test Suite.
+
+Supported Options:
+
+  - all
+  - all.ambient
+  - ambient.temp
+  - all.cgroup
+  - cgroup.cpu-usage
+  - all.cpu
+  - cpu.fan-speed
+  - cpu.freq
+  - all.cpu.freq
+  - cpu.freq.cpu0
+  - cpu.freq.cpu1
+  - cpu.freq.cpu2
+  - cpu.freq.cpu3
+  - cpu.freq.cpu4
+  - cpu.freq.cpu5
+  - cpu.freq.cpu6
+  - cpu.freq.cpu7
+  - cpu.peak-freq
+  - cpu.power
+  - cpu.temp
+  - cpu.usage
+  - all.cpu.usage
+  - cpu.usage.cpu0
+  - cpu.usage.cpu1
+  - cpu.usage.cpu2
+  - cpu.usage.cpu3
+  - cpu.usage.cpu4
+  - cpu.usage.cpu5
+  - cpu.usage.cpu6
+  - cpu.usage.cpu7
+  - cpu.usage.summary
+  - cpu.voltage
+  - all.gpu
+  - gpu.fan-speed
+  - gpu.freq
+  - gpu.memory-usage
+  - gpu.power
+  - gpu.temp
+  - gpu.usage
+  - gpu.voltage
+  - all.hdd
+  - hdd.read-speed
+  - all.hdd.read-speed
+  - hdd.read-speed.sda
+  - hdd.read-speed.sdb
+  - hdd.read-speed.nvme0n1
+  - hdd.temp
+  - all.hdd.temp
+  - hdd.temp.sda
+  - hdd.temp.sdb
+  - hdd.temp.nvme0n1
+  - hdd.write-speed
+  - all.hdd.write-speed
+  - hdd.write-speed.sda
+  - hdd.write-speed.sdb
+  - hdd.write-speed.nvme0n1
+  - all.memory
+  - memory.temp
+  - memory.usage
+  - all.swap
+  - swap.usage
+  - all.sys
+  - sys.fan-speed
+  - sys.iowait
+  - sys.power
+  - sys.temp
+  - sys.voltage
+  - all.sys.voltage
+
+NOTE: Use the "system-sensors" command to see what sensors are available for monitoring on the system.</p>
+<h2>Test Timeout</h2>
+<p>This module allows killing a test if it exceeds a defined threshold, such as if the test is hung, etc. TEST_TIMEOUT_AFTER= environment variable can be used for controlling the behavior. When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit on its own for whatever reason. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run.</p>
+<p>This module utilizes the following environmental variables: TEST_TIMEOUT_AFTER.</p>
 <h2>Timed Screenshot</h2>
 <p>This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.</p>
 <p>This module utilizes the following environmental variables: SCREENSHOT_INTERVAL.</p>
 <h2>Toggle Screensaver</h2>
 <p>This module toggles the system's screensaver while the Phoronix Test Suite is running. At this time, the GNOME and KDE screensavers are supported.</p>
 <p>This module utilizes the following environmental variables: HALT_SCREENSAVER.</p>
+<h2>Linux Turbostat Dumper</h2>
+<p>Setting TURBOSTAT_LOG_DIR=_DIR_ will auto-load and enable this Phoronix Test Suite module. The module will -- if turbostat is installed on the system and the user is root -- allow dumping of the TurboStat data to the specified directly on a per-test basis. This allows easily collecting of turbostat logs for each test being run.</p>
+<p>This module utilizes the following environmental variables: TURBOSTAT_LOG_DIR.</p>
 <h2>Update Checker</h2>
-<p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.</p>
+<p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date plus also handles message of the day information.</p>
 <h2>Utilize Wine On Linux Benchmarking</h2>
 <p>This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.</p>
 <p>This module utilizes the following environmental variables: USE_WINE.</p>

documentation/stubs/01_install.html

@@ -10,7 +10,7 @@
 <h1>Dependencies</h1>
 <p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.</p>
 
-<p>For installing PHP on Windows, the <a href="https://www.microsoft.com/web/platform/phponwindows.aspx">Microsoft Web Platform Installer</a> provides an automated alternative that will install PHP into the directory expected by Phoronix Test Suite. Users opting to install PHP manually must extract the files to <em>C:\Program Files (x86)\PHP\</em> or a subdirectory.</p>
+<p>The <em>phoronix-test-suite.bat</em> Windows launcher for the Phoronix Test Suite will automatically download and setup PHP on the local system if PHP is not present already.</p>
 <p>As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:</p>
 <ul>
 <li><strong>PHP DOM</strong> is needed for XML operations and must be installed for the Phoronix Test Suite to function.</li>
@@ -39,23 +39,9 @@
 <p>Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the <em>phoronix-test-suite</em> executable in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in <em>/usr/share/doc/phoronix-test-suite/</em>.</p>
 <h2>Fedora / Red Hat Installation</h2>
 <p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.</p>
-<h2>Gentoo Installation</h2>
-<p>Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the <a href="http://bugs.gentoo.org/show_bug.cgi?id=216656">Gentoo BugZilla</a>.</p>
-<h2>Mandriva Installation</h2>
-<p>Phoronix Test Suite packages are available through Mandriva Cooker starting with the Mandriva Linux 2009 release.</p>
-<h2>OpenSolaris Installation</h2>
-<p>The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris platforms, but some features may not fully function. Additionally, the PTS External Dependencies feature is supported on OpenSolaris using IPS (the Image Packaging System). However, not all test profiles are supported on Solaris / OpenSolaris, and it may be in violation of your end-user agreement to perform such benchmarks on the Sun Solaris operating system. Using the OpenSolaris Image Packaging System, installing <em>SUNWphp524</em> will provide PHP, PHP GD, and related dependencies. The Phoronix Test Suite has also been tested on Oracle Solaris 11 Express, OpenIndiana, StormOS, and Nexenta Core Platform.</p>
 <h2>BSD Installation</h2>
-<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
-<p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. With DragonFlyBSD using dports, the command is <em>pkg install php5 php5-dom php5-zip php5-json php5-simplexml</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
+<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but should run well on the likes of FreeBSD and DragonFlyBSD.</p>
 <h2>MacOS Installation</h2>
 <p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
-<h2>Windows Installation</h2>
-<p>The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
-<p>At this time the Phoronix Test Suite client is dependent upon the <em>php.exe</em> executable being installed to <em>C:\Program Files (x86)\PHP\ </em> or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run <em>phoronix-test-suite.bat</em></p>
-<p>NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.</p>
-<h2>Facebook HipHop</h2>
-<p>The Phoronix Test Suite can use Facebook's <a href="https://github.com/facebook/hiphop-php">HipHop HHVM</a> virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the <em>PHP_BIN</em> environment variable can be set to point to an alternative <em>hhvm</em> binary.</p>
-<p>The Phoronix Test Suite also supports the older version of Facebook HipHop that serves as a PHP to C++ translator and compiler with GCC. This support though is primitive and not officially supported. To compile the Phoronix Test Suite using HipHop/GCC, run <em>find . -name "*.php" > files.list && hphp --input-list=files.list -k 1 --log=3 --include-path="." --cluster-count=50 -v "AllDynamic=true" -v "AllVolatile=true"</em> from the root <em>phoronix-test-suite/</em> directory. It can then be executed in the form of <em>/tmp/hphp_XXX/program -f pts-core/phoronix-test-suite.php system-info</em>.</p>
 </body>
 </html>

documentation/stubs/02_external_dependencies.html

@@ -5,6 +5,6 @@
 </head>
 <body>
 <p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
-<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p>
+<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p>
 </body>
 </html>

documentation/stubs/40_configuration.html

@@ -21,6 +21,8 @@
 <p><strong>~/.phoronix-test-suite/test-suites/</strong></p>
 <p>This is the directory where test suites are stored.</p>
 <h1>Environment Variables</h1>
+<p><strong>TEST_TIMEOUT_AFTER</strong></p>
+<p>When this variable is set, the value will can be set to <em>auto</em> or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit. If the value is <em>auto</em>, it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run. In the future, auto might be enabled by default in a future PTS release.</p>
 <p><strong>TEST_RESULTS_NAME</strong></p>
 <p>When this variable is set, the value will be used as the name for automatically saving the test results.</p>
 <p><strong>TEST_RESULTS_IDENTIFIER</strong></p>

documentation/stubs/55_virtual_suites.html

@@ -6,66 +6,111 @@
 <body>
 <p>Virtual test suites are not like a traditional test suite defined by the XML suite specification. Virtual test suites are dynamically generated in real-time by the Phoronix Test Suite client based upon the specified test critera. Virtual test suites can automatically consist of all test profiles that are compatible with a particular operating system or test profiles that meet other critera. When running a virtual suite, the OpenBenchmarking.org repository of the test profiles to use for generating the dynamic suite must be prefixed. </p>
 <p>Virtual test suites can be installed and run just like a normal XML test suite and shares nearly all of the same capabilities. However, when running a virtual suite, the user will be prompted to input any user-configuration options for needed test profiles just as they would need to do if running the test individually. When running a virtual suite, the user also has the ability to select individual tests within the suite to run or to run all of the contained test profiles. Virtual test suites are also only supported for an OpenBenchmarking.org repository if there is no test profile or test suite of the same name in the repository. Below is a list of common virtual test suites for the main Phoronix Test Suite repository, but the dynamic list of available virtual test suites based upon the enabled repositories is available by running <em>phoronix-test-suite list-available-virtual-suites.</em><hr></p>
-<h3>All PTS Tests<em>  pts/all</em>
+<h3>All Tests In Pts<em>  pts/all</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Application Tests<em>  pts/application</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  pts/benchmark</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  pts/blas</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
+<h3>Boost Development Libraries Tests<em>  pts/boost</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
+<h3>Bsd Operating System Tests<em>  pts/bsd</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Disk Subsystem Tests<em>  pts/disk</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
+<h3>Everything In Pts<em>  pts/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
+<h3>Fortran Tests<em>  pts/fortran</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Fortran</p>
+<h3>Game Tests<em>  pts/game</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
+<h3>Go Language Tests<em>  pts/golang</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Go Language</p>
+<h3>Graphics Subsystem Tests<em>  pts/graphics</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
 <h3>Installed Tests<em>  pts/installed</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every PTS Test<em>  pts/everything</em>
+<h3>Java Tests<em>  pts/java</em>
 </h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing java.</p>
+<h3>LAPACK (Linear Algebra Pack) Tests<em>  pts/lapack</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on LAPACK (Linear Algebra Pack)</p>
 <h3>Linux Operating System Tests<em>  pts/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  pts/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  pts/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  pts/bsd</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
+<h3>Memory Subsystem Tests<em>  pts/memory</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  pts/macosx</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the memory sub-system.</p>
+<h3>Network Subsystem Tests<em>  pts/network</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  pts/windows</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
+<h3>Node.js + NPM Tests<em>  pts/node-npm</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  pts/hurd</em>
+<p>This is a collection of test profiles having an external dependency on Node.js + NPM</p>
+<h3>OpenCV Tests<em>  pts/opencv</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  pts/system</em>
+<p>This is a collection of test profiles having an external dependency on OpenCV</p>
+<h3>Openmpi Tests<em>  pts/openmpi</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.</p>
+<h3>Os Subsystem Tests<em>  pts/os</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the os sub-system.</p>
 <h3>Processor Subsystem Tests<em>  pts/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  pts/graphics</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Python Tests<em>  pts/python</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Other Subsystem Tests<em>  pts/other</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.</p>
+<h3>Ruby Tests<em>  pts/ruby</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  pts/utility</em>
+<p>This is a collection of test profiles having an external dependency on Ruby</p>
+<h3>Rust Tests<em>  pts/rust</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  pts/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
+<p>This is a collection of test profiles having an external dependency on Rust</p>
 <h3>Scientific Tests<em>  pts/scientific</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  pts/benchmark</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
+<h3>Simulator Tests<em>  pts/simulator</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  pts/application</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a simulator software test.</p>
+<h3>Solaris Operating System Tests<em>  pts/solaris</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  pts/game</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
+<h3>System Subsystem Tests<em>  pts/system</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
+<h3>Utility Tests<em>  pts/utility</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
+<h3>Windows Operating System Tests<em>  pts/windows</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
 <h3>Smp Tests<em>  pts/smp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
+<h3>Opencl Tests<em>  pts/opencl</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
 <h3>Cuda Tests<em>  pts/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
@@ -81,9 +126,6 @@
 <h3>Docker Tests<em>  pts/docker</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing docker.</p>
-<h3>Python Tests<em>  pts/python</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.</p>
 <h3>Go Tests<em>  pts/go</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.</p>
@@ -96,215 +138,110 @@
 <h3>Responsiveness Tests<em>  pts/responsiveness</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.</p>
-<h3>Openmpi Tests<em>  pts/openmpi</em>
+<h3>All Tests In System<em>  system/all</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.</p>
-<h3>All SYSTEM Tests<em>  system/all</em>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Application Tests<em>  system/application</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  system/benchmark</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  system/blas</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
+<h3>Boost Development Libraries Tests<em>  system/boost</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
+<h3>Bsd Operating System Tests<em>  system/bsd</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Disk Subsystem Tests<em>  system/disk</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
+<h3>Everything In System<em>  system/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
+<h3>Game Tests<em>  system/game</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
+<h3>Graphics Subsystem Tests<em>  system/graphics</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
 <h3>Installed Tests<em>  system/installed</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every SYSTEM Test<em>  system/everything</em>
-</h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
 <h3>Linux Operating System Tests<em>  system/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  system/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  system/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  system/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  system/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  system/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  system/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  system/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
-<h3>Processor Subsystem Tests<em>  system/processor</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  system/disk</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  system/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  system/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
 <h3>Network Subsystem Tests<em>  system/network</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  system/other</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
+<h3>OpenCV Tests<em>  system/opencv</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  system/utility</em>
+<p>This is a collection of test profiles having an external dependency on OpenCV</p>
+<h3>OpenMPI Tests<em>  system/openmpi</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  system/simulator</em>
+<p>This is a collection of test profiles having an external dependency on OpenMPI</p>
+<h3>Processor Subsystem Tests<em>  system/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Python Tests<em>  system/python</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Python</p>
 <h3>Scientific Tests<em>  system/scientific</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  system/benchmark</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
+<h3>Solaris Operating System Tests<em>  system/solaris</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  system/application</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
+<h3>System Subsystem Tests<em>  system/system</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  system/game</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
+<h3>Utility Tests<em>  system/utility</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
+<h3>Windows Operating System Tests<em>  system/windows</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
 <h3>Opencl Tests<em>  system/opencl</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
 <h3>Cuda Tests<em>  system/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
-<h3>All GIT Tests<em>  git/all</em>
+<h3>Smp Tests<em>  system/smp</em>
 </h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>Installed Tests<em>  git/installed</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
+<h3>All Tests In Git<em>  git/all</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every GIT Test<em>  git/everything</em>
+<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
+<h3>Bsd Operating System Tests<em>  git/bsd</em>
 </h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
+<h3>Everything In Git<em>  git/everything</em>
+</h3>
+<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
 <h3>Linux Operating System Tests<em>  git/linux</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  git/solaris</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
+<h3>Macosx Operating System Tests<em>  git/macosx</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  git/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  git/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  git/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  git/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  git/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
 <h3>Processor Subsystem Tests<em>  git/processor</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  git/disk</em>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
+<h3>Rust Tests<em>  git/rust</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  git/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  git/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
-<h3>Network Subsystem Tests<em>  git/network</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  git/other</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
+<p>This is a collection of test profiles having an external dependency on Rust</p>
 <h3>Utility Tests<em>  git/utility</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  git/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
-<h3>Scientific Tests<em>  git/scientific</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  git/benchmark</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  git/application</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  git/game</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
 <h3>Smp Tests<em>  git/smp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
-<h3>All WINDOWS Tests<em>  windows/all</em>
-</h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>Installed Tests<em>  windows/installed</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>Every WINDOWS Test<em>  windows/everything</em>
-</h3>
-<p>This is a collection of every test profile found within the specified OpenBenchmarking.org repository, including unsupported tests.</p>
-<h3>Linux Operating System Tests<em>  windows/linux</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.</p>
-<h3>Solaris Operating System Tests<em>  windows/solaris</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.</p>
-<h3>BSD Operating System Tests<em>  windows/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.</p>
-<h3>MacOSX Operating System Tests<em>  windows/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.</p>
-<h3>Windows Operating System Tests<em>  windows/windows</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.</p>
-<h3>Hurd Operating System Tests<em>  windows/hurd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.</p>
-<h3>System Subsystem Tests<em>  windows/system</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.</p>
-<h3>Processor Subsystem Tests<em>  windows/processor</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.</p>
-<h3>Disk Subsystem Tests<em>  windows/disk</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.</p>
-<h3>Graphics Subsystem Tests<em>  windows/graphics</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.</p>
-<h3>Memory Subsystem Tests<em>  windows/memory</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.</p>
-<h3>Network Subsystem Tests<em>  windows/network</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.</p>
-<h3>Other Subsystem Tests<em>  windows/other</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.</p>
-<h3>Utility Tests<em>  windows/utility</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.</p>
-<h3>Simulator Tests<em>  windows/simulator</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.</p>
-<h3>Scientific Tests<em>  windows/scientific</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.</p>
-<h3>Benchmark Tests<em>  windows/benchmark</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.</p>
-<h3>Application Tests<em>  windows/application</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.</p>
-<h3>Game Tests<em>  windows/game</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.</p>
 </body>
 </html>

documentation/stubs/69_creating_tests.html

@@ -1,72 +0,0 @@
-<html>
-<head>
-<Title>Creating Test Profiles</Title>
-</head>
-<body>
-<h1>Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile</h1>
-<p>A test profile itself consists of a main XML file containing the test's meta-data (<em>test-definition.xml</em>), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes (<em>downloads.xml</em>), and the <em>install.sh</em> script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like <em>pre.sh</em>, <em>interim.sh</em>, and <em>post.sh</em> for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. <em>_linux</em> or <em>_windows</em> if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.</p>
-<p>Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like <em>c-ray</em>, <em>tesseract</em> or the hundreds of others that are publicly available. Simply run for example <em>phoronix-test-suite benchmark tesseract</em> and then look at the test profile's contents either via the default locations in <em>~/.phoronix-test-suite/test-profiles</em> or <em>/var/lib/phoronix-test-suite/test-profiles/</em> if running as root on non-Windows platforms.</p>
-<h2>Versioning Mandate</h2>
-<p>The Phoronix Test Suite / OpenBenchmarking.org philosophy <em>mandates that with any change, a new version of the test profile be tagged</em>. This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a <strong>test-X.Y.Z</strong> format where <em>X</em> or <em>Y</em> are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The <em>Z</em> is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. <em>phoronix-test-suite benchmark scimark2-1.2.1</em> instead of <em>phoronix-test-suite benchmark scimark2</em>, which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.</p>
-<h2>Writing A Sample Program</h2>
-<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.</p>
-<p>The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.</p>
-<p>For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.</p>
-<blockquote>#include &lt;iostream&gt;<br>
-#include &lt;math.h&gt;</p>
-<p>int main()<br>
-{<br>
-double pi = 0;</p>
-<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
-pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
-<p> pi *= 4;<br>
-std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;<br>
-return 0;<br>
-}</blockquote>
-<p>The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> and the file-name would be <em>sample-program/test-definition.xml</em>. Our (very basic) profile is showcased below.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.</p>
-<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The <em>install.sh</em> file (in our instance) is to be placed inside <em>test-profiles/sample-program</em>. Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
-<blockquote>#!/bin/sh<br /><br />
-tar -xjf sample-pi-program-1.tar.bz2<br />
-g++ sample-pi-program.cpp -o sample-pi-program<br />
-echo &quot;#!/bin/sh<br />
-./sample-pi-program 2&gt;&amp;1<br />
-&quot; &gt; sample-program<br>
-chmod +x sample-program</blockquote>
-<p>This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for <em>sample-program</em> that covers all of this.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or <em>$LOG_FILE</em>. In the case of a test profile just measuring how long it takes to run, it is as simple as a <em>results-definition.xml</em> looking like:</p>
-<blockquote>&lt;?xml version="1.0"?&gt;<br>
-&lt;PhoronixTestSuite&gt;<br>
-  &lt;SystemMonitor&gt;<br>
-    &lt;Sensor&gt;sys.time&lt;/Sensor&gt;<br>
-  &lt;/SystemMonitor&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via <em>phoronix-test-suite benchmark sample-program</em> and then by looking at the test profile source via <em>~/.phoronix-test-suite/test-profiles/pts/sample-program*</em> or within <em>/var/lib/phoronix-test-suite/test-profiles/pts/</em> if running as root.</p>
-</body>
-</html>

install.bat

@@ -2,7 +2,7 @@
 
 :: Phoronix Test Suite
 :: URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-:: Copyright (C) 2018, Phoronix Media
+:: Copyright (C) 2018 - 2020, Phoronix Media
 ::
 :: This program is free software; you can redistribute it and/or modify
 :: it under the terms of the GNU General Public License as published by
@@ -29,5 +29,5 @@ md %destination%
 :: cd /d %destination%
 :: for /F "delims=" %%i in ('dir /b') do (rmdir "%%i" /s/q || del "%%i" /s/q)
 
-xcopy %cd% %destination% /E
+xcopy "%cd%" %destination% /E
 echo Phoronix Test Suite installed to %destination%

ob-cache/openbenchmarking.org/git.index

@@ -1 +1 @@
-{"main":{"generated":1576104549},"tests":{"aom-av1":{"title":"AOM AV1","description":"This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"684","last_updated":"1552063091","first_added":"1552063091","popularity":"988","downloads":"1516","versions":["1.1.0"]},"svt-av1":{"title":"SVT-AV1","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-AV1 CPU-based multi-threaded video encoder for the AV1 video format with a sample 1080p YUV video file. This test profile fork builds the encoder from Git source rather than a snapshot.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"120","last_updated":"1552059151","first_added":"1552059151","popularity":"981","downloads":"1513","versions":["1.2.0"]},"svt-hevc":{"title":"SVT-HEVC","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-HEVC CPU-based multi-threaded video encoder for the HEVC \/ H.265 video format with a sample 1080p YUV video file. This test uses SVT-HEVC from Git master.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"56","last_updated":"1552060068","first_added":"1552060068","popularity":"986","downloads":"1505","versions":["1.0.0"]},"svt-vp9":{"title":"SVT-VP9","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-VP9 CPU-based multi-threaded video encoder for the VP9 video format with a sample 1080p YUV video file. This test profile uses the Git snapshot of SVT-VP9.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"50","last_updated":"1552059681","first_added":"1552059681","popularity":"985","downloads":"1503","versions":["1.0.0"]},"vpxenc":{"title":"VP9 libvpx Encoding","description":"This is a standard video encoding performance test of Google's libvpx library and the vpxenc command for the VP9\/WebM format using a sample 1080p video.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"773","last_updated":"1552061773","first_added":"1552061773","popularity":"993","downloads":"1486","versions":["2.3.0"]},"x265":{"title":"x265","description":"This is a simple test of the x265 encoder run on the CPU with a sample 1080p video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"118","last_updated":"1552061034","first_added":"1552061034","popularity":"975","downloads":"1552","versions":["1.1.0"]}},"suites":[]}
+{"main":{"generated":1603296740},"tests":{"aom-av1":{"title":"AOM AV1","description":"This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"771","average_install_time":"179","last_updated":"1552063091","first_added":"1552063091","popularity":"1299","downloads":"1897","versions":["1.1.0"]},"dav1d":{"title":"dav1d","description":"Dav1d is an open-source, speedy AV1 video decoder. This test profile times how long it takes to decode sample AV1 video content.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"914","average_install_time":"181","last_updated":"1579721829","first_added":"1579697045","popularity":"1093","downloads":2343,"versions":["1.0.1","1.0.0"]},"rav1e":{"title":"rav1e","description":"Xiph rav1e is a Rust-written AV1 video encoder.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"698","average_install_time":"264","last_updated":"1578155433","first_added":"1578155433","popularity":"1043","downloads":"2334","versions":["1.0.0"]},"svt-av1":{"title":"SVT-AV1","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-AV1 CPU-based multi-threaded video encoder for the AV1 video format with a sample 1080p YUV video file. This test profile fork builds the encoder from Git source rather than a snapshot.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"157","average_install_time":"80","last_updated":"1552059151","first_added":"1552059151","popularity":"1287","downloads":"1893","versions":["1.2.0"]},"svt-hevc":{"title":"SVT-HEVC","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-HEVC CPU-based multi-threaded video encoder for the HEVC \/ H.265 video format with a sample 1080p YUV video file. This test uses SVT-HEVC from Git master.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"55","average_install_time":"53","last_updated":"1552060068","first_added":"1552060068","popularity":"1282","downloads":"1889","versions":["1.0.0"]},"svt-vp9":{"title":"SVT-VP9","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-VP9 CPU-based multi-threaded video encoder for the VP9 video format with a sample 1080p YUV video file. This test profile uses the Git snapshot of SVT-VP9.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"42","average_install_time":"50","last_updated":"1552059681","first_added":"1552059681","popularity":"1291","downloads":"1882","versions":["1.0.0"]},"vpxenc":{"title":"VP9 libvpx Encoding","description":"This is a standard video encoding performance test of Google's libvpx library and the vpxenc command for the VP9\/WebM format using a sample 1080p video.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"747","average_install_time":"75","last_updated":"1552061773","first_added":"1552061773","popularity":"1295","downloads":"1865","versions":["2.3.0"]},"x265":{"title":"x265","description":"This is a simple test of the x265 encoder run on the CPU with a sample 1080p video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"133","average_install_time":"187","last_updated":"1552061034","first_added":"1552061034","popularity":"1271","downloads":"1941","versions":["1.1.0"]}},"suites":[]}

ob-cache/test-profiles/git/dav1d-1.0.1/downloads.xml

@@ -0,0 +1,42 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.0m1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ffmpeg.org/releases/ffmpeg-4.2.1.tar.bz2</URL>
+      <MD5>f3cba622990c8ac8a57330f571db1230</MD5>
+      <SHA256>682a9fa3f6864d7f0dbf224f86b129e337bc60286e0d00dffcd710998d521624</SHA256>
+      <FileName>ffmpeg-4.2.1.tar.bz2</FileName>
+      <FileSize>10953293</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.elecard.com/storage/video/Stream2_AV1_HD_6.8mbps.webm</URL>
+      <MD5>5e4c83cf494cbe6dc1668cbbd7ff774c</MD5>
+      <SHA256>2f23d29750a0663a6df656e8137cf934bddfc96b31e5088db2c3624f19ed14d4</SHA256>
+      <FileName>Stream2_AV1_HD_6.8mbps.webm</FileName>
+      <FileSize>122378926</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.elecard.com/storage/video/Stream2_AV1_4K_22.7mbps.webm</URL>
+      <MD5>8acc9c60c10a37bf4e568b465b03e35a</MD5>
+      <SHA256>52f3aa1d4b4487af62d37b0f295aabbc4b57f03fdc4c76402c6358193e4aa490</SHA256>
+      <FileName>Stream2_AV1_4K_22.7mbps.webm</FileName>
+      <FileSize>409458359</FileSize>
+    </Package>
+    <Package>
+      <URL>http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/Old/Chimera-AV1-8bit-1920x1080-6736kbps.mp4</URL>
+      <MD5>d444e11f8b049de2a9739899a9c0b4f6</MD5>
+      <SHA256>d566d294e2c18bb274a54aad03352c92312a62c393656d38e1f7dda10c0bf10c</SHA256>
+      <FileName>Chimera-AV1-8bit-1920x1080-6736kbps.mp4</FileName>
+      <FileSize>313490236</FileSize>
+    </Package>
+    <Package>
+      <URL>http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/Old/Chimera-AV1-10bit-1920x1080-6191kbps.mp4</URL>
+      <MD5>4c0f24bac42ddcda3d46087a36492ce4</MD5>
+      <SHA256>df2080fd77e0dbd9138bd4f172bf008d2ade17da7ab4532fba54ceccf40a9439</SHA256>
+      <FileName>Chimera-AV1-10bit-1920x1080-6191kbps.mp4</FileName>
+      <FileSize>288122748</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/dav1d-1.0.1/install.sh

@@ -0,0 +1,37 @@
+#!/bin/sh
+
+# FFmpeg install to demux AV1 WebM to IVF that can then be consumed by dav1d...
+tar -xjf ffmpeg-4.2.1.tar.bz2
+mkdir ffmpeg_/
+
+cd ffmpeg-4.2.1/
+./configure --disable-zlib --disable-doc --prefix=$HOME/ffmpeg_/
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+make install
+cd ~/
+
+./ffmpeg_/bin/ffmpeg -i Stream2_AV1_HD_6.8mbps.webm -vcodec copy -an -f ivf summer_nature_1080p.ivf
+./ffmpeg_/bin/ffmpeg -i Stream2_AV1_4K_22.7mbps.webm -vcodec copy -an -f ivf summer_nature_4k.ivf
+./ffmpeg_/bin/ffmpeg -i Chimera-AV1-8bit-1920x1080-6736kbps.mp4 -vcodec copy -an -f ivf chimera_8b_1080p.ivf
+./ffmpeg_/bin/ffmpeg -i Chimera-AV1-10bit-1920x1080-6191kbps.mp4 -vcodec copy -an -f ivf chimera_10b_1080p.ivf
+
+rm -rf ffmpeg-4.2.1
+rm -rf ffmpeg_
+
+# Build Dav1d
+rm -rf dav1d-git
+git clone https://code.videolan.org/videolan/dav1d.git dav1d-git
+cd dav1d-git
+git checkout e79e5ceb2cb74fc466e2868c4725d98ccca7cac7
+git rev-parse --short HEAD > ~/install-footnote
+mkdir build
+meson build --buildtype release
+ninja -C build
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+./dav1d-git/build/tools/dav1d \$@ --muxer null --framethreads \$NUM_CPU_CORES --tilethreads 4 --filmgrain 0 > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > dav1d
+chmod +x dav1d

ob-cache/test-profiles/git/dav1d-1.0.1/results-definition.xml

@@ -0,0 +1,10 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.0m1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Decoded 8929 8929 frames (100.0%) - #_RESULT_# 25 fps (9.20x)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <MultiMatch>AVERAGE</MultiMatch>
+    <TurnCharsToSpace>/</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/dav1d-1.0.1/test-definition.xml

@@ -0,0 +1,53 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.0m1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>dav1d</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>Dav1d is an open-source, speedy AV1 video decoder. This test profile times how long it takes to decode sample AV1 video content.</Description>
+    <ResultScale>FPS</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.1</Version>
+    <SupportedPlatforms>Linux, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, yasm, meson, git</ExternalDependencies>
+    <EnvironmentSize>1200</EnvironmentSize>
+    <ProjectURL>http://code.videolan.org/videolan/dav1d</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments> </Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Video Input</DisplayName>
+      <Identifier>video</Identifier>
+      <ArgumentPrefix>-i </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>Summer Nature 1080p</Name>
+          <Value>summer_nature_1080p.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Summer Nature 4K</Name>
+          <Value>summer_nature_4k.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Chimera 1080p</Name>
+          <Value>chimera_8b_1080p.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Chimera 1080p 10-bit</Name>
+          <Value>chimera_10b_1080p.ivf</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/rav1e-1.0.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/rav1e-1.0.0/install.sh

@@ -0,0 +1,20 @@
+#!/bin/sh
+
+rm -rf rav1e-master
+git clone https://github.com/xiph/rav1e.git rav1e-master
+cargo build --release
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+cd rav1e-master
+cargo build --bin rav1e --release -j $NUM_CPU_PHYSICAL_CORES
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+./rav1e-master/target/release/rav1e ./Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m --threads \$NUM_CPU_CORES --tiles 4 --output /dev/null \$@ > log.out 2>&1
+echo \$? > ~/test-exit-status
+tr -s '\r' '\n' < log.out > \$LOG_FILE" > rav1e
+chmod +x rav1e

ob-cache/test-profiles/git/rav1e-1.0.0/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>XX  encoded 20/20 frames, #_RESULT_# fps, 2844.02 Kb/s, est. size: 0.23 MB, est. time: 0s</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/rav1e-1.0.0/test-definition.xml

@@ -0,0 +1,54 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>rav1e</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>Xiph rav1e is a Rust-written AV1 video encoder.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>1080p To AV1 Video Encode</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>p7zip, rust, yasm, perl, cmake, git</ExternalDependencies>
+    <EnvironmentSize>3000</EnvironmentSize>
+    <ProjectURL>https://github.com/xiph/rav1e</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Option>
+      <DisplayName>Speed</DisplayName>
+      <Identifier>s</Identifier>
+      <Menu>
+        <Entry>
+          <Name>9</Name>
+          <Value>-s 9 -l 80</Value>
+          <Message>Fastest</Message>
+        </Entry>
+        <Entry>
+          <Name>6</Name>
+          <Value>-s 6 -l 60</Value>
+          <Message>Default</Message>
+        </Entry>
+        <Entry>
+          <Name>5</Name>
+          <Value>-s 5 -l 60</Value>
+          <Message>Mid-Speed</Message>
+        </Entry>
+        <Entry>
+          <Name>1</Name>
+          <Value>-s 1 -l 20</Value>
+          <Message>Slowest / Best Quality</Message>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/install.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.2.0
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+	echo 2 > ~/install-exit-status
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/.local/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.8.0m2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Device Inference Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Inference Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device Training Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Training Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device AI Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device AI Score</ArgumentsDescription>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/test-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.8.0m2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AI Benchmark Alpha</Title>
+    <AppVersion>0.1.2</AppVersion>
+    <Description>AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library.</Description>
+    <ResultScale>Score</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>1</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>python</ExternalDependencies>
+    <RequiresInternet>TRUE</RequiresInternet>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>http://ai-benchmark.com/alpha</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.2/downloads.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://fsbench.filesystems.org/bench/aio-stress.c</URL>
+      <MD5>727e5fc3566837b3ea72f887d048769b</MD5>
+      <FileName>aio-stress.c</FileName>
+      <FileSize>39513</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.2/install.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+
+cc -Wall -O3 -pthread -o aio-stress-bin aio-stress.c -laio
+echo $? > ~/install-exit-status
+
+# add support for allowing aio-test-file to be on removable media devices
+echo "#!/bin/sh
+./aio-stress-bin \$@ aio-test-file > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status
+rm -f aio-test-file" > aio-stress
+chmod +x aio-stress

ob-cache/test-profiles/pts/aio-stress-1.1.2/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>adding stage random write
+dropping thread count to the number of contexts 1
+starting with random write
+file size 1024MB, record size 64KB, depth 64, ios per iteration 8
+max io_submit 8, buffer alignment set to 4KB
+threads 1 files 1 contexts 1 context offset 2MB verification off
+random write on aio-test-file (#_RESULT_# MB/s) 1024.00 MB in 3.84s
+thread 0 random write totals (63.36 MB/s) 1024.00 MB in 16.16s
+Running single thread version</OutputTemplate>
+    <LineHint>random write on aio-test-file</LineHint>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.2/test-definition.xml

@@ -0,0 +1,40 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AIO-Stress</Title>
+    <AppVersion>0.21</AppVersion>
+    <Description>AIO-Stress is an a-synchronous I/O benchmark created by SuSE. Current this profile uses a 2048MB test file and a 64KB record size.</Description>
+    <ResultScale>MB/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.2</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Benchmark</SoftwareType>
+    <TestType>Disk</TestType>
+    <License>Free</License>
+    <Status>Deprecated</Status>
+    <ExternalDependencies>build-utilities, libaio-development</ExternalDependencies>
+    <EnvironmentSize>0.5</EnvironmentSize>
+    <EnvironmentTestingSize>1024</EnvironmentTestingSize>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-s 2g -r 64k -t 3</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Test</DisplayName>
+      <Identifier>test</Identifier>
+      <ArgumentPrefix>-o </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>Random Write</Name>
+          <Value>2</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.1/downloads.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>https://download.aircrack-ng.org/aircrack-ng-1.3.tar.gz</URL>
+      <MD5>c7c5b076dee0c25ee580b0f56f455623</MD5>
+      <SHA256>8ae08a7c28741f6ace2769267112053366550e7f746477081188ad38410383ca</SHA256>
+      <FileName>aircrack-ng-1.3.tar.gz</FileName>
+      <FileSize>4501507</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>https://download.aircrack-ng.org/aircrack-ng-1.3-win.zip</URL>
+      <MD5>319913c6e5e11aa91788ca07df40f250</MD5>
+      <SHA256>cd5f8dc621211b87e6e73b24e4d3f83ce8da7979f14bd439bd933b04be6fac2c</SHA256>
+      <FileName>aircrack-ng-1.3-win.zip</FileName>
+      <FileSize>14043368</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.1/install.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+tar -xf aircrack-ng-1.3.tar.gz
+cd aircrack-ng-1.3
+./autogen.sh 
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd aircrack-ng-1.3
+./src/aircrack-ng -p \$NUM_CPU_CORES \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.1.1/install_windows.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+unzip -o aircrack-ng-1.3-win.zip
+
+echo "#!/bin/sh
+cd aircrack-ng-1.3-win
+./bin/64bit/aircrack-ng.exe \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.1.1/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>#_RESULT_# k/s</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.1/test-definition.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Aircrack-ng</Title>
+    <AppVersion>1.3</AppVersion>
+    <Description>Aircrack-ng is a tool for assessing WiFi/WLAN network security.</Description>
+    <ResultScale>k/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.1</Version>
+    <SupportedPlatforms>Linux, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, openssl-development, libtool</ExternalDependencies>
+    <EnvironmentSize>25</EnvironmentSize>
+    <ProjectURL>http://www.aircrack-ng.org/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-S -Z 30</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.0/downloads.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://download.aircrack-ng.org/aircrack-ng-1.5.2.tar.gz</URL>
+      <MD5>2648c192d206e953c67dca64967d2982</MD5>
+      <SHA256>9e592fe7658046220e0ac0a6d05c4026903f3077b248893e0056ccbe4ee88241</SHA256>
+      <FileName>aircrack-ng-1.5.2.tar.gz</FileName>
+      <FileSize>7138360</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://download.aircrack-ng.org/aircrack-ng-1.5.2-win.zip</URL>
+      <MD5>4c705429ca51e79b490f1dca863d2217</MD5>
+      <SHA256>61a91b44918eb196f06f46a3dbc359ddeced5712095c814fabc473b25a1c2e44</SHA256>
+      <FileName>aircrack-ng-1.5.2-win.zip</FileName>
+      <FileSize>16903422</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.0/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+tar -xf aircrack-ng-1.5.2.tar.gz
+cd aircrack-ng-1.5.2
+if [ "$OS_TYPE" = "BSD" ]
+then
+	if [ -e /usr/local/lib/libcrypto.so ]
+	then
+		env MAKE=gmake CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./autogen.sh
+	else
+		env MAKE=gmake ./autogen.sh
+	fi
+	gmake -j $NUM_CPU_CORES
+else
+	./autogen.sh
+	make -j $NUM_CPU_CORES
+fi
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd aircrack-ng-1.5.2
+./src/aircrack-ng -p \$NUM_CPU_CORES \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.2.0/install_windows.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+unzip -o aircrack-ng-1.5.2-win.zip
+
+echo "#!/bin/sh
+cd aircrack-ng-1.5.2-win
+./bin/64bit/aircrack-ng.exe \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.2.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>#_RESULT_# k/s</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.0/test-definition.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Aircrack-ng</Title>
+    <AppVersion>1.5.2</AppVersion>
+    <Description>Aircrack-ng is a tool for assessing WiFi/WLAN network security.</Description>
+    <ResultScale>k/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.2.0</Version>
+    <SupportedPlatforms>Linux, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, openssl-development, libtool</ExternalDependencies>
+    <EnvironmentSize>25</EnvironmentSize>
+    <ProjectURL>http://www.aircrack-ng.org/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-S -Z 30</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.1/downloads.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://download.aircrack-ng.org/aircrack-ng-1.5.2.tar.gz</URL>
+      <MD5>2648c192d206e953c67dca64967d2982</MD5>
+      <SHA256>9e592fe7658046220e0ac0a6d05c4026903f3077b248893e0056ccbe4ee88241</SHA256>
+      <FileName>aircrack-ng-1.5.2.tar.gz</FileName>
+      <FileSize>7138360</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://download.aircrack-ng.org/aircrack-ng-1.5.2-win.zip</URL>
+      <MD5>4c705429ca51e79b490f1dca863d2217</MD5>
+      <SHA256>61a91b44918eb196f06f46a3dbc359ddeced5712095c814fabc473b25a1c2e44</SHA256>
+      <FileName>aircrack-ng-1.5.2-win.zip</FileName>
+      <FileSize>16903422</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.1/install.sh

@@ -0,0 +1,26 @@
+#!/bin/sh
+tar -xf aircrack-ng-1.5.2.tar.gz
+cd aircrack-ng-1.5.2
+export CFLAGS="-O3 -fcommon"
+export CXXFLAGS="-O3 -fcommon"
+if [ "$OS_TYPE" = "BSD" ]
+then
+	if [ -e /usr/local/lib/libcrypto.so ]
+	then
+		env MAKE=gmake CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./autogen.sh
+	else
+		env MAKE=gmake ./autogen.sh
+	fi
+	gmake -j $NUM_CPU_CORES
+else
+	./autogen.sh
+	make -j $NUM_CPU_CORES
+fi
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd aircrack-ng-1.5.2
+./src/aircrack-ng -p \$NUM_CPU_CORES \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.2.1/install_windows.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+unzip -o aircrack-ng-1.5.2-win.zip
+
+echo "#!/bin/sh
+cd aircrack-ng-1.5.2-win
+./bin/64bit/aircrack-ng.exe \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.2.1/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>#_RESULT_# k/s</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.2.1/test-definition.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Aircrack-ng</Title>
+    <AppVersion>1.5.2</AppVersion>
+    <Description>Aircrack-ng is a tool for assessing WiFi/WLAN network security.</Description>
+    <ResultScale>k/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.2.1</Version>
+    <SupportedPlatforms>Linux, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, openssl-development, libtool</ExternalDependencies>
+    <EnvironmentSize>25</EnvironmentSize>
+    <ProjectURL>http://www.aircrack-ng.org/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-S -Z 30</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/AMG-20200304.tar.xz</URL>
+      <MD5>aa0b1856f5f2d5c7e9250de8a4ef5084</MD5>
+      <SHA256>918964262a30023b65a1e67efc15ec7e434d48c3e886497d1fb2b2fb131043f8</SHA256>
+      <FileName>AMG-20200304.tar.xz</FileName>
+      <FileSize>1311336</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.0/install.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+
+tar -xf AMG-20200304.tar.xz
+rm -rf AMG-bin
+mv AMG AMG-bin
+cd AMG-bin
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+
+cd ~/
+cat>amg<<EOT
+#!/bin/sh
+cd AMG-bin
+mpirun --allow-run-as-root -np \$NUM_CPU_PHYSICAL_CORES ./test/amg > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status
+EOT
+chmod +x amg
+

ob-cache/test-profiles/pts/amg-1.0.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Figure of Merit (FOM_1): #_RESULT_#</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.0/test-definition.xml

@@ -0,0 +1,23 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Algebraic Multi-Grid Benchmark</Title>
+    <Description>AMG is a parallel algebraic multigrid solver for linear systems arising from problems on unstructured grids.  The driver provided with AMG builds linear systems for various 3-dimensional problems.</Description>
+    <ResultScale>Figure Of Merit</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.1</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Scientific</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>openmpi-development, build-utilities</ExternalDependencies>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>https://github.com/LLNL/AMG</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.1/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/AMG-20200304.tar.xz</URL>
+      <MD5>aa0b1856f5f2d5c7e9250de8a4ef5084</MD5>
+      <SHA256>918964262a30023b65a1e67efc15ec7e434d48c3e886497d1fb2b2fb131043f8</SHA256>
+      <FileName>AMG-20200304.tar.xz</FileName>
+      <FileSize>1311336</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.1/install.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+
+tar -xf AMG-20200304.tar.xz
+rm -rf AMG-bin
+mv AMG AMG-bin
+cd AMG-bin
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+
+cd ~/
+cat>amg<<EOT
+#!/bin/sh
+cd AMG-bin
+mpirun --allow-run-as-root -np \$NUM_CPU_PHYSICAL_CORES ./test/amg > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status
+EOT
+chmod +x amg
+

ob-cache/test-profiles/pts/amg-1.0.1/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Figure of Merit (FOM_1): #_RESULT_#</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/amg-1.0.1/test-definition.xml

@@ -0,0 +1,23 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.4.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Algebraic Multi-Grid Benchmark</Title>
+    <Description>AMG is a parallel algebraic multigrid solver for linear systems arising from problems on unstructured grids.  The driver provided with AMG builds linear systems for various 3-dimensional problems.</Description>
+    <ResultScale>Figure Of Merit</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.1</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Scientific</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>openmpi-development, build-utilities</ExternalDependencies>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>https://github.com/LLNL/AMG</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.0.2/downloads.xml

@@ -0,0 +1,28 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-20200110.tar.xz</URL>
+      <MD5>5df82a423e865a3da0ec15f8b95e87cd</MD5>
+      <SHA256>502e432e8d17eb46a1159e21c0d84eb02bea89219594df9ea852fd012b9f1a4b</SHA256>
+      <FileName>aom-20200110.tar.xz</FileName>
+      <FileSize>2364676</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aomenc-windows-20200110.zip</URL>
+      <MD5>5127f6a7d665a247cc0e998f9475bd43</MD5>
+      <SHA256>45ef0f54255d93c6c6ff3943adc985eb400709ce310563ed72d10546e6fde646</SHA256>
+      <FileName>aomenc-windows-20200110.zip</FileName>
+      <FileSize>3518377</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.0.2/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+
+tar -xf aom-20200110.tar.xz
+cd aom/build
+cmake ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+# Current AOMedia Git has MAX_NUM_THREADS value of 64, don't go over 64 threads or error
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aom/build/aomenc --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.0.2/install_windows.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+
+unzip -o aomenc-windows-20200110.zip
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+chmod +x aomenc-20200110.exe
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aomenc-20200110.exe --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.0.2/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame    3/4        768B    2048b/f   61440b/s  165907 us #_RESULT_# fps)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.0.2/test-definition.xml

@@ -0,0 +1,58 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.2.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOM AV1</Title>
+    <AppVersion>2020-01-10</AppVersion>
+    <Description>This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>AV1 Video Encoding</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>2.0.2</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, perl, yasm</ExternalDependencies>
+    <EnvironmentSize>950</EnvironmentSize>
+    <ProjectURL>https://aomedia.googlesource.com/aom/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Option>
+      <DisplayName>Encoder Mode</DisplayName>
+      <Identifier>enc-mode</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Speed 8 Realtime</Name>
+          <Value>--cpu-used=8 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 6 Realtime</Name>
+          <Value>--cpu-used=6 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 4 Realtime</Name>
+          <Value>--cpu-used=4 --rt --limit=60</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 5 Two-Pass</Name>
+          <Value>--cpu-used=5 --limit=40</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 2 Two-Pass</Name>
+          <Value>--cpu-used=2 --limit=20</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 0 Two-Pass</Name>
+          <Value>--cpu-used=0 --limit=10</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.1/downloads.xml

@@ -0,0 +1,28 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-200.tar.xz</URL>
+      <MD5>93c614944b53ed69f36439e433411ddc</MD5>
+      <SHA256>8996bb24c5768a109e3d20cb4b7adb617d7a4095e11801cef1ffccba5e5e6e2c</SHA256>
+      <FileName>aom-200.tar.xz</FileName>
+      <FileSize>2390800</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-200-windows.zip</URL>
+      <MD5>693fe89d844c2735c347b78d3b2df32f</MD5>
+      <SHA256>b27e905f990dffb4071475b0129809ab30f49ed578e69857af61c63400a8572b</SHA256>
+      <FileName>aom-200-windows.zip</FileName>
+      <FileSize>3691863</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.1/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+
+tar -xf aom-200.tar.xz
+cd aom/build
+cmake ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+# Current AOMedia Git has MAX_NUM_THREADS value of 64, don't go over 64 threads or error
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aom/build/aomenc --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.1.1/install_windows.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+
+unzip -o aom-200-windows.zip
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+chmod +x aomenc.exe
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aomenc.exe --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.1.1/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame    3/4        768B    2048b/f   61440b/s  165907 us #_RESULT_# fps)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.1/test-definition.xml

@@ -0,0 +1,56 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.6.0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOM AV1</Title>
+    <AppVersion>2.0</AppVersion>
+    <Description>This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>2.1.1</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, perl, yasm</ExternalDependencies>
+    <EnvironmentSize>950</EnvironmentSize>
+    <ProjectURL>https://aomedia.googlesource.com/aom/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>--tile-columns=2 --tile-rows=1</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Encoder Mode</DisplayName>
+      <Identifier>enc-mode</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Speed 8 Realtime</Name>
+          <Value>--cpu-used=8 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 6 Realtime</Name>
+          <Value>--cpu-used=6 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 6 Two-Pass</Name>
+          <Value>--cpu-used=6 --limit=80</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 4 Two-Pass</Name>
+          <Value>--cpu-used=4 --limit=40</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 0 Two-Pass</Name>
+          <Value>--cpu-used=0 --limit=10</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.2/downloads.xml

@@ -0,0 +1,28 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-200.tar.xz</URL>
+      <MD5>93c614944b53ed69f36439e433411ddc</MD5>
+      <SHA256>8996bb24c5768a109e3d20cb4b7adb617d7a4095e11801cef1ffccba5e5e6e2c</SHA256>
+      <FileName>aom-200.tar.xz</FileName>
+      <FileSize>2390800</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-200-windows.zip</URL>
+      <MD5>693fe89d844c2735c347b78d3b2df32f</MD5>
+      <SHA256>b27e905f990dffb4071475b0129809ab30f49ed578e69857af61c63400a8572b</SHA256>
+      <FileName>aom-200-windows.zip</FileName>
+      <FileSize>3691863</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.2/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+
+tar -xf aom-200.tar.xz
+cd aom/build
+cmake -DENABLE_DOCS=0 -DENABLE_TESTS=0 -DCONFIG_AV1_DECODER=0 ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+# Current AOMedia Git has MAX_NUM_THREADS value of 64, don't go over 64 threads or error
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aom/build/aomenc --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.1.2/install_windows.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+
+unzip -o aom-200-windows.zip
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+chmod +x aomenc.exe
+
+echo "#!/bin/sh
+
+if [ \"\$NUM_CPU_CORES\" -gt 64 ]; then
+	NUM_CPU_CORES=64
+fi
+
+./aomenc.exe --threads=\$NUM_CPU_CORES \$@ -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE
+rm -f test.av1" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-2.1.2/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame    3/4        768B    2048b/f   61440b/s  165907 us #_RESULT_# fps)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-2.1.2/test-definition.xml

@@ -0,0 +1,56 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOM AV1</Title>
+    <AppVersion>2.0</AppVersion>
+    <Description>This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>2.1.2</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, perl, yasm</ExternalDependencies>
+    <EnvironmentSize>950</EnvironmentSize>
+    <ProjectURL>https://aomedia.googlesource.com/aom/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>--tile-columns=2 --tile-rows=1</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Encoder Mode</DisplayName>
+      <Identifier>enc-mode</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Speed 8 Realtime</Name>
+          <Value>--cpu-used=8 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 6 Realtime</Name>
+          <Value>--cpu-used=6 --rt</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 6 Two-Pass</Name>
+          <Value>--cpu-used=6 --limit=80</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 4 Two-Pass</Name>
+          <Value>--cpu-used=4 --limit=40</Value>
+        </Entry>
+        <Entry>
+          <Name>Speed 0 Two-Pass</Name>
+          <Value>--cpu-used=0 --limit=10</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.6.1/downloads.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.0.0 (Plavsk)-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://archive.apache.org/dist/httpd/httpd-2.4.7.tar.bz2, http://ftp.lfs-matrix.net/pub/blfs/conglomeration/httpd/httpd-2.4.7.tar.bz2</URL>
+      <MD5>170d7fb6fe5f28b87d1878020a9ab94e</MD5>
+      <SHA256>64368d8301836815ae237f2b62d909711c896c1bd34573771e0ee5ad808ce71b</SHA256>
+      <FileSize>5004719</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apache-ab-test-files-1.tar.gz, http://www.phoronix.net/downloads/phoronix-test-suite/benchmark-files/apache-ab-test-files-1.tar.gz</URL>
+      <MD5>ca0c5bf0dd482a2847c68a52c0759942</MD5>
+      <FileSize>6008</FileSize>
+    </Package>
+    <Package>
+      <URL>http://apache.cs.utah.edu//apr/apr-1.5.0.tar.bz2, http://archive.apache.org/dist/apr/apr-1.5.0.tar.bz2</URL>
+      <MD5>cc93bd2c12d0d037f68e21cc6385dc31</MD5>
+      <SHA256>17287d36a5917e27281e60d47e7b147bd5ddcd1ca832702c2318f5e0724f1221</SHA256>
+      <FileSize>813976</FileSize>
+    </Package>
+    <Package>
+      <URL>http://archive.apache.org/dist/apr/apr-util-1.5.3.tar.bz2</URL>
+      <MD5>6f3417691c7a27090f36e7cf4d94b36e</MD5>
+      <SHA256>78edb174f13e25ee15ded2b849b741a248b879a93a77a2d31c20f7e225be3968</SHA256>
+      <FileSize>695303</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.6.1/install.sh

@@ -0,0 +1,29 @@
+#!/bin/sh
+
+mkdir $HOME/httpd_
+
+tar -zxvf apache-ab-test-files-1.tar.gz
+tar -jxvf httpd-2.4.7.tar.bz2
+tar -jxvf apr-util-1.5.3.tar.bz2
+tar -jxvf apr-1.5.0.tar.bz2
+mv apr-1.5.0 httpd-2.4.7/srclib/apr
+mv apr-util-1.5.3 httpd-2.4.7/srclib/apr-util
+
+cd httpd-2.4.7/
+./configure --prefix=$HOME/httpd_ --with-included-apr
+make -j $NUM_CPU_JOBS
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf httpd-2.4.7/
+rm -rf httpd_/manual/
+
+patch -p0 < CHANGE-APACHE-PORT.patch
+mv -f test.html httpd_/htdocs/
+mv -f pts.png httpd_/htdocs/
+
+echo "#!/bin/sh
+./httpd_/bin/ab \$@ > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > apache
+
+chmod +x apache

ob-cache/test-profiles/pts/apache-1.6.1/post.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k stop
+rm -f httpd_/logs/*
+sleep 3

ob-cache/test-profiles/pts/apache-1.6.1/pre.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k start -f $HOME/httpd_/conf/httpd.conf
+sleep 5

ob-cache/test-profiles/pts/apache-1.6.1/results-definition.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.0.0 (Plavsk)-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Document Path:          /test.html
+Document Length:        3064 bytes
+
+Concurrency Level:      100
+Time taken for tests:   29.224 seconds
+Complete requests:      500000
+Failed requests:        0
+Write errors:           0
+Total transferred:      1659096222 bytes
+HTML transferred:       1532088856 bytes
+Requests per second:    #_RESULT_# [#/sec] (mean)
+Time per request:       5.845 [ms] (mean)
+Time per request:       0.058 [ms] (mean, across all concurrent requests)
+Transfer rate:          55440.49 [Kbytes/sec] received</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.6.1/test-definition.xml

@@ -0,0 +1,31 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.0.0 (Plavsk)-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Apache Benchmark</Title>
+    <AppVersion>2.4.7</AppVersion>
+    <Description>This is a test of ab, which is the Apache benchmark program. This test profile measures how many requests per second a given system can sustain when carrying out 1,000,000 requests with 100 requests being carried out concurrently.</Description>
+    <ResultScale>Requests Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>Static Web Page Serving</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.6.1</Version>
+    <SupportedPlatforms>Linux, Solaris, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, perl, pcre</ExternalDependencies>
+    <EnvironmentSize>365</EnvironmentSize>
+    <ProjectURL>http://www.apache.org/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-n 1000000 -c 100 http://localhost:8088/test.html</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.1/downloads.xml

@@ -0,0 +1,39 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://archive.apache.org/dist/httpd/httpd-2.4.29.tar.bz2, http://ftp.lfs-matrix.net/pub/blfs/conglomeration/httpd/httpd-2.4.29.tar.bz2</URL>
+      <MD5>0c599404ef6b69eee95bcd9fcd094407</MD5>
+      <SHA256>777753a5a25568a2a27428b2214980564bc1c38c1abf9ccc7630b639991f7f00</SHA256>
+      <FileSize>6567926</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apache-ab-test-files-1.tar.gz, http://www.phoronix.net/downloads/phoronix-test-suite/benchmark-files/apache-ab-test-files-1.tar.gz</URL>
+      <MD5>ca0c5bf0dd482a2847c68a52c0759942</MD5>
+      <FileSize>6008</FileSize>
+    </Package>
+    <Package>
+      <URL>http://apache.cs.utah.edu//apr/apr-1.6.3.tar.bz2, http://archive.apache.org/dist/apr/apr-1.6.3.tar.bz2</URL>
+      <MD5>12f2a349483ad6f12db49ba01fbfdbfa</MD5>
+      <SHA256>131f06d16d7aabd097fa992a33eec2b6af3962f93e6d570a9bd4d85e95993172</SHA256>
+      <FileSize>854100</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.apache.org/dist/apr/apr-util-1.6.1.tar.bz2, http://apache.cs.utah.edu//apr/apr-util-1.6.1.tar.bz2</URL>
+      <MD5>8ff5dc36fa39a2a3db1df196d3ed6086</MD5>
+      <SHA256>d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b</SHA256>
+      <FileSize>428595</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/Apache24-2.4.29-x64-vc14-r2-ah.zip</URL>
+      <MD5>8f7e773c8db85d55e4cc81ef153fef61</MD5>
+      <SHA256>1e6f26e8fbe3decd4c31cff3347a074c7db1f8af5ac71bf4386a1e80304e47ef</SHA256>
+      <FileSize>10444082</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.1/install.sh

@@ -0,0 +1,29 @@
+#!/bin/sh
+
+mkdir $HOME/httpd_
+
+tar -zxvf apache-ab-test-files-1.tar.gz
+tar -jxvf httpd-2.4.29.tar.bz2
+tar -jxvf apr-util-1.6.1.tar.bz2
+tar -jxvf apr-1.6.3.tar.bz2
+mv apr-1.6.3 httpd-2.4.29/srclib/apr
+mv apr-util-1.6.1 httpd-2.4.29/srclib/apr-util
+
+cd httpd-2.4.29/
+./configure --prefix=$HOME/httpd_ --with-included-apr
+make -j $NUM_CPU_JOBS
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf httpd-2.4.29/
+rm -rf httpd_/manual/
+
+patch -p0 < CHANGE-APACHE-PORT.patch
+mv -f test.html httpd_/htdocs/
+mv -f pts.png httpd_/htdocs/
+
+echo "#!/bin/sh
+./httpd_/bin/ab \$@ > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > apache
+
+chmod +x apache

ob-cache/test-profiles/pts/apache-1.7.1/install_windows.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+unzip -o Apache24-2.4.29-x64-vc14-r2-ah.zip
+
+tar -xf apache-ab-test-files-1.tar.gz
+mv -f test.html Apache24/htdocs/
+mv -f pts.png Apache24/htdocs/
+
+cd Apache24/conf
+tail -n +2 httpd.conf > httpd.conf.2
+echo "Define SRVROOT \"$DEBUG_HOME\Apache24\"
+" > httpd.conf
+cat httpd.conf.2 >> httpd.conf
+cd ~
+
+echo "#!/bin/sh
+cd Apache24/bin
+./ab.exe \$@ > \$LOG_FILE" > apache
+chmod +x apache

ob-cache/test-profiles/pts/apache-1.7.1/post.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k stop
+rm -f httpd_/logs/*
+sleep 3

ob-cache/test-profiles/pts/apache-1.7.1/post_windows.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+kill -TERM `cat $HOME/Apache24/logs/httpd.pid`
+sleep 10
+rm -f $HOME/Apache24/logs/*

ob-cache/test-profiles/pts/apache-1.7.1/pre.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k start -f $HOME/httpd_/conf/httpd.conf
+sleep 5

ob-cache/test-profiles/pts/apache-1.7.1/pre_windows.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+cd Apache24/bin
+./httpd.exe &
+sleep 10

ob-cache/test-profiles/pts/apache-1.7.1/results-definition.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Document Path:          /test.html
+Document Length:        3064 bytes
+
+Concurrency Level:      100
+Time taken for tests:   29.224 seconds
+Complete requests:      500000
+Failed requests:        0
+Write errors:           0
+Total transferred:      1659096222 bytes
+HTML transferred:       1532088856 bytes
+Requests per second:    #_RESULT_# [#/sec] (mean)
+Time per request:       5.845 [ms] (mean)
+Time per request:       0.058 [ms] (mean, across all concurrent requests)
+Transfer rate:          55440.49 [Kbytes/sec] received</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.1/test-definition.xml

@@ -0,0 +1,31 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Apache Benchmark</Title>
+    <AppVersion>2.4.29</AppVersion>
+    <Description>This is a test of ab, which is the Apache benchmark program. This test profile measures how many requests per second a given system can sustain when carrying out 1,000,000 requests with 100 requests being carried out concurrently.</Description>
+    <ResultScale>Requests Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>Static Web Page Serving</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.7.1</Version>
+    <SupportedPlatforms>Linux, Solaris, BSD, MacOSX, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, perl, pcre</ExternalDependencies>
+    <EnvironmentSize>208</EnvironmentSize>
+    <ProjectURL>http://www.apache.org/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-n 1000000 -c 100 http://localhost:8088/test.html</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.2/downloads.xml

@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://archive.apache.org/dist/httpd/httpd-2.4.29.tar.bz2, http://ftp.lfs-matrix.net/pub/blfs/conglomeration/httpd/httpd-2.4.29.tar.bz2</URL>
+      <MD5>0c599404ef6b69eee95bcd9fcd094407</MD5>
+      <SHA256>777753a5a25568a2a27428b2214980564bc1c38c1abf9ccc7630b639991f7f00</SHA256>
+      <FileName>httpd-2.4.29.tar.bz2</FileName>
+      <FileSize>6567926</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apache-ab-test-files-1.tar.gz, http://www.phoronix.net/downloads/phoronix-test-suite/benchmark-files/apache-ab-test-files-1.tar.gz</URL>
+      <MD5>ca0c5bf0dd482a2847c68a52c0759942</MD5>
+      <FileName>apache-ab-test-files-1.tar.gz</FileName>
+      <FileSize>6008</FileSize>
+    </Package>
+    <Package>
+      <URL>http://apache.cs.utah.edu//apr/apr-1.6.3.tar.bz2, http://archive.apache.org/dist/apr/apr-1.6.3.tar.bz2</URL>
+      <MD5>12f2a349483ad6f12db49ba01fbfdbfa</MD5>
+      <SHA256>131f06d16d7aabd097fa992a33eec2b6af3962f93e6d570a9bd4d85e95993172</SHA256>
+      <FileName>apr-1.6.3.tar.bz2</FileName>
+      <FileSize>854100</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.apache.org/dist/apr/apr-util-1.6.1.tar.bz2, http://apache.cs.utah.edu//apr/apr-util-1.6.1.tar.bz2</URL>
+      <MD5>8ff5dc36fa39a2a3db1df196d3ed6086</MD5>
+      <SHA256>d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b</SHA256>
+      <FileName>apr-util-1.6.1.tar.bz2</FileName>
+      <FileSize>428595</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://download.joedog.org/siege/siege-3.1.4.tar.gz</URL>
+      <MD5>a893b92a453e1c1d596314aa951981f4</MD5>
+      <SHA256>0436970ca81ade76eb419bc42d3f8460b475079eb5d8f199d2fa83328893099d</SHA256>
+      <FileName>siege-3.1.4.tar.gz</FileName>
+      <FileSize>530149</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.2/install.sh

@@ -0,0 +1,40 @@
+#!/bin/sh
+
+mkdir $HOME/httpd_
+
+tar -zxvf apache-ab-test-files-1.tar.gz
+tar -jxvf httpd-2.4.29.tar.bz2
+tar -jxvf apr-util-1.6.1.tar.bz2
+tar -jxvf apr-1.6.3.tar.bz2
+mv apr-1.6.3 httpd-2.4.29/srclib/apr
+mv apr-util-1.6.1 httpd-2.4.29/srclib/apr-util
+
+cd httpd-2.4.29/
+./configure --prefix=$HOME/httpd_ --with-included-apr
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf httpd-2.4.29/
+rm -rf httpd_/manual/
+
+patch -p0 < CHANGE-APACHE-PORT.patch
+mv -f test.html httpd_/htdocs/
+mv -f pts.png httpd_/htdocs/
+
+
+cd ~
+tar -xf siege-3.1.4.tar.gz
+cd siege-3.1.4
+./configure
+make -j $NUM_CPU_CORES
+cd utils
+bash siege.config
+cd ~
+
+echo "#!/bin/sh
+cd siege-3.1.4/src
+./siege \$@ 2>&1 | grep -v HTTP > \$LOG_FILE
+echo \$? > ~/test-exit-status" > apache-siege
+
+chmod +x apache-siege

ob-cache/test-profiles/pts/apache-siege-1.0.2/post.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k stop
+rm -f httpd_/logs/*
+sleep 3

ob-cache/test-profiles/pts/apache-siege-1.0.2/pre.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k start -f $HOME/httpd_/conf/httpd.conf
+sleep 5
+
+cd siege-3.1.4/utils
+bash siege.config
+cd ~

ob-cache/test-profiles/pts/apache-siege-1.0.2/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Transaction rate:	    #_RESULT_# trans/sec</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.2/test-definition.xml

@@ -0,0 +1,62 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.2.0m1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Apache Siege</Title>
+    <AppVersion>2.4.29</AppVersion>
+    <Description>This is a test of the Apache web server performance being facilitated by the Siege web serverb enchmark program.</Description>
+    <ResultScale>Transactions Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.2</Version>
+    <SupportedPlatforms>Linux, Solaris, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, perl, pcre, openssl-development</ExternalDependencies>
+    <EnvironmentSize>208</EnvironmentSize>
+    <ProjectURL>https://www.joedog.org/siege-home/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-r10000 -d1 -b </Arguments>
+      <PostArguments> http://localhost:8088/test.html</PostArguments>
+    </Default>
+    <Option>
+      <DisplayName>Concurrent Users</DisplayName>
+      <Identifier>concurrent</Identifier>
+      <ArgumentPrefix>-c</ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>1</Name>
+          <Value>1</Value>
+        </Entry>
+        <Entry>
+          <Name>10</Name>
+          <Value>10</Value>
+        </Entry>
+        <Entry>
+          <Name>50</Name>
+          <Value>50</Value>
+        </Entry>
+        <Entry>
+          <Name>100</Name>
+          <Value>100</Value>
+        </Entry>
+        <Entry>
+          <Name>200</Name>
+          <Value>200</Value>
+        </Entry>
+        <Entry>
+          <Name>250</Name>
+          <Value>250</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.1/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/apitest-20140726.tar.bz2</URL>
+      <MD5>28ac817817d35ab335ad40dcec682b6d</MD5>
+      <SHA256>67a48803c10c147f99632e1aa54cf1afce1b785d2ecf850b5f960e72d2dd5258</SHA256>
+      <FileName>apitest-20140726.tar.bz2</FileName>
+      <FileSize>23855194</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.1/install.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+tar -xjf apitest-20140726.tar.bz2
+cd apitest-master
+mkdir out
+cd out
+cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DHAVE_LIBUDEV_H=0  ..
+make -j $NUM_CPU_JOBS
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd apitest-master/bin/
+./apitest \$@ > \$LOG_FILE 2>&1
+echo $? > ~/test-exit-status" > apitest
+chmod +x apitest

ob-cache/test-profiles/pts/apitest-1.1.1/results-definition.xml

@@ -0,0 +1,8 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate> NullProblem             oglcore    NullSolution                     30316       15.000     #_RESULT_#        0.495</OutputTemplate>
+    <LineBeforeHint>fps</LineBeforeHint>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.1/test-definition.xml

@@ -0,0 +1,136 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>APITest</Title>
+    <AppVersion>2014-07-26</AppVersion>
+    <Description>APITest is a micro-benchmark developed by John McDonald of OpenGL 4 functionality.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.1</Version>
+    <SupportedPlatforms>Linux, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Graphics</TestType>
+    <License>Free</License>
+    <Status>Deprecated</Status>
+    <ExternalDependencies>build-utilities, sdl2-development, cmake</ExternalDependencies>
+    <RequiresDisplay>TRUE</RequiresDisplay>
+    <EnvironmentSize>225</EnvironmentSize>
+    <ProjectURL>http://github.com/nvMcJohn/apitest</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-b -t 10 -a oglcore</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Resolution</DisplayName>
+      <Identifier>auto-resolution</Identifier>
+      <ArgumentPrefix>-r </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>$VIDEO_WIDTH x $VIDEO_HEIGHT</Name>
+          <Value>$VIDEO_WIDTH $VIDEO_HEIGHT</Value>
+        </Entry>
+      </Menu>
+    </Option>
+    <Option>
+      <DisplayName>Test</DisplayName>
+      <Identifier>test</Identifier>
+      <ArgumentPrefix>-s </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>DynamicStreaming GLBufferSubData</Name>
+          <Value>GLBufferSubData</Value>
+        </Entry>
+        <Entry>
+          <Name>DynamicStreaming GLMapUnsynchronized</Name>
+          <Value>GLMapUnsynchronized</Value>
+        </Entry>
+        <Entry>
+          <Name>DynamicStreaming GLMapPersistent</Name>
+          <Value>GLMapPersistent</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLUniform</Name>
+          <Value>GLUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLDrawLoop</Name>
+          <Value>GLDrawLoop</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMultiDrawBuffer-SDP</Name>
+          <Value>GLMultiDrawBuffer-SDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMultiDrawBuffer-NoSDP</Name>
+          <Value>GLMultiDrawBuffer-NoSDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferRange</Name>
+          <Value>GLBufferRange</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferSubData</Name>
+          <Value>GLBufferSubData</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferStorage-SDP</Name>
+          <Value>GLBufferStorage-SDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferStorage-NoSDP</Name>
+          <Value>GLBufferStorage-NoSDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLDynamicBuffer</Name>
+          <Value>GLDynamicBuffer</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMapUnsynchronized</Name>
+          <Value>GLMapUnsynchronized</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMapPersistent</Name>
+          <Value>GLMapPersistent</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLTexCoord</Name>
+          <Value>GLTexCoord</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLBindless</Name>
+          <Value>GLBindless</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNaive</Name>
+          <Value>GLNaive</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNaiveUniform</Name>
+          <Value>GLNaiveUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNoTex</Name>
+          <Value>GLNoTex</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNoTexUniform</Name>
+          <Value>GLNoTexUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLSBTA</Name>
+          <Value>GLSBTA</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLTextureArray</Name>
+          <Value>GLTextureArray</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitrace-1.0.2/downloads.xml

@@ -0,0 +1,17 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apitrace-apitrace-3.0-0-gde9f3e5.zip</URL>
+      <MD5>9ef70fa990f165e30a4408762fdb4a76</MD5>
+      <FileSize>1279634</FileSize>
+    </Package>
+    <Package>
+      <URL></URL>
+      <MD5>21708bcee9c1e539b051779a76cc8880</MD5>
+      <FileName>redeclipse-1-trace.7z</FileName>
+      <FileSize>156191890</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>