pts-core: PHP 8.5 warning fixes

Some benchmark output reports are in the format:
xyz score:XYZ

Here there is no space between ':' and actual result.

To parse such format create a template something like this:
<OutputTemplate>xxx:#_RESULT_#</OutputTemplate>

This type of the format can't be parsed correctly.

The current implementation expects a "space" after column.
For example:
In test-profiles/system/selenium-1.0.18/results-definition.xml
<ResultsParser>
  <OutputTemplate>WEBXPRT SCORE RESULT: #_RESULT_#</OutputTemplate>

To be able to parse when there is no space between ":" and "#" add a
a space character in the template line.

Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>

.mailmap

@@ -0,0 +1,6 @@
+Michael Larabel <Michael@phoronix.com> <michael@phx-laptop.(none)>
+Michael Larabel <Michael@phoronix.com> <michael@phoronix.com>
+Michael Larabel <Michael@phoronix.com> <michael@phxmichael.hsd1.il.comcast.net>
+Michael Larabel <Michael@phoronix.com> <michael@mubuntu.(none)>
+Michael Larabel <Michael@phoronix.com> <michaellarabel@users.noreply.github.com>
+Michael Larabel <Michael@phoronix.com> <Michael AT phoronix.com>

AUTHORS

@@ -1,4 +1,4 @@
-Copyright 2008 - 2020 by Phoronix Media.
+Copyright 2008 - 2025 by Phoronix Media.
 
 Lead Architects:
 

ChangeLog

@@ -1,5 +1,299 @@
 PHORONIX TEST SUITE CHANGE-LOG
 
+Phoronix Test Suite 10.8.5
+[GIT]
+
+pts-core: PHP 8.2/8.3 warning fixes
+pts-core: Various bug fixes
+phodevi: Hardware/software detection updates
+
+Phoronix Test Suite 10.8.4
+3 July 2022
+
+pts-core: Allow sub-folders within PTS_EXTRA_SYSTEM_LOGS_DIR when uploading to Phoromatic server
+pts-core: Fix TOTAL_LOOP_TIME handling
+pts-core: openSUSE external dependency handling fixes
+phodevi: Support for reporting video BIOS version
+phodevi: Budgie desktop version detection
+phodevi: Avoid deprecated wmic usage on Windows
+phoromatic: Input sanitization updates in the result viewer
+
+Phoronix Test Suite 10.8.3
+23 April 2022
+
+pts-core: Workaround Ubuntu 22.04 apt-get crashing the entire process on dependency installation
+pts-core: Improved TTF font file detection
+pts-core: Minor random fixes
+phodevi: Apple M1 detection on Linux
+
+Phoronix Test Suite 10.8.2
+13 February 2022
+
+pts-core: Arch Linux external dependency updates
+pts-core: Workaround/fix PHP 5.4 usage with RHEL7-based distributions
+pts-core: Convert compiler / multicore / single-threaded test suites into virtual test suites since their properties can be auto-determined based on OB metadata to avoid manually maintaining them
+pts-core: Add virtual test suites for "riscv" and "aarch64" to make it easy to show test profiles that can run successfully on RISC-V and 64-bit Arm hardware
+phodevi: Improved detection for components found on the Steam Deck and other hardware
+phodevi: Add Arm Cortex-X1C detection
+phodevi: CPU temperature reporting for additional SoCs such as the Raspberry Pi 400
+phoromatic: Additional input sanitization and validation improvements
+
+Phoronix Test Suite 10.8.1
+23 January 2022
+
+pts-core: Log external dependencies attempted to be installed to phoronix-test-suite-dependencies.log
+pts-core: Allow PTS_DISPLAY_MODE to override other display mode preferences (e.g. short mode requested inline)
+pts-core: Optional Download Cache Server interface support for HTTPS
+pts-core: Support for archiving test profile change-logs locally rather than having to always query OpenBenchmarking.org server
+pts-core: Allow caching of some OpenBenchmarking.org overview metrics locally for reducing repetitive OpenBenchmarking.org load/queries
+pts-core: Allow displaying more overview metrics from OpenBenchmarking.org in the web view and info sub-command now that it's cached
+pts-core: Make rendering of Perf Per Core/Thread/Clock/RAM opt-in from the result viewer
+phoromatic: Allow binary log file uploads by default when Phoromatic Server is 10.8+
+phoromatic: Input sanitization updates for Phoromatic Server and other security improvements
+phoromatic: Move "Create A Test Profile" page behind an opt-in rootadmin Phoromatic Server option
+phoromatic: Log update script output
+phodevi: GPU vRAM usage reporting for modern AMDGPU DRM versions
+phodevi: Use "-NoProfile" with powershell for possible performance benefits
+phodevi: Avoid including kernel version in Mesa/OpenGL string
+phodevi: Arm Neoverse-V1 / Cortex-X2 identification
+
+Phoronix Test Suite 10.8.0
+25 December 2021
+
+pts-core: Remaining PHP 8.1 warnings/fixes addressed
+pts-core: Various macOS 12 warning fixes
+pts-core: Allow using OUTPUT_DIR / OUTPUT_FILE environment variables to control result-file-to* sub-commands
+pts-core: Improved JSON output generation, add option to the result viewer
+pts-core: Add result-file-to-html sub-command and simple inline HTML result output option from result viewer
+pts-core: Allow sorting results by test date/time
+pts-core: Add TEST_EXECUTION_SORT environment variable to allow sorting test run order in more ways
+pts-core: Consolidate/centralize handling of environment variable options to pts_env
+pts-core: Add remove-incomplete-results-from-result-file helper command to remove results with incomplete or missing data
+pts-core: Add REMOVE_TESTS_ON_COMPLETION environment variable as another way to have tests auto-uninstall/remove after running
+pts-core: Improved test installation / test run-time error reporting and detection
+pts-core: Allow recording/archiving install and runtime errors within installation metadata
+pts-core: Add list-failed-installs sub-command for showing tests that failed to install
+pts-core: Add list-test-errors sub-command for showing test run-time errors
+pts-core: Add variables sub-command for showing environment variable descriptions and what is set
+pts-core: Dynamic detection of Windows download folder location
+pts-core: Improved Zip handling on Windows
+phodevi: Add detection for Arm Cortex-A710 and Neoverse-N2 cores
+phodevi: Add detection for AMD Zen 4 cores
+phodevi: Add detection for Intel Raptor Lake
+phodevi: Allow better detecting model number on Windows/Linux for Dell and Apple laptops
+phodevi: CPU temperature monitoring on Windows
+phodevi: Show "energy_performance_preference" CPU frequency scaling value where relevant
+phodevi: Linux CPUFreq EPP reporting support
+phodevi: Report Windows 10+ security features
+phodevi: Add PHODEVI_SANITIZE environment variable for those wanting to strip out certain hardware/software strings from system information
+cleanup: Add module that can auto uninstall tests not run in a given amount of time
+phoromatic: Allow relevant PTS environment variables to be exposed for setting by Phoromatic Server for benchmarks/schedules as "advanced options" area
+phoromatic: Use the unified result viewer code for presenting result graphs and other result page data
+phoromatic: Make use of shared/common result viewer code to allow adding/editing annotations, editing result file title/description, and deleting individual results from within the result viewer
+phoromatic: Efficiency improvement for when viewing result with just one result file
+phoromatic: Allow Phoromatic client to connect to Phoromatic HTTPS servers
+phoromatic: Allow optionally uploading test install/run logs to the Phoromatic Server
+phoromatic: Alow "Run A Benchmark" area to execute conventional PTS/system test suites
+phoromatic: Allow test schedules page to add suites to a schedule rather than only adding just test profiles
+phoromatic: Add option to settings page to always un-install/remove tests after execution for all Phoromatic testing
+phoromatic: Allow environment variables to also be set on a global basis from the Phoromatic Server Settings page
+phoromatic: Allow test installation metadata to be submitted to Phoromatic Server so from systems area can show currently installed tests and any errors
+phoromatic: Opt-in (settings page) support for streaming in-progress benchmark results to the Phoromatic Server between test runs so results can be viewed from the server as they are completed
+phoromatic: Unifying more result viewer and Phoromatic common display code
+
+Phoronix Test Suite 10.6.1
+10 October 2021
+
+pts-core: Allow reporting test run errors inline / as part of the result file XML with null/empty results
+pts-core: Remove the old, long deprecated static/JavaScript PTS result viewer that hasn't been used except for rare fallbacks since PTS pre-9.0
+pts-core: PHP 8.1 deprecation warning/error fixes
+pts-core: Support launching PHP web server with PHP_CLI_SERVER_WORKERS to allow multiple workers/threads
+pts-core: Windows batch file handling test execution fix
+pts-core: Allow running merge-results when pointing to specific XML files outside of standard paths
+pts-core: Various minor fixes
+phoromatic: Fix possible case of local test profile download issues from Phoromatic Server
+phoromatic: Warn when lacking database write access for Phoromatic Server
+phoromatic: Support (opt-in) handling of non-text file log files
+phoromatic: Allow uploading of system logs after initial upload of result XML to deal with HTTP size limitations and other factors with large logs
+phoromatic: Use the unified/common log viewer for viewing logs from the Phoromatic web interface
+phoromatic: Tweaking of the Phoromatic CSS style to bring it more uniform with OpenBenchmarking.org / PTS Local Result Viewer / first update in ~6 years...
+phodevi: Improve detection of new Intel oneAPI DPC++/C++ Compiler
+phodevi: ACPI Platform Profile reporting support on Linux
+phodevi: Support reporting NVIDIA BAR1 total memory / Radeon vRAM visible memory (Resizable BAR / ReBAR status)
+
+Phoronix Test Suite 10.4.0
+16 May 2021
+
+pts-core: Ubuntu 21.04 dependency updates for changed/removed package names
+pts-core: Add "auto-resolution-wide" option
+pts-core: Add $CPU_THREADS_PER_CORE helper
+pts-core: Allow perf-per-dollar-per-hour in the result viewer, extending perf-per-dollar HTML helper
+pts-core: Add PTS_EXTRA_SYSTEM_LOGS_DIR environment variable to allow pointing to directory of extra system log files
+pts-core: Add RepositoryURL option for test profiles
+phodevi: Intel Linux graphics string format improvement
+phoromatic: Treat set context scripts as a file rather than executing it as a buffer
+phoromatic: Allow passing Phodevi system properties to Phoromatic Server
+phodevi: Add some new properties, port some pts_network info to phodevi interfaces
+system_monitor: Adjust perf-per precision
+
+Phoronix Test Suite 10.4.0 Milestone 1
+17 March 2021
+
+pts-core: Optimize out sorting percentile values array excess times
+pts-core: Optimization to avoid re-parsing test profile definition multiple times
+pts-core: Optimization to avoid re-parsing installed test metadata multiple times
+pts-core: Allow longer HTTP timeouts on OpenBenchmarking.org result uploads if needed
+pts-core: Allow sorting runs within a result file from within the result viewer web interface
+
+Phoronix Test Suite 10.2.2
+17 February 2021
+
+pts-core: Add helper text when hitting a download with no working mirrors
+pts-core: Various external dependency updates
+pts-core: Add enable-repo sub-command for more easily enabling new OpenBenchmarking.org repos
+pts-core: Result viewer fix for the limit drop down menu potentially not being activated in some cases
+pts-core: Allow renaming runs within the result viewer
+phoromatic: Ensure pcntl_signal is present before starting Phoromatic Server
+phoromatic: PHP 8 compatibility fixes
+phodevi: Fix display driver name potentially being reported inaccurate for some obscure graphics cards / driver configurations
+phodevi: Export current GPU graphics card PCI device ID (GPU_DEVICE_ID) for some games/tests relying on it for configuration
+
+Phoronix Test Suite 10.2.1
+1 February 2021
+
+pts-core: Add "InstallRequiresInternet" option for test profiles that is more narrow (install time only) than the existing "RequiresInternet"
+pts-core: Add check-tests sub-command
+pts-core: Fix possible crash on PDF output
+pts-core: Various small fixes and tweaks
+pts-core: Fix PHP8 + cURL download error
+pts_Graph: A number of minor SVG standards compliance updates
+phodevi: Log Transparent Huge Pages setting as part of system details
+phodevi: Don't bother parsing the X.org log if it grows too big
+
+Phoronix Test Suite 10.2.0
+5 January 2021
+
+pts-core: PHP8 detection improvements
+pts-core: For estimated run time, avoid over-calculating time if test has no options but multiple outputs
+pts-core: Updated (Open)Mandriva external dependencies
+pts_Graph: Minor enhancements
+phodevi: Add support for reporting ACPI Platform Profile and INT340x Workload Hint
+
+Phoronix Test Suite 10.2.0 Milestone 3
+21 December 2020
+
+pts-core: Add debug-unhandled-dependencies option
+pts-core: Default to 2.5% std dev threshold default on new installs
+pts-core: BSD external dependency updates
+pts-core: Various refinements to the text user interface
+pts-core: When a test profile hits a failed download / broken mirror, report that URL to OpenBenchmarking.org
+phodevi: Don't restore the Phodevi cache if the /etc/machine-id changed (or /etc/hostname as fallback)
+
+Phoronix Test Suite 10.2.0 Milestone 2
+6 December 2020
+
+pts-core: Transition to new JSON-based data format for managing test install metadata while maintaining backwards compatibility with prior XML format
+pts-core: Begin tracking test run-times per-hash (all configured test options) for more detailed time tracking and in turn time estimates on subsequent runs
+pts-core: Increase compression factor when uploading benchmark results
+pts-core: Last minute fixes for PHP 8.0 compatibility
+phodevi: Report if CPUFreq Boost enabled/disabled
+turbostat: Add TurboStat module to easily generate turbostat dumps on a per-test basis to log files
+phoromatic: Much faster web viewer performance especially with large databases
+
+Phoronix Test Suite 10.2.0 Milestone 1
+21 November 2020
+
+pts-core: Log lock_stat, config.gz, dmidecode to system logs
+pts-core: Only show Apple options when running on macOS
+pts-core: Add extra safeguard around FORCE_TIMES_TO_RUN for disabling dynamic run count
+ob_auto_compare: Intelligently choose more relevant reference results
+phodevi: macOS CPU temperature monitoring support
+phodevi: Support for reading WattsUp Pro power meter data directly without utility using WATTS_UP_METER=<USB TTY>
+phodevi: Report Google Compute Engine machine type as part of motherboard string
+phodevi: Add dumpe2fs to system logs when running on EXT4 and root
+phodevi: Add block size reporting
+phodevi: CPU power reporting for Ampere Altra
+phodevi: Various Ampere Altra hardware detection/reporting improvements
+phodevi: Support reading CPU package power consumption on macOS using powermetrics
+
+Phoronix Test Suite 10.0.1
+21 October 2020
+
+pts-core: Fix for older versions of PHP such as in RHEL7
+pts-core: Add Thermald version reporting to system table when active
+phodevi: Detection improvements for new CPUs
+phodevi: Avoid "11th Gen Intel" CPU string
+phodevi: Update for NVMe drive reporting with KIOXIA, Toshiba sometimes not reporting their brand first
+
+Phoronix Test Suite 10.0.0
+13 October 2020
+
+pts-core: Ensure any post-run script is triggered still under halt-testing/skip-test conditions
+pts-core: Allow batch processing of performance per dollar metrics
+phodevi: Add Intel AMX presence detection
+phodevi: Add detection for Intel Alder Lake, Jasper Lake, Sapphire Rapids, and others
+phoromatic: Ensure Windows tests get displayed even if running the Phoromatic Server on Linux
+flush_caches: Add module to flush system caches between runs
+test_timeout: Add module to quit a test run if it exceeds a predefined time limit or 'auto' determination
+
+Phoronix Test Suite 10.0.0 Milestone 3
+1 October 2020
+
+pts-core: Don't show Vulkan options if Vulkan support is known to be unavailable/broken
+pts-core: Add OptionSelect tag support within test suite definitions
+pts-core: Allow * wildcard match in FileCheck handling within external dependency XML
+pts-core: Add "OS" TestType
+pts-core: Various external dependency updates
+pts-core: Continued tweaking to the analytics engine
+phodevi: Add Neoverse N2, Cortex-X1 detection and other Arm CPU updates
+pts_Graph: Various improvements
+
+Phoronix Test Suite 10.0.0 Milestone 2
+12 September 2020
+
+pts-core: Honor NO_COLOR environment variable for disabling terminal coloring
+pts-core: Allow launching just the result viewer from systemd (phoronix-result-server.service)
+pts-core: Result viewer UI tweaks
+pts-core: Minor fixes
+pts-core: Add RequiresRootInstall test profile option
+pts-core: Result file text output tweaks
+pts-core: More robust test result parsing in certain time formats
+
+Phoronix Test Suite 10.0.0 Milestone 1
+19 August 2020
+
+pts-core: Add keep-results-in-result-file sub-command
+pts-core: Split the dynamic result viewer into its own embed class for result file handling to allow for easy re-use by OpenBenchmarking, Phoromatic, etc
+pts-core: Various changes for supporting the new OpenBenchmarking.org infrastructure
+pts_Graph: Add histogram type
+phodevi: CPU core name / family detection to friendly strings (OpenBenchmarking.org driven)
+phodevi: Report core name via system-info command
+phodevi: Various storage HDD/SSD detection reporting improvements
+phodevi: Ampere Altra detection
+phodevi: macOS/Apple hardware detection improvements
+
+Phoronix Test Suite 9.8.0
+9 July 2020
+
+pts-core: Handling of cases where all options are unsupported (such as CUDA and OpenCL tests with no valid backends found) to avoid prompts
+pts-core: Be more liberal in allowing CUDA/OpenCL GPU compute tests to run fine without display server active
+pts-core: Fix for not filling test suites of contained tests in some select cases (sub-suites listing in results viewer)
+pts-core: Allow creating new virtual test suites that are not prefixed by a OB repository
+phodevi: NVIDIA GPU detection within WSL2
+docker: Rewritten phoronix/pts Docker implementation based on Ubuntu 20.04 LTS
+
+Phoronix Test Suite 9.8.0 Milestone 2
+24 June 2020
+
+pts-core: Add client side plumbing for estimated test install time from OpenBenchmarking.org
+pts-core: Add estimate-install-time sub-command, show estimated install time when installing a test
+pts-core: Improve web browser selection logic for result viewer
+pts-core: Add search field to the result viewer areas for limiting based on title/argument matching
+pts-core: Various minor fixes
+pts-core: Don't show OpenCL test options if no OpenCL drivers/adapters appear present
+phodevi: Fix for incorrect monitor count when parsing X.Org log
+phodevi: Warning fixes for latest WSL
+
 Phoronix Test Suite 9.8.0 Milestone 1
 3 June 2020
 
@@ -8,7 +302,7 @@ pts-core: When a newer minor version of a test profile is installed and a reques
 pts-core: Rewrite virtual test suite implementation
 pts-core: Add handler for ensuring reported system time is accurate, otherwise calculate current date/time using offsets
 pts-core: When scanning test suites on disk, ensure the latest test suite version is used
-pts-core: Add test-to-suite map sub-command
+pts-core: Add test-to-suite-map sub-command
 pts-core: Fix for loading the result viewer JavaScript / CSS when running on a Windows host
 pts-core: For tests requiring root but not run initially as root and using monitoring data, drop the first run to ensure sudo promot doesn't skew time
 pts-core: Fix PHP8 error in the 'info' sub-command

README.md

@@ -1,5 +1,5 @@
-# Phoronix Test Suite 9.8.0m1
-https://www.phoronix-test-suite.com/
+# Phoronix Test Suite 10.8.5
+![Phoronix Test Suite](pts-core/static/images/pts-308x160.png)
 
 The **Phoronix Test Suite** is the most comprehensive testing and benchmarking
 platform available for Linux, Solaris, macOS, Windows, and BSD operating
@@ -9,26 +9,28 @@ are meant to be easily reproducible, easy-to-use, and support fully automated
 execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license
 and is developed by Phoronix Media in cooperation with partners.
 
+[Phoronix Test Suite
+Documentation](https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite.md)
+
 The Phoronix Test Suite itself is an open-source framework for conducting
 automated tests along with reporting of test results, detection of installed
-system software/hardware, and other features. Modules for the Phoronix Test
-Suite also allow for integration with git-bisect and other revision control
-systems for per-commit regression testing, system sensor monitoring, and other
-extras.
+system software/hardware, and other features. This framework is designed to be
+an extensible architecture so that new test profiles and suites can be easily
+added to represent performance benchmarks, unit tests, and other quantitative
+and qualitative (e.g. image quality comparison and pass/fail) measurements.
+Available through OpenBenchmarking.org, a collaborative storage platform
+developed in conjunction with the Phoronix Test Suite, are more than 600
+individual test profiles and more than 200 test suites available by default from
+the Phoronix Test Suite. Independent users are also able to upload their test
+results, profiles, and suites to OpenBenchmarking.org.
 
-This framework is designed to be an extensible architecture so that new test
-profiles and suites can be easily added to represent performance benchmarks,
-unit tests, and other quantitative and qualitative (e.g. image quality
-comparison) measurements. Available through OpenBenchmarking.org, a
-collaborative storage platform developed in conjunction with the Phoronix Test
-Suite, are more than 200 individual test profiles and more than 60 test suites
-available by default from the Phoronix Test Suite. Independent users are also
-able to upload their test results, profiles, and suites to OpenBenchmarking.org.
 A test profile is a single test that can be executed by the Phoronix Test Suite
 -- with a series of options possible within every test -- and a test suite is a
 seamless collection of test profiles and/or additional test suites. A test
 profile consists of a set of Bash/shell scripts and XML files while a test suite
-is a single XML file.
+is a single XML file. Modules for the Phoronix Test Suite also allow for
+integration with git-bisect and other revision control systems for per-commit
+regression testing, system sensor monitoring, and other extras.
 
 [OpenBenchmarking.org](https://www.openbenchmarking.org/) also allows for
 conducting side-by-side result comparisons, a central location for storing and
@@ -46,7 +48,7 @@ Phoromatic, and OpenBenchmarking.org is available by contacting
 
 Full details on the Phoronix Test Suite setup and usage is available from the
 included HTML/PDF documentation within the phoronix-test-suite package and from
-the Phoronix Test Suite website.
+the [Phoronix Test Suite documentation directory](documentation/).
 
 ## Installation & Setup
 
@@ -54,9 +56,9 @@ The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows
 systems. However, the most full-featured and well supported operating system for
 conducting the tests is Linux with some non-basic functionality not being
 available under all platforms. The Phoronix Test Suite software/framework is
-compatible with all major CPU architectures (e.g. i686, x86_64, ARM, RISC-V,
-PowerPC), but not all of the test profiles/suites are compatible with all
-architectures.
+compatible with all major CPU architectures (e.g. i686, x86_64, ARM / AArch64,
+RISC-V, POWER 64-bit), but not all of the test profiles/suites are compatible
+with all architectures.
 
 The Phoronix Test Suite can be installed for system-wide usage or run locally
 without installation from the extracted tar.gz/zip package. The only hard
@@ -64,7 +66,7 @@ dependency on the Phoronix Test Suite is having command-line support for PHP
 (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not**
 needed, but merely the PHP command-line support, which is widely available from
 operating system package managers under the name `php`, `php7-cli`, `php5-cli`,
-or `php7`.
+`php7` or `php8`.
 
 ## Usage
 
@@ -74,7 +76,9 @@ smallpt` to run a simple CPU test profile. If wishing to simply install a test,
 it's a matter of running `phoronix-test-suite install <test or suite name>` and
 to run it's `phoronix-test-suite run <test or suite name>`. There's also a batch
 mode for non-interactive benchmarking by first running `phoronix-test-suite
-batch-setup` and then using the `batch-run` sub-command rather than `run`.
+batch-setup` and then using the `batch-run` sub-command rather than `run`. For
+more guidance see the [Phoronix Test Suite
+documentation](documentation/phoronix-test-suite.md).
 
 Viewing installed system hardware and software is available via
 `phoronix-test-suite system-info`.
@@ -84,9 +88,3 @@ running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if
 wishing to compare the results of the
 `https://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
 
-Additional information is available from the Phoronix Test Suite website
-<https://www.phoronix-test-suite.com/> and the material bundled within the
-`phoronix-test-suite/documentation/` directory or via
-<https://www.phoronix-test-suite.com/documentation/>. A man page is also bundled
-with the phoronix-test-suite software.
-

SECURITY.md

@@ -0,0 +1,5 @@
+# Issue Reporting
+
+Public issues can be reported on GitHub via https://github.com/phoronix-test-suite/phoronix-test-suite/issues
+
+Security issues can be reported via email to security@phoronix-test-suite.com

deploy/docker/ubuntu-pts-docker-build.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+
+export TESTS_TO_PRECACHE=""
+
+# Ensure Docker on system
+DIR_NAME=phoronix-pts-docker
+mkdir -p ./$DIR_NAME/base/
+OS_OUTER=`pwd`
+cd $DIR_NAME/base
+OS_ROOT_PATH=`pwd`
+
+rm -rf phoronix-test-suite/
+git clone https://github.com/phoronix-test-suite/phoronix-test-suite.git
+cd phoronix-test-suite
+rm -rf .git
+
+# cache OpenBenchmarking.org metadata
+export PTS_USER_PATH_OVERRIDE=$OS_ROOT_PATH/var/lib/phoronix-test-suite/
+rm -f $PTS_USER_PATH_OVERRIDE
+mkdir -p $PTS_USER_PATH_OVERRIDE
+./phoronix-test-suite make-openbenchmarking-cache lean
+
+# cache select tests
+export PTS_DOWNLOAD_CACHE_OVERRIDE=$OS_ROOT_PATH/var/cache/phoronix-test-suite/download-cache/
+mkdir -p $PTS_DOWNLOAD_CACHE_OVERRIDE
+export PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT=1
+# ./phoronix-test-suite make-download-cache $TESTS_TO_PRECACHE
+# ./phoronix-test-suite info 1809091-PTS-CLEARLIN01
+
+rm -f $PTS_USER_PATH_OVERRIDE/core.pt2so
+
+# cleanup
+cd $OS_OUTER/$DIR_NAME
+
+tar -C base -cf base.tar .
+rm -f base.tar.xz
+xz -v -T0 base.tar
+
+cat > Dockerfile << EOF
+FROM ubuntu:20.04
+MAINTAINER Phoronix Media <commercial@phoronix-test-suite.com>
+ADD base.tar.xz /
+ARG DEBIAN_FRONTEND=noninteractive
+RUN apt update
+RUN apt install -y unzip php-cli apt-utils mesa-utils php-xml git-core apt-file sudo
+RUN apt-file update
+CMD ["/phoronix-test-suite/phoronix-test-suite", "shell"]
+EOF
+
+docker build -t $DIR_NAME .
+
+# docker run -it phoronix-pts-docker
+
+# docker tag phoronix-pts-docker phoronix/pts
+# docker push phoronix/pts
+

deploy/farm-system-customizations/intel-xorg-headless.conf

@@ -1,27 +0,0 @@
-Section "Monitor"
-    Identifier             "Monitor0"
-    Modeline "1920x1080" 174.83 1920 2056 2248 2536 1080 1081 1084 1149             # 174.83 MHz,  68.94 kHz,  60.00 Hz
-    Option      "PreferredMode"  "1920x1080"
-        Option "Enable" "true"
-EndSection
-
-Section "Device"
-    Identifier             "Device0"
-    Driver                 "intel" #Choose the driver used for this monitor
-        Option "DDC" "0"
-        Option "Virtualheads" "1"
-        Option "Vsync" "0"
-        Option "SwapbuffersWait" "0"
-	Option "DRI" "3"
-EndSection
-
-Section "Screen"
-    Identifier             "Screen0"  #Collapse Monitor and Device section to Screen section
-    Device                 "Device0"
-    Monitor                "Monitor0"
-    DefaultDepth           24 #Choose the depth (16||24)
-    SubSection             "Display"
-        Depth              24
-        Modes              "1920x1080" #Choose the resolution
-    EndSubSection
-EndSection

deploy/farm-system-customizations/radeon-xorg-headless.conf

@@ -1,23 +0,0 @@
-Section "Device"
-    Identifier             "Device0"
-    Driver                 "radeon"
-	Option "SwapBuffersWait" "0"
-		Option "DRI "3"
-EndSection
-
-Section "Monitor"
-    Identifier             "Monitor0"
-        Option "enable" "true"
-     Modeline "1920x1080" 148.50 1920 2448 2492 2640 1080 1084 1089 1125 +hsync +vsync
-EndSection
-
-Section "Screen"
-        Identifier "Fake"
-        Device "Device0"
-        Monitor "Monitor0"
-        Option "ConnectedMonitor" "Monitor0"
-        SubSection "Display"
-                Depth 24
-                Modes "1920x1080"
-        EndSubSection
-EndSection

deploy/farm-system-customizations/ubuntu-initial-setup.sh

@@ -1,10 +0,0 @@
-#!/bin/sh
-
-sudo apt-get install openssh-server git-core php5-cli
-
-echo "GRUB_RECORDFAIL_TIMEOUT=0" >> /etc/default/grub
-update-grub
-
-sudo passwd
-
-# check if needing "tty -s &&" for auto log-in user in ~/.profile

deploy/phoromatic-systemd/phoronix-result-server.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Phoronix Test Suite Result viewer
+After=syslog.target network.target
+Documentation=man:phoronix-test-suite(1)
+
+[Service]
+TimeoutStartSec=300
+ExecStart=/usr/bin/phoronix-test-suite start-result-viewer daemon
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target
+
+

deploy/rpm-package/build-package-rpm.php

@@ -75,7 +75,7 @@ $spec_file .= "%{_bindir}/phoronix-test-suite\n";
 $spec_file .= "%{_datadir}/phoronix-test-suite/*\n";
 $spec_file .= "%{_datadir}/applications/*\n";
 $spec_file .= "%{_datadir}/icons/hicolor/*\n";
-$spec_file .= "%{_datadir}/appdata/%{name}.appdata.xml\n";
+$spec_file .= "%{_datadir}/metainfo/com.phoronix_test_suite.phoronix_test_suite.metainfo.xml\n";
 $spec_file .= "%{_datadir}/doc/*\n";
 $spec_file .= "%{_mandir}/man1/%{name}.1*\n";
 $spec_file .= "%config(noreplace) %{_sysconfdir}/bash_completion.d\n";

documentation/man-pages/phoronix-test-suite.1

@@ -1,4 +1,4 @@
-.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "9.8.0m1"
+.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "10.8.5"
 .SH NAME
 phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.
 .SH SYNOPSIS
@@ -10,9 +10,6 @@ The **Phoronix Test Suite** is the most comprehensive testing and benchmarking p
 .SH OPTIONS
 .TP
 .SH SYSTEM
-.B diagnostics
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
 .B interactive
 A simple text-driven interactive interface to the Phoronix Test Suite.
 .TP
@@ -25,6 +22,9 @@ A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal
 .B system-info
 Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
 .TP
+.B system-properties
+Display various hardware/software system properties detected by the Phoronix Device Interface (Phodevi) library.
+.TP
 .B system-sensors
 Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
 .TP
@@ -48,11 +48,14 @@ This option will permanently remove a installed test by the Phoronix Test Suite.
 .B benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.
 .TP
+.B estimate-install-time [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will provide estimates for test install/setup time length.
+.TP
 .B estimate-run-time [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will provide estimates for test run-time / length.
 .TP
 .B finish-run [Test Result]
-This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish testing on the remaining tests where there are missing results.
+This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish / resume testing on the remaining tests where there are missing results to be completed.
 .TP
 .B run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s).
@@ -107,12 +110,24 @@ This option and its arguments pre-set the Phoronix Test Suite batch run mode wit
 .B clone-result [OpenBenchmarking ID]  ...
 This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.
 .TP
+.B dump-suites-to-git
+This option will create a Git repository of OpenBenchmarking.org test suites.
+.TP
+.B dump-tests-to-git
+This option will create a Git repository of OpenBenchmarking.org test profiles.
+.TP
+.B enable-repo
+This option is used if wanting to add a new OpenBenchmarking.org account/repository to your system for enabling third-party/unofficial test profiles and test suites.
+.TP
 .B list-recommended-tests
 This option will list recommended test profiles for benchmarking sorted by hardware sub-system. The recommended tests are determined via querying OpenBenchmarking.org and determining the most popular tests for a given environment based upon the number of times a test profile has been downloaded, the number of test results available on OpenBenchmarking.org for a given test profile, the age of the test profile, and other weighted factors.
 .TP
 .B make-openbenchmarking-cache
 This option will attempt to cache the test profile/suite meta-data from OpenBenchmarking.org for all linked repositories. This is useful if you're going to be running the Phoronix Test Suite / Phoromatic behind a firewall or without any Internet connection. Those with unrestricted Internet access or not utilizing a large local deployment of the Phoronix Test Suite / Phoromatic shouldn't need to run this command.
 .TP
+.B ob-test-profile-analyze
+This option is intended for test profile creators and generates a range of meta-data and other useful information that can be submitted to OpenBenchmarking.org to provide more verbose information for users of your test profiles.
+.TP
 .B openbenchmarking-changes
 This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.
 .TP
@@ -120,7 +135,7 @@ This option will list recent changes to test profiles of enabled OpenBenchmarkin
 This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.
 .TP
 .B openbenchmarking-refresh
-This option is used for refreshing the stored OpenBenchmarking.org repostory information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.
+This option is used for refreshing the stored OpenBenchmarking.org repository information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.
 .TP
 .B openbenchmarking-repositories
 This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.
@@ -154,7 +169,7 @@ This option will list all test profiles that are available from the enabled Open
 This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
 .TP
 .B list-available-tests
-This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necessary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
 .TP
 .B list-available-virtual-suites
 This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
@@ -183,11 +198,14 @@ This option will list all of the packages / external test dependencies that are
 .B list-saved-results
 This option will list all of the saved test results found on the system.
 .TP
+.B list-test-status
+This sub-command provides a verbose look at all tests installed/uninstalled on the system and whether any errors were encountered at install-time or run-time and other test installation/runtime metrics for complementing other Phoronix Test Suite sub-command outputs.
+.TP
 .B list-test-usage
 This option will list various details about installed tests and their usage.
 .TP
-.B list-unsupported-tests
-This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.
+.B print-tests [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will print the test identifiers of the specified result file(s), test suite(s), OpenBenchmarking.org ID(s), or other runnable object(s).
 .TP
 .B search
 This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..
@@ -217,15 +235,15 @@ This option will check all download links within the specified test profile(s) t
 .B download-test-files [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.
 .TP
+.B dump-documentation
+This option is used for re-generating the Phoronix Test Suite documentation.
+.TP
 .B inspect-test-profile [Test]
 This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
 .TP
 .B rebuild-test-suite [Suite]
 This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.
 .TP
-.B result-file-to-suite [Test Result]
-This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
-.TP
 .B validate-result-file
 This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.
 .TP
@@ -236,14 +254,11 @@ This option can be used for validating a Phoronix Test Suite test profile as bei
 This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.
 .TP
 .SH RESULT MANAGEMENT
-.B analyze-run-times [Test Result]
-This option will read a saved test results file and print the statistics about how long the testing took to complete.
-.TP
 .B auto-sort-result-file [Test Result]
-This option is used if you wish to automatically attempt to sort the results by their result identifier string.
+This option is used if you wish to automatically attempt to sort the results by their result identifier string. Alternatively, if using the environment variable "SORT_BY" other sort modes can be used, such as SORT_BY=date / SORT_BY=date-desc for sorting by the test run-time/date.
 .TP
 .B compare-results-to-baseline [Test Result] [Test Result]
-This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.
+This option allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.
 .TP
 .B compare-results-two-way [Test Result]
 This option will allows you to specify a result file and from there to compare two individual runs within that result file for looking at wins/losses and other metrics in a head-to-head type comparison.
@@ -252,13 +267,22 @@ This option will allows you to specify a result file and from there to compare t
 This option is used if you wish to edit the title and description of an existing result file.
 .TP
 .B extract-from-result-file [Test Result]
-This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.
+This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is then prompted to specify a new result file name and select which result identifier to extract.
+.TP
+.B keep-results-in-result-file [Test Result]
+This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.
 .TP
 .B merge-results [Test Result]  ...
 This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.
 .TP
-.B refresh-graphs [Test Result]
-This option will re-render and save all result graphs within a saved file. This option can be used when making modifications to the graphing code or its color/option configuration file and testing the changes.
+.B prune-empty-results [Test Result]
+This option is used if there are test results (benchmarks) to be dropped from a given result file that had no successful runs. I.e. if any run attempt(s) only errored out without producing any result for any of the test run(s) saved in that file. The user must specify a saved results file.
+.TP
+.B prune-zero-data [Test Result]
+This option is used if there are test results (benchmarks) to be where there is line graph data that currently has zero values (e.g. inaccurate/invalid sensor readings) and you wish to just drop those zero reading values from the result file.
+.TP
+.B remove-incomplete-results-from-result-file [Test Result]
+This option is used if there are test results (benchmarks) to be dropped from a given result file for having incomplete data, either a test run did not attempt to run that benchmark or failed to properly run. The user must specify a saved results file and the command will then attempt to find any results with incomplete/missing data and prompt the user with confirmation to remove them.
 .TP
 .B remove-result [Test Result]
 This option will permanently remove the saved file set that is set as the first argument.
@@ -281,27 +305,9 @@ This option is used if you wish to change the name of the saved name of a result
 .B reorder-result-file [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 .TP
-.B result-file-raw-to-csv [Test Result]
-This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
-.TP
-.B result-file-to-csv [Test Result]
-This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
-.TP
-.B result-file-to-json [Test Result]
-This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation).
-.TP
-.B result-file-to-pdf [Test Result]
-This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file.
-.TP
-.B result-file-to-text [Test Result]
-This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.
-.TP
 .B show-result [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 .TP
-.B workload-topology [Test Result]
-This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.
-.TP
 .SH OTHER
 .B commands
 This option will display a short list of possible Phoronix Test Suite commands.
@@ -310,10 +316,10 @@ This option will display a short list of possible Phoronix Test Suite commands.
 This option is used for testing the distribution-specific dependency handler for external dependencies.
 .TP
 .B debug-render-test
-This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.
+This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code paths. This option will download a large number of reference test results from LinuxBenchmarking.com.
 .TP
 .B debug-self-test
-This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.
+This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code paths and looking for hot areas to optimize, etc.
 .TP
 .B help
 This option will display a list of available Phoronix Test Suite commands and possible parameter types.
@@ -322,8 +328,8 @@ This option will display a list of available Phoronix Test Suite commands and po
 This option will display the Phoronix Test Suite client version.
 .TP
 .SH RESULT ANALYSIS
-.B analyze-all-runs [Test Result]
-This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+.B analyze-run-times [Test Result]
+This option will read a saved test results file and print the statistics about how long the testing took to complete.
 .TP
 .B executive-summary [Test Result]
 This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.
@@ -337,6 +343,9 @@ This option is used if you wish to analyze a result file by seeing various stati
 .B wins-and-losses [Test Result]
 This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
 .TP
+.B workload-topology [Test Result]
+This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology / make-up / logical groupings of the benchmarks being run.
+.TP
 .SH MODULES
 .B auto-load-module
 This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
@@ -356,9 +365,40 @@ This option can be used for debugging a Phoronix Test Suite module.
 .B unload-module
 This option can be used for easily removing a module from the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.
 .TP
+.SH DEBUGGING
+.B check-tests [Test]
+This option will perform a check on one or more test profiles to determine if there have been any vendor changes to the filename, filesize, URL location, MD5 and SHA256 checksums.
+.TP
+.B diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B dump-file-info
+This option will dump the MD5 / SHA256 checksums and file size for a given file.
+.TP
+.B dump-openbenchmarking-indexes
+This option is used for dumping the parsed output of OpenBenchmarking.org index files (metadata).
+.TP
+.B dump-phodevi-smart-cache
+This option is used for displaying the contents of the Phodevi smart cache on the system.
+.TP
+.B dump-possible-options
+This option will print all possible phoronix-test-suite sub-commands.
+.TP
+.B dump-unhandled-dependencies
+This option will list missing entries in the external dependencies XML file for the operating system under test. This option is used if wanting to help find missing dependency XML data to fill in for contributing to upstream Phoronix Test Suite.
+.TP
+.B list-failed-installs
+This option will list all test profiles that were attempted to be installed on the local system but failed to be installed. Where applicable, the possible error(s) from the test installation are also reported to assist in debugging.
+.TP
+.B list-test-errors
+This sub-command is complementary to list-failed-installs. Rather than listing test installation errors, list-test-errors is used for displaying past test run-time errors. This option will list all test profiles that produced an error previously when running the test profile / benchmark. If a test profile later successfully ran the test with any given option(s) without errors, the error is then removed from the archive. This option is intended to be helpful in debugging test profile issues later on for having a persistent collection of run-time errors.
+.TP
+.B list-unsupported-tests
+This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.
+.TP
 .SH USER CONFIGURATION
 .B enterprise-setup
-This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.
+This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes. It also defaults the anonymous usage reporting to being disabled, along with other conservative settings.
 .TP
 .B network-info
 This option will print information detected by the Phoronix Test Suite around the system's network configuration.
@@ -372,6 +412,31 @@ This option can be used for resetting the Phoronix Test Suite user configuration
 .B user-config-set
 This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
 .TP
+.B variables
+This option will print all of the official environment variables supported by the Phoronix Test Suite for user configuration purposes. These environment variables are also listed as part of the official Phoronix Test Suite documentation while this command will also show the current value of the variables if currently set.
+.TP
+.SH RESULT EXPORT
+.B result-file-raw-to-csv [Test Result]
+This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
+.TP
+.B result-file-to-csv [Test Result]
+This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output (Comma Separated Values) file. This file can then be loaded into a spreadsheet for easy viewing. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.
+.TP
+.B result-file-to-html [Test Result]
+This option will read a saved test results file and output the system hardware and software information along with the results to pure HTML file. No external files are required for CSS/JavaScript or other assets. The graphs are rendered as inline SVG. This is a pure HTML-only representation of the results for emailing or other easy analysis outside of the Phoronix Test Suite. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.
+.TP
+.B result-file-to-json [Test Result]
+This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation). The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.
+.TP
+.B result-file-to-pdf [Test Result]
+This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.
+.TP
+.B result-file-to-suite [Test Result]
+This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
+.TP
+.B result-file-to-text [Test Result]
+This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.
+.TP
 .SH PHOROMATIC
 .B start-phoromatic-server
 Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.
@@ -391,7 +456,6 @@ https://www.openbenchmarking.org/
 .br
 https://www.phoronix.com/
 .br
-https://www.phoronix.com/forums/
 .SH AUTHORS
-Copyright 2008 - 2020 by Phoronix Media, Michael Larabel.
+Copyright 2008 - 2024 by Phoronix Media, Michael Larabel.
 .TP

documentation/phoromatic.html

@@ -1,90 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite Phoromatic</Title>
-</head>
-<body>
-<h1>Phoromatic Server</h1>
-<h2>Introduction</h2>
-<p>Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name <em>Phoro-matic</em>) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.</p>
-<p>Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.</p>
-<p>Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.</p>
-<h2>Features</h2>
-<p>Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:</p>
-<h3>Automated Scheduling</h3>
-<p>Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.</p>
-<h3>Extensible</h3>
-<p>Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).</p>
-<h3>Remote Testing</h3>
-<p>Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.</p>
-<h3>Multi-System Support</h3>
-<p>A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.</p>
-<h3>Turn-Key Deployment</h3>
-<p>No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.</p>
-<h3>Result Management</h3>
-<p>Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.</p>
-<h3>Decentralized</h3>
-<p>Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.</p>
-<h3>Fully Open-Source</h3>
-<p>Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.</p>
-<h2>Phoromatic Server Setup</h2>
-<p>Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the <strong>phoronix-test-suite</strong>. Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within <em>~/.phoronix-test-suite/user-config.xml</em>. The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.</p>
-<p>On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as <u>avahi-tools</u>), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.</p>
-<p>Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the <em>~/.phoronix-test-suite/user-config.xml</em> file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.</p>
-<p>The Phoromatic Server utilizes PHP/HHVM's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.</p>
-<p>Systemd and Upstart service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP or HHVM is recommended for offering the best support.</p>
-<h2>Example Deployments</h2>
-<h3>Use Case A: Unrestricted Internet Access, Local Result Storage</h3>
-<p>Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, simply run:</p>
-<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
-<p>Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.</p>
-<p>From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running <strong>phoronix-test-suite phoromatic.connect</strong> with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.</p>
-<h3>Use Case B: No Internet Available To Client Systems</h3>
-<p>It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, a few commands from the system while having an Internet connection will be able to cache the needed data:</p>
-<p><strong>$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg</strong></p>
-<p>This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to <em>~/.phoronix-test-suite/download-cache/</em> or <em>/usr/share/phoronix-test-suite/download-cache/</em> depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.</p>
-<p><strong>$ phoronix-test-suite make-openbenchmarking-cache</strong></p>
-<p>This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.</p>
-<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
-<p>Proceed to start the Phoromatic Server and operate as normal.</p>
-<p>For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.</p>
-<h3>Use Case C: Phoromatic Across The Internet</h3>
-<p>If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's <em>PhoromaticServers</em> element within the <em>user-config.xml</em>. Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.</p>
-<h2>Client Setup</h2>
-<p>From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:</p>
-<p><strong>$ phoronix-test-suite phoromatic.explore</strong></p>
-<p>With the following example output on finding one successful server:</p>
-<p><em>IP: 192.168.1.211<br />
-HTTP PORT: 5447<br />
-WEBSOCKET PORT: 5427<br />
-SERVER: PHP 5.5.9-1ubuntu4.4 Development Server<br />
-PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]<br />
-DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE<br />
-SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:<br />
-      pts - Last Generated: 05 Oct 2014 07:16</em></p>
-<p>Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's <em>~/.phoronix-test-suite/user-config.xml</em> with the <em>PhoromaticServers</em> element. Adding the <em>IP:port</em> (the Phoromatic Server's HTTP port) to the PhoromaticServers <em>user-config.xml</em> element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.</p>
-<p>To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. <em>phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY</em>. When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the <strong>phoronix-test-suite phoromatic.connect</strong> command running on the system (after the initial account has been synced, simply running <strong>phoronix-test-suite phoromatic.connect</strong> is enough for the system to find the server and its account).</p>
-<h2>Root Administrator</h2>
-<p>The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.</p>
-<p>To enable the root administrator log-in, first from the server's command-line interface run <strong>phoronix-test-suite phoromatic.set-root-admin-password</strong> to set the password. Following that, you can log into the root administrator account via the web interface via the <em>rootadmin</em> user-name and the set password.</p>
-<h2>Other Advice</h2>
-<h3>Disable Internet Precaution</h3>
-<p>If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the <em>~/.phoronix-test-suite/user-config.xml</em> you can set <em>NoInternetCommunication</em> to <em>TRUE</em>. There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.</p>
-<h3>Ports / Services</h3>
-<p>The Phoromatic Server process currently relies upon a PHP/HHVM built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.</p>
-<h3>Systemd / Upstart</h3>
-<p>Packaged with the Phoronix Test Suite are basic <em>phoromatic-client</em> and <em>phoromatic-server</em> configurations for both Upstart and systemd init systems. The <em>phoromatic-server</em> configuration will launch the Phoronix Test Suite's Phoromatic Server and the <em>phoromatic-client</em> service will attempt to connect to a <u>pre-configured</u> Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite <em>install-sh</em> process while the Upstart jobs can be copied from <em>deploy/phoromatic-upstart/*</em> to <em>/etc/init</em>.</p>
-<h3>Cache Verification</h3>
-<p>To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the <em>settings</em> page followed by the <em>cache settings</em> link to view information about the download and OpenBenchmarking.org caches. From the client systems, running <strong>phoronix-test-suite phoromatic.explore</strong> will also supply cache statistics.</p>
-<h3>Log Files</h3>
-<p>The Phoromatic Server will produce a log file of events / debugging information to <em>~/.phoronix-test-suite/phoromatic.log</em> or <em>/var/log/phoromatic.log</em> depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in <em>phoronix-test-suite.log</em>.</p>
-<h3>Multi-User Accounts</h3>
-<p>For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the <em>Users</em> tab of their account. The main administrator can make an additional administrator account or a &quot;viewer&quot; account that can consume the account's data but not create/modify the schedules, systems, or other account details.</p>
-<h3>
-<h3>File Locations</h3>
-<p>When running the Phoronix Test Suite Phoromatic Server as root, rather than using the <em>~/.phoronix-test-suite/</em> directory, the standard Linux file-system hierarchy standard is honored. The main storage path is <em>/var/lib/phoronix-test-suite/</em>, the user configuration file is <em>/etc/phoronix-test-suite.xml</em>, and <em>/var/cache/phoronix-test-suite/</em> for cache files.</p>
-<h3>Uploading Other Test Results</h3>
-<p>Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the <em>phoronix-test-suite phoromatic.upload-result &lt;result file identifier &gt;</em> sub-command.</p>
-<h3>User Context File Logging</h3>
-<p>For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the <em>PHOROMATIC_LOG_FILE</em> environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include <em>PHOROMATIC_TRIGGER</em>, <em>PHOROMATIC_SCHEDULE_ID</em>, and <em>PHOROMATIC_SCHEDULE_PROCESS</em>.</p>
-</body>
-</html>

documentation/phoromatic.md

@@ -1,139 +0,0 @@
-
-# Phoronix Test Suite Phoromatic
-
-## Phoromatic Server
-
-### Introduction
-Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name *Phoro-matic* ) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.
-
-Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.
-
-Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.
-
-
-### Features
-Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:
-
-#### Automated Scheduling
-Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.
-
-#### Extensible
-Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).
-
-#### Remote Testing
-Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.
-
-#### Multi-System Support
-A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.
-
-#### Turn-Key Deployment
-No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.
-
-#### Result Management
-Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.
-
-#### Decentralized
-Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.
-
-#### Fully Open-Source
-Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.
-
-
-### Phoromatic Server Setup
-Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the **phoronix-test-suite** . Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within *~/.phoronix-test-suite/user-config.xml* . The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.
-
-On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as _avahi-tools_ ), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.
-
-Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the *~/.phoronix-test-suite/user-config.xml* file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.
-
-The Phoromatic Server utilizes PHP/HHVM's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.
-
-Systemd and Upstart service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP or HHVM is recommended for offering the best support.
-
-
-### Example Deployments
-#### Use Case A: Unrestricted Internet Access, Local Result Storage
-Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, simply run:
-
-**$ phoronix-test-suite start-phoromatic-server**
-
-Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.
-
-From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running **phoronix-test-suite phoromatic.connect** with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.
-
-#### Use Case B: No Internet Available To Client Systems
-It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, a few commands from the system while having an Internet connection will be able to cache the needed data:
-
-**$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg**
-
-This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to *~/.phoronix-test-suite/download-cache/* or */usr/share/phoronix-test-suite/download-cache/* depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.
-
-**$ phoronix-test-suite make-openbenchmarking-cache**
-
-This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.
-
-**$ phoronix-test-suite start-phoromatic-server**
-
-Proceed to start the Phoromatic Server and operate as normal.
-
-For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.
-
-#### Use Case C: Phoromatic Across The Internet
-If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's *PhoromaticServers* element within the *user-config.xml* . Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.
-
-
-### Client Setup
-From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:
-
-**$ phoronix-test-suite phoromatic.explore**
-
-With the following example output on finding one successful server:
-
-*IP: 192.168.1.211
-HTTP PORT: 5447
-WEBSOCKET PORT: 5427
-SERVER: PHP 5.5.9-1ubuntu4.4 Development Server
-PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]
-DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE
-SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:
-      pts - Last Generated: 05 Oct 2014 07:16*
-
-Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's *~/.phoronix-test-suite/user-config.xml* with the *PhoromaticServers* element. Adding the *IP:port* (the Phoromatic Server's HTTP port) to the PhoromaticServers *user-config.xml* element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.
-
-To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. *phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY* . When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the **phoronix-test-suite phoromatic.connect** command running on the system (after the initial account has been synced, simply running **phoronix-test-suite phoromatic.connect** is enough for the system to find the server and its account).
-
-
-### Root Administrator
-The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.
-
-To enable the root administrator log-in, first from the server's command-line interface run **phoronix-test-suite phoromatic.set-root-admin-password** to set the password. Following that, you can log into the root administrator account via the web interface via the *rootadmin* user-name and the set password.
-
-
-### Other Advice
-#### Disable Internet Precaution
-If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the *~/.phoronix-test-suite/user-config.xml* you can set *NoInternetCommunication* to *TRUE* . There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.
-
-#### Ports / Services
-The Phoromatic Server process currently relies upon a PHP/HHVM built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.
-
-#### Systemd / Upstart
-Packaged with the Phoronix Test Suite are basic *phoromatic-client* and *phoromatic-server* configurations for both Upstart and systemd init systems. The *phoromatic-server* configuration will launch the Phoronix Test Suite's Phoromatic Server and the *phoromatic-client* service will attempt to connect to a _pre-configured_ Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite *install-sh* process while the Upstart jobs can be copied from *deploy/phoromatic-upstart/** to */etc/init* .
-
-#### Cache Verification
-To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the *settings* page followed by the *cache settings* link to view information about the download and OpenBenchmarking.org caches. From the client systems, running **phoronix-test-suite phoromatic.explore** will also supply cache statistics.
-
-#### Log Files
-The Phoromatic Server will produce a log file of events / debugging information to *~/.phoronix-test-suite/phoromatic.log* or */var/log/phoromatic.log* depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in *phoronix-test-suite.log* .
-
-#### Multi-User Accounts
-For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the *Users* tab of their account. The main administrator can make an additional administrator account or a "viewer" account that can consume the account's data but not create/modify the schedules, systems, or other account details.
-
-#### File Locations
-When running the Phoronix Test Suite Phoromatic Server as root, rather than using the *~/.phoronix-test-suite/* directory, the standard Linux file-system hierarchy standard is honored. The main storage path is */var/lib/phoronix-test-suite/* , the user configuration file is */etc/phoronix-test-suite.xml* , and */var/cache/phoronix-test-suite/* for cache files.
-
-#### Uploading Other Test Results
-Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the *phoronix-test-suite phoromatic.upload-result <result file identifier >* sub-command.
-
-#### User Context File Logging
-For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the *PHOROMATIC_LOG_FILE* environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include *PHOROMATIC_TRIGGER* , *PHOROMATIC_SCHEDULE_ID* , and *PHOROMATIC_SCHEDULE_PROCESS* .
-

documentation/phoronix-test-suite-window.md

@@ -1,40 +0,0 @@
-
-# Phoronix Test Suite On Windows
-
-### Introduction
-Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.
-
-The Phoronix Test Suite Windows support currently targets **Windows 10 x64** and **Windows Server 2016 x64** . Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.
-
-
-### Windows Setup / Dependencies
-As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the *phoronix-test-suite.bat* file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under *C:\PHP* as the default location should PHP support not be found within your system's *Program Files* directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.
-
-The Phoronix Test Suite on Windows does depend upon [Cygwin](https://www.cygwin.com/) for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of *C:\cygwin64* .
-
-Various test profiles may depend upon other "external dependencies" like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.
-
-
-### Running The Phoronix Test Suite On Windows
-The Phoronix Test Suite can run from its local directory and does not need to be "installed" to a system path or any other "setup" process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:
-
-1. Download the Phoronix Test Suite 8.0+ or [Phoronix-Test-Suite from GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite) ( [zip file](https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip) ).
-
-2. From the Command Prompt or PowerShell, enter the *phoronix-test-suite* directory whether it be from Git or a zipped download.
-
-3. Run the *phoronix-test-suite.bat* file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.
-
-4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the **phoronix-test-suite shell** command.
-
-
-### Test Profiles On Windows
-As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.
-
-
-### Getting Started
-Besides **phoronix-test-suite shell** and **phoronix-test-suite help** , there is also **phoronix-test-suite interactive** for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running **phoronix-test-suite benchmark c-ray** or **phoronix-test-suite benchmark crafty** as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.
-
-
-### Support
-Community technical support is available via [GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite/issues) or general inquiries via [the Phoronix Forums](https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite) . For enterprise inquiries, commercial support, and custom engineering services, [contact us](http://phoronix-test-suite.com/?k=commercial) .
-

documentation/phoronix-test-suite-windows.html

@@ -1,27 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite On Windows</Title>
-</head>
-<body>
-
-<h2>Introduction</h2>
-<p>Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.</p>
-<p>The Phoronix Test Suite Windows support currently targets <strong>Windows 10 x64</strong> and <strong>Windows Server 2016 x64</strong>. Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.</p>
-<h2>Windows Setup / Dependencies</h2>
-<p>As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the <em>phoronix-test-suite.bat</em> file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under <em>C:\PHP</em> as the default location should PHP support not be found within your system's <em>Program Files</em> directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.</p>
-<p>The Phoronix Test Suite on Windows does depend upon <a href="https://www.cygwin.com/">Cygwin</a> for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of <em>C:\cygwin64</em>.</p>
-<p>Various test profiles may depend upon other &quot;external dependencies&quot; like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.</p>
-<h2>Running The Phoronix Test Suite On Windows</h2>
-<p>The Phoronix Test Suite can run from its local directory and does not need to be &quot;installed&quot; to a system path or any other &quot;setup&quot; process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:</p>
-<p>1. Download the Phoronix Test Suite 8.0+ or <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">Phoronix-Test-Suite from GitHub</a> (<a href="https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip">zip file</a>).</p>
-<p>2. From the Command Prompt or PowerShell, enter the <em>phoronix-test-suite</em> directory whether it be from Git or a zipped download.</p>
-<p>3. Run the <em>phoronix-test-suite.bat</em> file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.</p>
-<p>4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the <strong>phoronix-test-suite shell</strong> command.</p>
-<h2>Test Profiles On Windows</h2>
-<p>As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.</p>
-<h2>Getting Started</h2>
-<p>Besides <strong>phoronix-test-suite shell</strong> and <strong>phoronix-test-suite help</strong>, there is also <strong>phoronix-test-suite interactive</strong> for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running <strong>phoronix-test-suite benchmark c-ray</strong> or <strong>phoronix-test-suite benchmark crafty</strong> as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.</p>
-<h2>Support</h2>
-<p>Community technical support is available via <a href="https://github.com/phoronix-test-suite/phoronix-test-suite/issues">GitHub</a> or general inquiries via <a href="https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite">the Phoronix Forums</a>. For enterprise inquiries, commercial support, and custom engineering services, <a href="http://phoronix-test-suite.com/?k=commercial">contact us</a>.</p>
-</body>
-</html>

documentation/stubs/00_overview.html

@@ -5,14 +5,15 @@
 <body>
 <h1>Overview</h1>
 <p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
-<p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 400 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
+<p>The Phoronix Test Suite client itself is an automated test framework for providing seamless execution of test profiles and test suites. There are more than 650 tests available by default, which are transparently available via <a href="https://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and workstations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
 <p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
 <h1>OpenBenchmarking.org</h1>
 <p>OpenBenchmarking.org is an open, collaborative testing platform that makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite test client. OpenBenchmarking.org serves as a repository for storing test profiles, test suites, and result data. Test profiles and suites are stored in the OpenBenchmarking.org cloud to allow for new/updated tests to be seamlessly obtained via the Phoronix Test Suite without needing to manually update the Phoronix Test Suite client. OpenBenchmarking.org also makes it easy to facilitate side-by-side comparisons with any other results stored in the OpenBenchmarking.org cloud. Any Phoronix Test Suite user is permitted to upload their test results, test profiles, and suites to OpenBenchmarking.org.</p>
-<p>When finding a set of results on OpenBenchmarking.org (e.g. <a href="http://openbenchmarking.org/result/1203160-BY-NVTEGRA3785">an example result file</a>), it's as easy as running the Phoronix Test Suite with that OpenBenchmarking.org ID to perform an automated side-by-side comparison (e.g. <em>phoronix-test-suite benchmark 1203160-BY-NVTEGRA3785</em>).</p>
+<p>When finding a set of results on OpenBenchmarking.org, it's as easy as running the Phoronix Test Suite with that OpenBenchmarking.org ID to perform an automated side-by-side comparison (e.g. <em>phoronix-test-suite benchmark 1203160-BY-NVTEGRA3785</em>).</p>
 <p>Thanks to the wealth of test data (results, system logs, etc) from crowd-sourced benchmarking via the Phoronix Test Suite, a plethora of analytical features are also available from OpenBenchmarking.org.</p>
 <h1>Phoromatic</h1>
 <p>Phoromatic is a remote management system for the Phoronix Test Suite that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. Phoromatic can also interface with revision control systems to offer support for issuing new tests on a context-basis, such as whenever a Git commit has been pushed or new daily image available. The test results are then available from this central, secure location.</p>
 <p>Phoromatic is an add-on to the Phoronix Test Suite that's primarily intended for enterprise users when facilitating tests across a wide-spectrum of hardware within a test lab or when needing to carry out tests on a routine basis.</p>
+<p>A Phoromatic server can be started using <em>phoronix-test-suite start-phoromatic-server</em> (or the included systemd phoromatic-server service file). Clients can connect to the server using the <em>phoronix-test-suite phoromatic.connect</em> command as well as a phoromatic-client systemd service. See the Phoromatic section of the documentation for more information on setting up Phoromatic.</p>
 </body>
 </html>

documentation/stubs/00_user_options.html

@@ -6,23 +6,17 @@
 <body>
 <p>The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running <em>phoronix-test-suite help.</em><hr></p>
 <h1>System</h1>
-<h3>diagnostics<em>  </em>
-</h3>
-<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>interactive<em>  </em>
-</h3>
+<h3>interactive</h3>
 <p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
-<h3>php-conf<em>  </em>
-</h3>
+<h3>php-conf</h3>
 <p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>shell<em>  </em>
-</h3>
+<h3>shell</h3>
 <p>A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite</p>
-<h3>system-info<em>  </em>
-</h3>
+<h3>system-info</h3>
 <p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
-<h3>system-sensors<em>  </em>
-</h3>
+<h3>system-properties</h3>
+<p>Display various hardware/software system properties detected by the Phoronix Device Interface (Phodevi) library.</p>
+<h3>system-sensors</h3>
 <p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
 <h1>Test Installation</h1>
 <h3>force-install<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
@@ -34,8 +28,7 @@
 <h3>install-dependencies<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the external dependencies needed by the selected test(s) using the distribution's package management system. For example, some tests depend upon GCC for compiling code. If GCC is not detected on the system, the Phoronix Test Suite will attempt to install GCC using the distribution's package management system. If you are running this command as a local user, you may be prompted for the root password while the process is running. For unsupported distributions, the dependency names will be displayed along with common names for the package. The install-dependencies option needs to be supplied with the test name or suite as an argument. When using the install option, the external dependencies are automatically checked.</p>
-<h3>make-download-cache<em>  </em>
-</h3>
+<h3>make-download-cache</h3>
 <p>This option will create a download cache for use by the Phoronix Test Suite. The download cache is created of test files already downloaded to the local system. If passing any test/suite names to make-download-cache, the needed files for those test profiles will first be automatically downloaded before creating the cache.</p>
 <h3>remove-installed-test<em>  [Test]</em>
 </h3>
@@ -44,17 +37,19 @@
 <h3>benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.</p>
+<h3>estimate-install-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
+</h3>
+<p>This option will provide estimates for test install/setup time length.</p>
 <h3>estimate-run-time<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will provide estimates for test run-time / length.</p>
 <h3>finish-run<em>  [Test Result]</em>
 </h3>
-<p>This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish testing on the remaining tests where there are missing results.</p>
+<p>This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish / resume testing on the remaining tests where there are missing results to be completed.</p>
 <h3>run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will run the selected test(s).</p>
-<h3>run-random-tests<em>  </em>
-</h3>
+<h3>run-random-tests</h3>
 <p>This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.</p>
 <h3>run-subset<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
@@ -84,8 +79,7 @@
 <h3>batch-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option and its arguments are equivalent to the run option, but the process will be run in the Phoronix Test Suite batch mode.</p>
-<h3>batch-setup<em>  </em>
-</h3>
+<h3>batch-setup</h3>
 <p>This option is used to configure the batch mode options for the Phoronix Test Suite, which is subsequently written to the user configuration file. Among the options are whether to automatically upload the test results to OpenBenchmarking.org and prompting for the saved file name.</p>
 <h3>default-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
@@ -103,35 +97,34 @@
 <h3>clone-result<em>  [OpenBenchmarking ID]  ...</em>
 </h3>
 <p>This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.</p>
-<h3>list-recommended-tests<em>  </em>
-</h3>
+<h3>dump-suites-to-git</h3>
+<p>This option will create a Git repository of OpenBenchmarking.org test suites.</p>
+<h3>dump-tests-to-git</h3>
+<p>This option will create a Git repository of OpenBenchmarking.org test profiles.</p>
+<h3>enable-repo</h3>
+<p>This option is used if wanting to add a new OpenBenchmarking.org account/repository to your system for enabling third-party/unofficial test profiles and test suites.</p>
+<h3>list-recommended-tests</h3>
 <p>This option will list recommended test profiles for benchmarking sorted by hardware sub-system. The recommended tests are determined via querying OpenBenchmarking.org and determining the most popular tests for a given environment based upon the number of times a test profile has been downloaded, the number of test results available on OpenBenchmarking.org for a given test profile, the age of the test profile, and other weighted factors.</p>
-<h3>make-openbenchmarking-cache<em>  </em>
-</h3>
+<h3>make-openbenchmarking-cache</h3>
 <p>This option will attempt to cache the test profile/suite meta-data from OpenBenchmarking.org for all linked repositories. This is useful if you're going to be running the Phoronix Test Suite / Phoromatic behind a firewall or without any Internet connection. Those with unrestricted Internet access or not utilizing a large local deployment of the Phoronix Test Suite / Phoromatic shouldn't need to run this command.</p>
-<h3>openbenchmarking-changes<em>  </em>
-</h3>
+<h3>ob-test-profile-analyze</h3>
+<p>This option is intended for test profile creators and generates a range of meta-data and other useful information that can be submitted to OpenBenchmarking.org to provide more verbose information for users of your test profiles.</p>
+<h3>openbenchmarking-changes</h3>
 <p>This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.</p>
-<h3>openbenchmarking-login<em>  </em>
-</h3>
+<h3>openbenchmarking-login</h3>
 <p>This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.</p>
-<h3>openbenchmarking-refresh<em>  </em>
-</h3>
-<p>This option is used for refreshing the stored OpenBenchmarking.org repostory information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.</p>
-<h3>openbenchmarking-repositories<em>  </em>
-</h3>
+<h3>openbenchmarking-refresh</h3>
+<p>This option is used for refreshing the stored OpenBenchmarking.org repository information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.</p>
+<h3>openbenchmarking-repositories</h3>
 <p>This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.</p>
-<h3>openbenchmarking-uploads<em>  </em>
-</h3>
+<h3>openbenchmarking-uploads</h3>
 <p>This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.</p>
-<h3>recently-added-tests<em>  </em>
-</h3>
+<h3>recently-added-tests</h3>
 <p>This option will list the most recently added (newest) test profiles.</p>
 <h3>upload-result<em>  [Test Result]</em>
 </h3>
 <p>This option is used for uploading a test result to OpenBenchmarking.org.</p>
-<h3>upload-test-profile<em>  </em>
-</h3>
+<h3>upload-test-profile</h3>
 <p>This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.</p>
 <h3>upload-test-suite<em>  [Suite]</em>
 </h3>
@@ -143,60 +136,45 @@
 <h3>intersect<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will print the test profiles present in all passed result files / test suites. Two or more results/suites must be passed and printed will be all of the common test profiles.</p>
-<h3>list-all-tests<em>  </em>
-</h3>
+<h3>list-all-tests</h3>
 <p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
-<h3>list-available-suites<em>  </em>
-</h3>
+<h3>list-available-suites</h3>
 <p>This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.</p>
-<h3>list-available-tests<em>  </em>
-</h3>
-<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.</p>
-<h3>list-available-virtual-suites<em>  </em>
-</h3>
+<h3>list-available-tests</h3>
+<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necessary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.</p>
+<h3>list-available-virtual-suites</h3>
 <p>This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.</p>
-<h3>list-cached-tests<em>  </em>
-</h3>
+<h3>list-cached-tests</h3>
 <p>This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.</p>
-<h3>list-installed-dependencies<em>  </em>
-</h3>
+<h3>list-installed-dependencies</h3>
 <p>This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.</p>
-<h3>list-installed-suites<em>  </em>
-</h3>
+<h3>list-installed-suites</h3>
 <p>This option will list all suites that are currently installed on the system.</p>
-<h3>list-installed-tests<em>  </em>
-</h3>
+<h3>list-installed-tests</h3>
 <p>This option will list all test profiles that are currently installed on the system.</p>
-<h3>list-missing-dependencies<em>  </em>
-</h3>
+<h3>list-missing-dependencies</h3>
 <p>This option will list all of the packages / external test dependencies that are missing from the system that the Phoronix Test Suite may potentially need by select test profiles.</p>
-<h3>list-not-installed-tests<em>  </em>
-</h3>
+<h3>list-not-installed-tests</h3>
 <p>This option will list all test profiles that are supported and available but presently NOT installed on the system.</p>
-<h3>list-possible-dependencies<em>  </em>
-</h3>
+<h3>list-possible-dependencies</h3>
 <p>This option will list all of the packages / external test dependencies that are are potentially used by the Phoronix Test Suite.</p>
-<h3>list-saved-results<em>  </em>
-</h3>
+<h3>list-saved-results</h3>
 <p>This option will list all of the saved test results found on the system.</p>
-<h3>list-test-usage<em>  </em>
-</h3>
+<h3>list-test-status</h3>
+<p>This sub-command provides a verbose look at all tests installed/uninstalled on the system and whether any errors were encountered at install-time or run-time and other test installation/runtime metrics for complementing other Phoronix Test Suite sub-command outputs.</p>
+<h3>list-test-usage</h3>
 <p>This option will list various details about installed tests and their usage.</p>
-<h3>list-unsupported-tests<em>  </em>
-</h3>
-<p>This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.</p>
-<h3>search<em>  </em>
+<h3>print-tests<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
+<p>This option will print the test identifiers of the specified result file(s), test suite(s), OpenBenchmarking.org ID(s), or other runnable object(s).</p>
+<h3>search</h3>
 <p>This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..</p>
-<h3>test-to-suite-map<em>  </em>
-</h3>
+<h3>test-to-suite-map</h3>
 <p>This option will list all test profiles and any test suites each test belongs to.</p>
 <h1>Asset Creation</h1>
-<h3>build-suite<em>  </em>
-</h3>
+<h3>build-suite</h3>
 <p>This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.</p>
-<h3>create-test-profile<em>  </em>
-</h3>
+<h3>create-test-profile</h3>
 <p>This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.</p>
 <h3>debug-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
@@ -213,17 +191,15 @@
 <h3>download-test-files<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.</p>
+<h3>dump-documentation</h3>
+<p>This option is used for re-generating the Phoronix Test Suite documentation.</p>
 <h3>inspect-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.</p>
 <h3>rebuild-test-suite<em>  [Suite]</em>
 </h3>
 <p>This option will regenerate the local test suite XML file against the OpenBenchmarking.org specification. This can be used to clean up any existing XML syntax / styling issues, etc.</p>
-<h3>result-file-to-suite<em>  [Test Result]</em>
-</h3>
-<p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
-<h3>validate-result-file<em>  </em>
-</h3>
+<h3>validate-result-file</h3>
 <p>This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.</p>
 <h3>validate-test-profile<em>  [Test]</em>
 </h3>
@@ -232,15 +208,12 @@
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.</p>
 <h1>Result Management</h1>
-<h3>analyze-run-times<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and print the statistics about how long the testing took to complete.</p>
 <h3>auto-sort-result-file<em>  [Test Result]</em>
 </h3>
-<p>This option is used if you wish to automatically attempt to sort the results by their result identifier string.</p>
+<p>This option is used if you wish to automatically attempt to sort the results by their result identifier string. Alternatively, if using the environment variable "SORT_BY" other sort modes can be used, such as SORT_BY=date / SORT_BY=date-desc for sorting by the test run-time/date.</p>
 <h3>compare-results-to-baseline<em>  [Test Result] [Test Result]</em>
 </h3>
-<p>This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.</p>
+<p>This option allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.</p>
 <h3>compare-results-two-way<em>  [Test Result]</em>
 </h3>
 <p>This option will allows you to specify a result file and from there to compare two individual runs within that result file for looking at wins/losses and other metrics in a head-to-head type comparison.</p>
@@ -249,13 +222,22 @@
 <p>This option is used if you wish to edit the title and description of an existing result file.</p>
 <h3>extract-from-result-file<em>  [Test Result]</em>
 </h3>
-<p>This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.</p>
+<p>This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is then prompted to specify a new result file name and select which result identifier to extract.</p>
+<h3>keep-results-in-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is the inverse of the remove-results-from-result-file sub-command. If you wish to remove all results but those listed from a given result file, this option can be used. The user must specify a saved results file and then they will be prompted to provide a string to search for in keeping those results in that given result file but removing all other data.</p>
 <h3>merge-results<em>  [Test Result]  ...</em>
 </h3>
 <p>This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.</p>
-<h3>refresh-graphs<em>  [Test Result]</em>
+<h3>prune-empty-results<em>  [Test Result]</em>
 </h3>
-<p>This option will re-render and save all result graphs within a saved file. This option can be used when making modifications to the graphing code or its color/option configuration file and testing the changes.</p>
+<p>This option is used if there are test results (benchmarks) to be dropped from a given result file that had no successful runs. I.e. if any run attempt(s) only errored out without producing any result for any of the test run(s) saved in that file. The user must specify a saved results file.</p>
+<h3>prune-zero-data<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there are test results (benchmarks) to be where there is line graph data that currently has zero values (e.g. inaccurate/invalid sensor readings) and you wish to just drop those zero reading values from the result file.</p>
+<h3>remove-incomplete-results-from-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there are test results (benchmarks) to be dropped from a given result file for having incomplete data, either a test run did not attempt to run that benchmark or failed to properly run. The user must specify a saved results file and the command will then attempt to find any results with incomplete/missing data and prompt the user with confirmation to remove them.</p>
 <h3>remove-result<em>  [Test Result]</em>
 </h3>
 <p>This option will permanently remove the saved file set that is set as the first argument.</p>
@@ -277,50 +259,26 @@
 <h3>reorder-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
-<h3>result-file-raw-to-csv<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
-<h3>result-file-to-csv<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.</p>
-<h3>result-file-to-json<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation).</p>
-<h3>result-file-to-pdf<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file.</p>
-<h3>result-file-to-text<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.</p>
 <h3>show-result<em>  [Test Result]</em>
 </h3>
 <p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
-<h3>workload-topology<em>  [Test Result]</em>
-</h3>
-<p>This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology/make-up / logical groupings of the benchmarks being run.</p>
 <h1>Other</h1>
-<h3>commands<em>  </em>
-</h3>
+<h3>commands</h3>
 <p>This option will display a short list of possible Phoronix Test Suite commands.</p>
-<h3>debug-dependency-handler<em>  </em>
-</h3>
+<h3>debug-dependency-handler</h3>
 <p>This option is used for testing the distribution-specific dependency handler for external dependencies.</p>
-<h3>debug-render-test<em>  </em>
-</h3>
-<p>This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.</p>
-<h3>debug-self-test<em>  </em>
-</h3>
-<p>This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.</p>
-<h3>help<em>  </em>
-</h3>
+<h3>debug-render-test</h3>
+<p>This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code paths. This option will download a large number of reference test results from LinuxBenchmarking.com.</p>
+<h3>debug-self-test</h3>
+<p>This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code paths and looking for hot areas to optimize, etc.</p>
+<h3>help</h3>
 <p>This option will display a list of available Phoronix Test Suite commands and possible parameter types.</p>
-<h3>version<em>  </em>
-</h3>
+<h3>version</h3>
 <p>This option will display the Phoronix Test Suite client version.</p>
 <h1>Result Analysis</h1>
-<h3>analyze-all-runs<em>  [Test Result]</em>
+<h3>analyze-run-times<em>  [Test Result]</em>
 </h3>
-<p>This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.</p>
+<p>This option will read a saved test results file and print the statistics about how long the testing took to complete.</p>
 <h3>executive-summary<em>  [Test Result]</em>
 </h3>
 <p>This option will attempt to auto-generate a textual executive summary for a result file to highlight prominent results / averages.</p>
@@ -333,12 +291,13 @@
 <h3>wins-and-losses<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
+<h3>workload-topology<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and print the test profiles contained within and their arrangement within different test suites for getting an idea as to the workload topology / make-up / logical groupings of the benchmarks being run.</p>
 <h1>Modules</h1>
-<h3>auto-load-module<em>  </em>
-</h3>
+<h3>auto-load-module</h3>
 <p>This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
-<h3>list-modules<em>  </em>
-</h3>
+<h3>list-modules</h3>
 <p>This option will list all of the available Phoronix Test Suite modules on this system.</p>
 <h3>module-info<em>  [Phoronix Test Suite Module]</em>
 </h3>
@@ -349,32 +308,70 @@
 <h3>test-module<em>  [Phoronix Test Suite Module]</em>
 </h3>
 <p>This option can be used for debugging a Phoronix Test Suite module.</p>
-<h3>unload-module<em>  </em>
-</h3>
+<h3>unload-module</h3>
 <p>This option can be used for easily removing a module from the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
+<h1>Debugging</h1>
+<h3>check-tests<em>  [Test]</em>
+</h3>
+<p>This option will perform a check on one or more test profiles to determine if there have been any vendor changes to the filename, filesize, URL location, MD5 and SHA256 checksums.</p>
+<h3>diagnostics</h3>
+<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
+<h3>dump-file-info</h3>
+<p>This option will dump the MD5 / SHA256 checksums and file size for a given file.</p>
+<h3>dump-openbenchmarking-indexes</h3>
+<p>This option is used for dumping the parsed output of OpenBenchmarking.org index files (metadata).</p>
+<h3>dump-phodevi-smart-cache</h3>
+<p>This option is used for displaying the contents of the Phodevi smart cache on the system.</p>
+<h3>dump-possible-options</h3>
+<p>This option will print all possible phoronix-test-suite sub-commands.</p>
+<h3>dump-unhandled-dependencies</h3>
+<p>This option will list missing entries in the external dependencies XML file for the operating system under test. This option is used if wanting to help find missing dependency XML data to fill in for contributing to upstream Phoronix Test Suite.</p>
+<h3>list-failed-installs</h3>
+<p>This option will list all test profiles that were attempted to be installed on the local system but failed to be installed. Where applicable, the possible error(s) from the test installation are also reported to assist in debugging.</p>
+<h3>list-test-errors</h3>
+<p>This sub-command is complementary to list-failed-installs. Rather than listing test installation errors, list-test-errors is used for displaying past test run-time errors. This option will list all test profiles that produced an error previously when running the test profile / benchmark. If a test profile later successfully ran the test with any given option(s) without errors, the error is then removed from the archive. This option is intended to be helpful in debugging test profile issues later on for having a persistent collection of run-time errors.</p>
+<h3>list-unsupported-tests</h3>
+<p>This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.</p>
 <h1>User Configuration</h1>
-<h3>enterprise-setup<em>  </em>
-</h3>
-<p>This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.</p>
-<h3>network-info<em>  </em>
-</h3>
+<h3>enterprise-setup</h3>
+<p>This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes. It also defaults the anonymous usage reporting to being disabled, along with other conservative settings.</p>
+<h3>network-info</h3>
 <p>This option will print information detected by the Phoronix Test Suite around the system's network configuration.</p>
-<h3>network-setup<em>  </em>
-</h3>
+<h3>network-setup</h3>
 <p>This option allows the user to configure how the Phoronix Test Suite connects to OpenBenchmarking.org and other web-services. Connecting through an HTTP proxy can be configured through this option.</p>
-<h3>user-config-reset<em>  </em>
-</h3>
+<h3>user-config-reset</h3>
 <p>This option can be used for resetting the Phoronix Test Suite user configuration file to its default state.</p>
-<h3>user-config-set<em>  </em>
-</h3>
+<h3>user-config-set</h3>
 <p>This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.</p>
-<h1>Phoromatic</h1>
-<h3>start-phoromatic-server<em>  </em>
+<h3>variables</h3>
+<p>This option will print all of the official environment variables supported by the Phoronix Test Suite for user configuration purposes. These environment variables are also listed as part of the official Phoronix Test Suite documentation while this command will also show the current value of the variables if currently set.</p>
+<h1>Result Export</h1>
+<h3>result-file-raw-to-csv<em>  [Test Result]</em>
 </h3>
+<p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
+<h3>result-file-to-csv<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output (Comma Separated Values) file. This file can then be loaded into a spreadsheet for easy viewing. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
+<h3>result-file-to-html<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the system hardware and software information along with the results to pure HTML file. No external files are required for CSS/JavaScript or other assets. The graphs are rendered as inline SVG. This is a pure HTML-only representation of the results for emailing or other easy analysis outside of the Phoronix Test Suite. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
+<h3>result-file-to-json<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation). The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
+<h3>result-file-to-pdf<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file. The outputted file appears in the user home directory or can otherwise be controlled via the OUTPUT_DIR and OUTPUT_FILE environment variables.</p>
+<h3>result-file-to-suite<em>  [Test Result]</em>
+</h3>
+<p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
+<h3>result-file-to-text<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.</p>
+<h1>Phoromatic</h1>
+<h3>start-phoromatic-server</h3>
 <p>Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.</p>
 <h1>Result Viewer</h1>
-<h3>start-result-viewer<em>  </em>
-</h3>
+<h3>start-result-viewer</h3>
 <p>Start the web-based result viewer.</p>
 </body>
 </html>

documentation/stubs/00_zmodule_options.html

@@ -4,76 +4,77 @@
 <title>Module Options</title>
 </head>
 <body>
-<p>The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the AutoLoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running <em>phoronix-test-suite list-modules.</em><hr></p>
+<p>The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the AutoLoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running <em>phoronix-test-suite list-modules.</em><hr></p>
 <h2>Backup Creation + Restore</h2>
-<p>This is a module for creating backups of the Phoronix Test Suite / Phoromatic and allows for restoring of created backups. The backup will be in ZIP or TAR format. If only a path is specified, the file-name will be auto-generated with a current time-stamp.</p>
-<p>phoronix-test-suite backup.create</p>
-<p>phoronix-test-suite backup.restore</p>
+<p>This is a module for creating backups of the Phoronix Test Suite / Phoromatic and allows for restoring of created backups. The backup will be in ZIP or TAR format. If only a path is specified, the filename will be auto-generated with a current timestamp.</p>
+<p><strong>phoronix-test-suite backup.create</strong></p>
+<p><strong>phoronix-test-suite backup.restore</strong></p>
+<h2>System Maintenance / Cleanup</h2>
+<p>This module can be used for system maintenance cleanup tasks around the Phoronix Test Suite. Currently implemented is support for automatically un-installing tests that have not been run in a period of time. When the module is loaded via the REMOVE_TESTS_OLDER_THAN environment variable, it will be automatically invoked at the end of running any benchmarks. Or this module can be manually invoked with the command: phoronix-test-suite cleanup.tests.</p>
+<p><strong>phoronix-test-suite cleanup.tests</strong></p>
+<p>This module utilizes the following environment variables: REMOVE_TESTS_OLDER_THAN.</p>
 <h2>Dummy Module</h2>
 <p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
-<p>phoronix-test-suite dummy_module.dummy-command</p>
+<p><strong>phoronix-test-suite dummy_module.dummy-command</strong></p>
 <p>This is a simple module intended for developers to just demonstrate some of the module functions.</p>
-<h2>Generate Perf FlameGraphs For Tests</h2>
-<p>Setting FLAME_GRAPH_PATH=&lt;path to flamegraph path&gt; will auto-load and enable this Phoronix Test Suite module. The module will generate a Linux perf FlameGraph for each test run during the benchmarking process. Details on FlameGraph @ https://github.com/brendangregg/FlameGraph</p>
-<p>This module utilizes the following environmental variables: FLAME_GRAPH_PATH.</p>
-<h2>Graphics Override</h2>
-<p>This module allows you to override some graphics rendering settings for the ATI and NVIDIA drivers while running the Phoronix Test Suite.</p>
-<p>This module utilizes the following environmental variables: FORCE_AA, FORCE_AF.</p>
+<h2>Flush Caches</h2>
+<p>Loading this module will ensure caches (page cache, swap, etc) automatically get flushed prior to running any test.</p>
+<p>This module utilizes the following environment variables: PTS_FLUSH_CACHES.</p>
 <h2>Result Exporter To HTML</h2>
 <p>This module allows basic exporting of results to HTML for saving either to a file locally (specified using the EXPORT_RESULTS_HTML_FILE_TO environment variable) or to a mail account (specified using the EXPORT_RESULTS_HTML_EMAIL_TO environment variable). EXPORT_RESULTS_HTML_EMAIL_TO supports multiple email addresses delimited by a comma.</p>
-<p>This module utilizes the following environmental variables: EXPORT_RESULTS_HTML_EMAIL_TO, EXPORT_RESULTS_HTML_FILE_TO.</p>
+<p>This module utilizes the following environment variables: EXPORT_RESULTS_HTML_EMAIL_TO, EXPORT_RESULTS_HTML_FILE_TO.</p>
 <h2>Linux Perf Framework Reporter</h2>
-<p>Setting LINUX_PERF=1 will auto-load and enable this Phoronix Test Suite module. The module also depends upon running a modern Linux kernel (supporting perf) and that the perf binary is available via standard system paths.</p>
-<p>This module utilizes the following environmental variables: LINUX_PERF.</p>
+<p>Setting LINUX_PERF=1 will auto-load and enable this Phoronix Test Suite module. The module also depends upon running a modern Linux kernel (supporting perf) and that the perf binary is available via standard system paths. Depending upon system permissions you may be limited to using perf as root or adjusting the /proc/sys/kernel/perf_event_paranoid setting.</p>
+<p>This module utilizes the following environment variables: LINUX_PERF.</p>
 <h2>Dynamic Result Viewer</h2>
 <p>This module pre-loads the HTTP dynamic result viewer for Phoronix Test Suite data.</p>
-<p>phoronix-test-suite load_dynamic_result_viewer.start</p>
+<p><strong>phoronix-test-suite load_dynamic_result_viewer.start</strong></p>
 <h2>Log Exporter</h2>
 <p>This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.</p>
-<p>This module utilizes the following environmental variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.</p>
+<p>This module utilizes the following environment variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.</p>
 <h2>MATISK</h2>
 <p>My Automated Test Infrastructure Setup Kit</p>
-<p>phoronix-test-suite matisk.run</p>
-<p>phoronix-test-suite matisk.template</p>
+<p><strong>phoronix-test-suite matisk.run</strong></p>
+<p><strong>phoronix-test-suite matisk.template</strong></p>
 <h2>OpenBenchmarking.org Auto Comparison</h2>
 <p>This module prints comparable OpenBenchmarking.org results in the command-line for reference purposes as tests are being run. OpenBenchmarking.org is automatically queried for results to show based on the test comparison hash and the system type (mobile, desktop, server, cloud, workstation, etc). No other system information or result data is transmitted.</p>
-<p>phoronix-test-suite ob_auto_compare.debug</p>
+<p><strong>phoronix-test-suite ob_auto_compare.debug</strong></p>
 <h2>Performance Per Dollar/Cost Calculator</h2>
 <p>Setting the COST_PERF_PER_DOLLAR= environment variable to whatever value of the system cost/component you are running a comparison on will yield extra graphs that calculate the performance-per-dollar based on the test being run. The COST_PERF_PER_DOLLAR environment variable is applied just to the current test run identifier. Set the COST_PERF_PER_UNIT= environment variable if wishing to use a metric besides dollar/cost. The COST_PERF_PER_HOUR value can be used rather than COST_PERF_PER_DOLLAR if wishing to calculate the e.g. cloud time or other compute time based on an hourly basis.</p>
-<p>phoronix-test-suite perf_per_dollar.add</p>
-<p>This module utilizes the following environmental variables: COST_PERF_PER_DOLLAR, COST_PERF_PER_UNIT, COST_PERF_PER_HOUR.</p>
+<p><strong>phoronix-test-suite perf_per_dollar.add</strong></p>
+<p>This module utilizes the following environment variables: COST_PERF_PER_DOLLAR, COST_PERF_PER_UNIT, COST_PERF_PER_HOUR.</p>
 <h2>Performance Tip Prompts</h2>
 <p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler and powersave governor checks.</p>
-<p>phoronix-test-suite perf_tips.show</p>
-<p>This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.</p>
+<p><strong>phoronix-test-suite perf_tips.show</strong></p>
+<p>This module utilizes the following environment variables: SUPPRESS_PERF_TIPS.</p>
 <p>This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler: it gives a warning if BFQ is being used with an incorrect configuration in a disk benchmark, and suggests the right configuration to use. For the moment it only works for existing, throughput-based tests. It will need to be extended for responsiveness and soft real-time-latency tests.</p>
 <h2>Benchmarking Compiler PGO Impact</h2>
 <p>This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.</p>
-<p>phoronix-test-suite pgo.benchmark</p>
+<p><strong>phoronix-test-suite pgo.benchmark</strong></p>
 <h2>Phoromatic Client</h2>
 <p>The Phoromatic client is used for connecting to a Phoromatic server (Phoromatic.com or a locally run server) to facilitate the automatic running of tests, generally across multiple test nodes in a routine manner. For more details visit http://www.phoromatic.com/. This module is intended to be used with Phoronix Test Suite 5.2+ clients and servers.</p>
-<p>phoronix-test-suite phoromatic.connect</p>
-<p>phoronix-test-suite phoromatic.explore</p>
-<p>phoronix-test-suite phoromatic.upload-result</p>
-<p>phoronix-test-suite phoromatic.set-root-admin-password</p>
-<p>phoronix-test-suite phoromatic.list-results</p>
-<p>phoronix-test-suite phoromatic.clone</p>
-<p>phoronix-test-suite phoromatic.export-results-for-account-schedules</p>
+<p><strong>phoronix-test-suite phoromatic.connect</strong></p>
+<p><strong>phoronix-test-suite phoromatic.explore</strong></p>
+<p><strong>phoronix-test-suite phoromatic.upload-result</strong></p>
+<p><strong>phoronix-test-suite phoromatic.set-root-admin-password</strong></p>
+<p><strong>phoronix-test-suite phoromatic.list-results</strong></p>
+<p><strong>phoronix-test-suite phoromatic.clone</strong></p>
+<p><strong>phoronix-test-suite phoromatic.export-results-for-account-schedules</strong></p>
 <p>The Phoromatic module contains the client support for interacting with Phoromatic and Phoromatic Tracker services.</p>
 <h2>Pushover.net</h2>
 <p>Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.</p>
-<p>This module utilizes the following environmental variables: PUSHOVER_NET_USER.</p>
+<p>This module utilizes the following environment variables: PUSHOVER_NET_USER.</p>
 <h2>Report Test Time Graphs</h2>
 <p>Setting the RUN_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the run-time needed for each test being executed. Setting the INSTALL_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the time required for the test installation. Setting the INSTALL_SIZES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the size of the installed test directory.</p>
-<p>This module utilizes the following environmental variables: RUN_TIMES_ARE_A_BENCHMARK, INSTALL_TIMES_ARE_A_BENCHMARK, INSTALL_SIZES_ARE_A_BENCHMARK.</p>
+<p>This module utilizes the following environment variables: RUN_TIMES_ARE_A_BENCHMARK, INSTALL_TIMES_ARE_A_BENCHMARK, INSTALL_SIZES_ARE_A_BENCHMARK.</p>
 <h2>Result Notifier</h2>
 <p>A notification module.</p>
 <h2>Custom Result Export Methods</h2>
 <p>A simple example module about interfacing with Phoronix Test Suite core for dumping result files in a custom format.</p>
-<p>phoronix-test-suite results_custom_export.nf</p>
+<p><strong>phoronix-test-suite results_custom_export.nf</strong></p>
 <h2>System Monitor</h2>
 <p>This module contains sensor monitoring support.</p>
-<p>This module utilizes the following environmental variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.</p>
+<p>This module utilizes the following environment variables: MONITOR, PERFORMANCE_PER_WATT, PERFORMANCE_PER_SENSOR, MONITOR_INTERVAL, MONITOR_PER_RUN.</p>
 <p>Monitoring these sensors is as easy as running your normal Phoronix Test Suite commands but at the beginning of the command add: MONITOR=&lt;selected sensors&gt;.  For example, this will monitor the CPU temperature and voltage during tests: 
 
     MONITOR=cpu.temp,cpu.voltage phoronix-test-suite benchmark universe
@@ -99,6 +100,14 @@ Supported Options:
   - cpu.freq.cpu5
   - cpu.freq.cpu6
   - cpu.freq.cpu7
+  - cpu.freq.cpu8
+  - cpu.freq.cpu9
+  - cpu.freq.cpu10
+  - cpu.freq.cpu11
+  - cpu.freq.cpu12
+  - cpu.freq.cpu13
+  - cpu.freq.cpu14
+  - cpu.freq.cpu15
   - cpu.peak-freq
   - cpu.power
   - cpu.temp
@@ -112,6 +121,14 @@ Supported Options:
   - cpu.usage.cpu5
   - cpu.usage.cpu6
   - cpu.usage.cpu7
+  - cpu.usage.cpu8
+  - cpu.usage.cpu9
+  - cpu.usage.cpu10
+  - cpu.usage.cpu11
+  - cpu.usage.cpu12
+  - cpu.usage.cpu13
+  - cpu.usage.cpu14
+  - cpu.usage.cpu15
   - cpu.usage.summary
   - cpu.voltage
   - all.gpu
@@ -141,8 +158,6 @@ Supported Options:
   - all.memory
   - memory.temp
   - memory.usage
-  - all.network
-  - network.usage
   - all.swap
   - swap.usage
   - all.sys
@@ -154,19 +169,25 @@ Supported Options:
   - all.sys.voltage
 
 NOTE: Use the "system-sensors" command to see what sensors are available for monitoring on the system.</p>
+<h2>Test Timeout</h2>
+<p>This module allows killing a test if it exceeds a defined threshold, such as if the test is hung, etc. TEST_TIMEOUT_AFTER= environment variable can be used for controlling the behavior. When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit on its own for whatever reason. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run.</p>
+<p>This module utilizes the following environment variables: TEST_TIMEOUT_AFTER.</p>
 <h2>Timed Screenshot</h2>
 <p>This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.</p>
-<p>This module utilizes the following environmental variables: SCREENSHOT_INTERVAL.</p>
+<p>This module utilizes the following environment variables: SCREENSHOT_INTERVAL.</p>
 <h2>Toggle Screensaver</h2>
 <p>This module toggles the system's screensaver while the Phoronix Test Suite is running. At this time, the GNOME and KDE screensavers are supported.</p>
-<p>This module utilizes the following environmental variables: HALT_SCREENSAVER.</p>
+<p>This module utilizes the following environment variables: HALT_SCREENSAVER.</p>
+<h2>Linux Turbostat Dumper</h2>
+<p>Setting TURBOSTAT_LOG=_DIR_ will auto-load and enable this Phoronix Test Suite module. The module will -- if turbostat is installed on the system and the user is root -- allow dumping of the TurboStat data to the specified directly on a per-test basis. This allows easily collecting of turbostat logs for each test being run. If the TURBOSTAT_LOG= value does not point to a directory, the TurboStat output will be appended to the test run log files.</p>
+<p>This module utilizes the following environment variables: TURBOSTAT_LOG.</p>
 <h2>Update Checker</h2>
 <p>This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date plus also handles message of the day information.</p>
 <h2>Utilize Wine On Linux Benchmarking</h2>
 <p>This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.</p>
-<p>This module utilizes the following environmental variables: USE_WINE.</p>
+<p>This module utilizes the following environment variables: USE_WINE.</p>
 <h2>System Event Watchdog</h2>
 <p>This module has support for stopping/interrupting tests if various system issues occur, like a temperature sensor exceeds a defined threshold.</p>
-<p>This module utilizes the following environmental variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD.</p>
+<p>This module utilizes the following environment variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD, WATCHDOG_MAXIMUM_WAIT.</p>
 </body>
 </html>

documentation/stubs/01_install.html

@@ -5,12 +5,13 @@
 <body>
 <h1>Setup Overview</h1>
 <p>The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
-<p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
-<p>Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.</p>
+<p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
+<p>Among the tested BSD distributions are FreeBSD, NetBSD, OpenBSD, and DragonflyBSD. Tested Solaris distributions include Oracle Solaris 11, OpenIndiana, and Illumos.</p>
 <h1>Dependencies</h1>
-<p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.</p>
+<p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 and PHP 8 are also fully supported by the Phoronix Test Suite.</p>
 
 <p>The <em>phoronix-test-suite.bat</em> Windows launcher for the Phoronix Test Suite will automatically download and setup PHP on the local system if PHP is not present already.</p>
+<p>The Phoronix Test Suite does not need to be installed system-wide but can simply be run from the extracted phoronix-test-suite folder as the local user.</p>
 <p>As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:</p>
 <ul>
 <li><strong>PHP DOM</strong> is needed for XML operations and must be installed for the Phoronix Test Suite to function.</li>
@@ -26,8 +27,8 @@
 <p>Without all of these extensions, some capabilities of the Phoronix Test Suite will not be available. Many of these packages are enabled by default and do not require any additional installation steps on most Linux distributions, otherwise they are often found in the package vendor's repository.</p>
 <h1>Notes</h1>
 <h2>General</h2>
-<p>You may need to modify the <em>php.ini</em> file on your system in order to support uploading results to OpenBenchmarking.org or logging into your OpenBenchmarking.org account. The <em>allow_url_fopen</em>, <em>file_uploads</em>, and <em>allow_url_include</em> options must be set to true in the PHP configuration. See additional information in this <a href="http://www.phoronix.com/forums/showpost.php?p=32484&amp;postcount=4">Phoronix Forums support thread</a>.</p>
-<p>Major updates to the Phoronix Test Suite are released on a quarterly basis. The latest stable and development versions of the Phoronix Test Suite are available at <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a>. The Git repository where the latest Phoronix Test Suite code is provided is hosted at <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">GitHub.com/phoronix-test-suite</a> and can be cloned/pulled from the <em>https://github.com/phoronix-test-suite/phoronix-test-suite.git</em> repository location. The latest upstream development code is housed in the master tree while older Phoronix Test Suite releases are available in their respective Git branches based upon the release's code-name.</p>
+<p>You may need to modify the <em>php.ini</em> file on your system in order to support uploading results to OpenBenchmarking.org or logging into your OpenBenchmarking.org account. The <em>allow_url_fopen</em>, <em>file_uploads</em>, and <em>allow_url_include</em> options must be set to true in the PHP configuration.</p>
+<p>Major updates to the Phoronix Test Suite are released on a quarterly basis. The latest stable and development versions of the Phoronix Test Suite are available at <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a>. The Git repository where the latest Phoronix Test Suite code is provided is hosted at <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">github.com/phoronix-test-suite</a> and can be cloned/pulled from the <em>https://github.com/phoronix-test-suite/phoronix-test-suite.git</em> repository location. The latest upstream development code is housed in the master tree while older Phoronix Test Suite releases are available in their respective Git branches based upon the release's code-name.</p>
 <p>If building the PHP package from upstream sources, it should just be a matter of running <em>./configure</em> with the <em>--enable-zip</em> flag (all other requirements should be apart of the stock PHP configuration) to satisfy the PHP needs of the Phoronix Test Suite.</p>
 <h2>File Structure</h2>
 <p>If manually changing the location of the <em>phoronix-test-suite</em> launcher file, the <em>PTS_USER_PATH</em> environment variable must be adjusted inside the file to reflect the absolute location that leads to the root directory of the <em>pts</em> and <em>pts-core</em> folders. The <em>pts-core</em> directory contains the &quot;engine&quot; of the Phoronix Test Suite.</p>
@@ -42,6 +43,6 @@
 <h2>BSD Installation</h2>
 <p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but should run well on the likes of FreeBSD and DragonFlyBSD.</p>
 <h2>MacOS Installation</h2>
-<p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
+<p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default on macOS 12 and older so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized. On newer versions of macOS not shipping with PHP by default, <a href="https://brew.sh/">Homebrew</a> can be used for installing PHP or building PHP from source. The Phoronix Test Suite also supports making use of Homebrew for acquiring necessary Phoronix Test Suite dependencies on macOS.</p>
 </body>
 </html>

documentation/stubs/01_windows.html

@@ -0,0 +1,25 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<Title>Phoronix Test Suite On Windows</Title>
+</head>
+<body>
+
+<h2>Introduction</h2>
+<p>Phoronix Test Suite 8.0 introduced rewritten Windows support that is at near feature parity to the program's long-standing support for Linux, macOS, and BSD operating systems.</p>
+<p>The Phoronix Test Suite Windows support currently targets <strong>Windows 10 x64</strong>, <strong>Windows 11 x64</strong> and <strong>Windows Server 2016 x64</strong> and later. Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.</p>
+<h2>Windows Setup / Dependencies</h2>
+<p>As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP. Running the <em>phoronix-test-suite.bat</em> file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under <em>C:\PHP</em> as the default location should PHP support not be found within your system's <em>Program Files</em> directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.</p>
+<p>The Phoronix Test Suite on Windows does depend upon <a href="https://www.cygwin.com/">Cygwin</a> for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of <em>C:\cygwin64</em>.</p>
+<p>Various test profiles may depend upon other &quot;external dependencies&quot; like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.</p>
+<h2>Running The Phoronix Test Suite On Windows</h2>
+<p>The Phoronix Test Suite can run from its local directory and does not need to be &quot;installed&quot; to a system path or any other &quot;setup&quot; process prior to execution. On a clean install of Windows or Windows Server, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:</p>
+<p>1. Download the Phoronix Test Suite from <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">Phoronix-Test-Suite on GitHub</a> (<a href="https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip">zip file</a>).</p>
+<p>2. From the Command Prompt or PowerShell, enter the <em>phoronix-test-suite</em> directory whether it be from Git or a zipped download.</p>
+<p>3. Run the <em>phoronix-test-suite.bat</em> file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.</p>
+<p>4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the <strong>phoronix-test-suite shell</strong> command.</p>
+<h2>Test Profiles On Windows</h2>
+<p>As of 2021, around 100 test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.</p>
+<h2>Getting Started</h2>
+<p>Besides <strong>phoronix-test-suite shell</strong> and <strong>phoronix-test-suite help</strong>, there is also <strong>phoronix-test-suite interactive</strong> for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running <strong>phoronix-test-suite benchmark stockfish</strong> or <strong>phoronix-test-suite benchmark crafty</strong> as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.</p>
+</body>
+</html>

documentation/stubs/02_external_dependencies.html

@@ -5,6 +5,6 @@
 </head>
 <body>
 <p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
-<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p>
+<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>Clear Linux</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Gentoo</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>NetBSD</li><li>OpenBSD</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE Enterprise Linux</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>macOS Brew</li><li>openSUSE</li><li>openSUSE Leap</li><li>openSUSE Tumbleweed</li></p>
 </body>
 </html>

documentation/stubs/40_configuration.html

@@ -4,14 +4,15 @@
 </head>
 <body>
 <h1>User Files &amp; Folders</h1>
+<p>These files/folders are the default locations when running as a non-root Phoronix Test Suite user. When running as root, the paths may appear in standard system paths like <em>/etc/phoronix-test-suite.xml</em>.</p>
 <p><strong>~/.phoronix-test-suite/user-config.xml</strong></p>
-<p>This is a per-user configuration file. Among the information stored here is the test options, locations for storing files, and batch mode options. This file is formatted in XML.</p>
+<p>This is a per-user configuration file. Among the information stored here is the test options, locations for storing files, and batch mode options. This file is formatted in XML. When run as root, this path is <em>/etc/phoronix-test-suite.xml</em>.</p>
 <p><strong>~/.phoronix-test-suite/graph-config.json</strong></p>
 <p>This is a per-user configuration file for storing graph attributes. The adjustable options include HTML hex color codes for different areas of the graph, dimensions of the graph, and font sizes. This file is formatted in JSON.</p>
 <p><strong>~/.phoronix-test-suite/download-cache/</strong></p>
 <p>This directory contains test packages that have been downloaded for test profiles. For more information on the download cache.</p>
 <p><strong>~/.phoronix-test-suite/installed-tests/</strong></p>
-<p>This directory is where tests are installed by default. Each test has its own directory within a sub-directory of <em>installed-tests/</em> based upon its OpenBenchmarking.org repository. In the test's folder is a <em>pts-install.xml</em> file used for managing the installation.</p>
+<p>This directory is where tests are installed by default. Each test has its own directory within a sub-directory of <em>installed-tests/</em> based upon its OpenBenchmarking.org repository. In the test's folder is a <em>pts-install.json</em> file used for managing the installation.</p>
 <p><strong>~/.phoronix-test-suite/test-results/</strong></p>
 <p>This directory is where tests results are saved by default. Each saved file has its own directory. In the saved directory is then a <em>composite.xml</em> file containing the useful results while in the <em>test-X.xml</em> files are back-ups of the results.</p>
 <p><strong>~/.phoronix-test-suite/modules-data/</strong></p>
@@ -20,72 +21,5 @@
 <p>This is the directory where test profiles are stored.</p>
 <p><strong>~/.phoronix-test-suite/test-suites/</strong></p>
 <p>This is the directory where test suites are stored.</p>
-<h1>Environment Variables</h1>
-<p><strong>TEST_RESULTS_NAME</strong></p>
-<p>When this variable is set, the value will be used as the name for automatically saving the test results.</p>
-<p><strong>TEST_RESULTS_IDENTIFIER</strong></p>
-<p>When this variable is set, the value will be used as the test identifier when automatically saving the test results.</p>
-<p><strong>TEST_RESULTS_DESCRIPTION</strong></p>
-<p>When this variable is set, the value will be used as the test results description when saving the test results.</p>
-<p><strong>PRESET_OPTIONS</strong></p>
-<p>For setting any test option(s) from an environment variable rather than being prompted for the options when running a test. Example: <em>PRESET_OPTIONS=&quot;stream.run-type=Add&quot; ./phoronix-test-suite benchmark stream</em>. Multiple options can be passed to this environment variable when delimited by a semicolon.</p>
-<p><strong>SKIP_TESTS</strong></p>
-<p>If there are any test(s) to exempt from the testing process, specify them in this variable. Multiple tests can be waived by delimiting each test identifier by a comma. A test hardware type (i.e. Graphics) can also be supplied for skipping a range of tests.</p>
-<p><strong>SKIP_TESTS_HAVING_ARGS</strong></p>
-<p>If any of the test(s) have an argument matching any strings contained in this environment variable, the test execution will be skipped. Multiple strings can be set when delimiting by a comma.</p>
-<p><strong>RUN_TESTS_IN_RANDOM_ORDER</strong></p>
-<p>Setting this environment variable will cause the tests to be run in a random order.</p>
-<p><strong>SKIP_TESTING_SUBSYSTEMS</strong></p>
-<p>If you are running a set of benchmarks (namely a result file) but wish to skip some of the tests that don't belong to a certain test type group, you can set the hardware types to test via this environment variable. E.g. setting <em>SKIP_TESTING_SUBSYSTEMS=Graphics</em> will skip all test profiles to run that are not of the graphics test group. Multiple types should be delimited by a comma.</p>
-<p><strong>PTS_MODULE_SETUP</strong></p>
-<p>This variable can be used to load Phoronix Test Suite module settings automatically when using the <em>module-setup</em> option. An example would be: <em>PTS_MODULE_SETUP=&quot;phoromatic.remote_host=http://www.phoromatic.com/; phoromatic.remote_account=123456; phoromatic.remote_verifier=ABCD&quot; phoronix-test-suite module-setup phoromatic</em>.</p>
-<p><strong>PTS_MODULES</strong></p>
-<p>If there are any Phoronix Test Suite modules to additionally load, they can be specified here. Multiple modules can be supplied by delimiting them with a comma. The more appropriate way of loading Phoronix Test Suite modules for longer periods of time is by using the <em>~/.phoronix-test-suite/user-config.xml</em> configuration.</p>
-<p><strong>NO_PHODEVI_CACHE</strong></p>
-<p>This is a debugging option to disable the Phodevi cache from being loaded of cached software/hardware information. Instead, all software/hardware will be polled from the Phodevi library without caching.</p>
-<p><strong>EXTERNAL_PHODEVI_CACHE</strong></p>
-<p>This option can be used for loading an external Phodevi cache. Such as loading the native hardware/software information from within a Windows Wine client from a native system host.</p>
-<p><strong>PTS_DISPLAY_MODE</strong></p>
-<p>If you wish to load a non-default display mode for a single instance, specify the mode in this variable.</p>
-<p><strong>TOTAL_LOOP_TIME</strong></p>
-<p>When running any test(s), if you would like the test(s) to continue running as a loop until a certain time has been reached, this variable can be used. The value should be the number of minutes to run the testing process before the loop is ended. The testing will finish whenever the currently active test has finished once the time has elapsed. The minimum value allowed is 10 minutes.</p>
-<p><strong>LIMIT_ELAPSED_TEST_TIME</strong></p>
-<p>If you want to ensure that the time for a given Phoronix Test Suite process doesn't elapse past a certain number of minutes, specify the number of minutes for this environment variable. When the amount of time spent testing exceeds that amount, the testing will end prematurely while still saving the tests that were completed in time.</p>
-<p><strong>TOTAL_LOOP_COUNT</strong></p>
-<p>When running any test(s), if you would like the test(s) to continue running for a number of times, this variable can be used. The value should be the number of times to loop the testing process before ending.</p>
-<p><strong>FORCE_TIMES_TO_RUN</strong></p>
-<p>If you wish to override the number of times to run each test -- rather than the Phoronix Test Suite using the number of times specified in each test profile -- this variable can be used.</p>
-<p><strong>FORCE_TIMES_TO_RUN_MULTIPLE</strong></p>
-<p>This option allows specifying a multiple for increasing the number of times a test will run based upon the original TimesToRun value specified in the test definition. This allows for increasing the expected times to run based on a multiple of that default rather than a static value.</p>
-<p><strong>FORCE_MIN_TIMES_TO_RUN</strong></p>
-<p>This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.</p>
-<p><strong>FORCE_MIN_TIMES_TO_RUN_CUTOFF</strong></p>
-<p>When used in conjunction with FORCE_MIN_TIMES_TO_RUN, the override value will only be applied to test profiles where its average run-time length (in minutes) is less than the value specified by FORCE_MIN_TIMES_TO_RUN_CUTOFF.</p>
-<p><strong>FORCE_MIN_DURATION_PER_TEST</strong></p>
-<p>This is similar  to FORCE_MIN_TIMES_TO_RUN but allows specifying a time (in minutes) that each test should be run for. Each test will loop at least until that amount of time has elapsed. This can be useful for short-running tests if wanting to ensure each test is run long enough to rule out system noise.</p>
-<p><strong>IGNORE_RUNS</strong></p>
-<p>IGNORE_RUNS can be passed a comma-separated list of runs to skip on each benchmark. For example, IGNORE_RUNS=1 would always drop the first run from being recorded.</p>
-<p><strong>NO_FILE_HASH_CHECKS</strong></p>
-<p>To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as <em>NO_MD5_CHECKS</em>, which is still honored but was changed to <em>NO_FILE_HASH_CHECKS</em> to reflect other kind of file hash sum checks.</p>
-<p><strong>NO_HTTPS</strong></p>
-<p>Set this environment variable to 1 if you don't wish to use HTTPS download links for test profiles (or the system/network lacks HTTPS support). When enabled, HTTPS links will then be done over HTTP.</p>
-<p><strong>PTS_DOWNLOAD_CACHE</strong></p>
-<p>While non-standard Phoronix Test Suite download caches can be specified within the <em>user-config.xml</em> file, an additional directory to look for potential Phoronix Test Suite download files can be specified by this variable.</p>
-<p><strong>GRAPH_HIGHLIGHT</strong></p>
-<p>If this variable is set with a valid test identifer from a result file whether you are using the <em>refresh-graphs</em> command or any other related to the rendering of test results on a bar graph, the specified test identifier's result will be rendered in a different color than the other test results. Multiple identifiers can be specified when delimited by a comma. Additionally, for each key it is possible to provide the actual color value, or an index in the color palette. Example: &quot;will_be_different,group1a=1,group1b=1,blue=#0000ff&quot;</p>
-<p><strong>TEST_EXEC_PREPEND</strong></p>
-<p>Set this variable to any command/environment variable that you may be passed prepended to the test execution string at runtime.</p>
-<p><strong>VIDEO_MEMORY</strong></p>
-<p>If Phodevi fails to detect the system's video memory capacity or is incorrectly detected, the video memory capacity (in MB) can be specified by this variable.</p>
-<p><strong>OVERRIDE_VIDEO_MODES</strong></p>
-<p>If Phodevi fails to detect all of the system's monitor video modes or a separate set of modes would be preferred, the modes can be specified in this variable. Example: <em>OVERRIDE_VIDEO_MODES=800x600,1024x768,1280x1024 phoronix-test-suite benchmark nexuiz</em>.</p>
-<p><strong>SKIP_TEST_SUPPORT_CHECKS</strong></p>
-<p>If this environment variable is set, it will not honor the support checks made by individual test profiles. I.e. test profiles that would normally be considered un-supported on a given platform are attempted to install and run regardless.</p>
-<p><strong>SKIP_ALL_TEST_SUPPORT_CHECKS</strong></p>
-<p>If this environment variable is set, all tests will be permitted on the client for execution. SKIP_ALL_TEST_SUPPORT_CHECKS is more liberal than SKIP_TEST_SUPPORT_CHECKS in letting disk tests run on RAM-based file-systems, attempt to run 2D/3D tests on VESA display drivers, and other special cases.</p>
-<p><strong>DEFAULT_VIDEO_MODE</strong></p>
-<p>If Phodevi fails to detect the system's monitor standard / default resolution, the mode can be specified in this variable. Example: <em>DEFAULT_VIDEO_MODE=1680x1050 phoronix-test-suite benchmark nexuiz</em>.</p>
-<p><strong>SKIP_EXTERNAL_DEPENDENCIES</strong></p>
-<p>To skip the Phoronix Test Suite external dependency checking/installation when installing a test, set this environment variable to <em>1</em>. If wishing to skip only certain external dependencies, set this variable's value to the name of the external dependencies (the generic dependency names used by the Phoronix Test Suite) to not install. Multiple dependencies to skip can be delimited by a comma.</p>
 </body>
 </html>

documentation/stubs/41_configuration_file.html

@@ -0,0 +1,129 @@
+<html>
+<head>
+<Title>Main Configuration File</Title>
+</head>
+<body>
+<p>The main configuration file is <em>user-config.xml</em> (located at <em>~/.phoronix-test-suite/user-config.xml</em> or <em>/etc/phoronix-test-suite.xml</em> when running as root/admin) contains the user configuration options for the Phoronix Test Suite. To edit any option, open the configuration file within your preferred text editor. Alternatively, you can use the <em>user-config-set</em> option with the Phoronix Test Suite to update settings. For example, to set the download cache with the Phoronix Test Suite, execute <em>phoronix-test-suite user-config-set CacheDirectory=~/cache-directory/</em>.</p>
+
+<h1>OpenBenchmarking Options</h1>
+<h3>AnonymousUsageReporting</h3>
+<p>If this option is set to <em>TRUE</em>, anonymous usage information and statistics, like the tests that are run and their length of run, will be reported to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> for analytical reasons. All submitted information is kept anonymous. For more information on the anonymous usage reporting, read the Phoronix Test Suite documentation.</p>
+<h3>IndexCacheTTL</h3>
+<p>The time to live for OpenBenchmarking.org index caches. This is an integer representing the number of days before an index cache should be automatically refreshed from OpenBenchmarking.org. The default value is <em>3</em> while setting the value to <em>0</em> will disable automatic refreshing of caches (caches can be manually updated at anytime using the respective command).</p>
+<h3>AlwaysUploadSystemLogs</h3>
+<p>If this option is set to <em>TRUE</em>, the system logs (i.e. dmesg, lspci, lsusb, Xorg.0.log) will always be uploaded to OpenBenchmarking.org when uploading your test results. Otherwise the user is prompted whether to attach the system logs with their results.</p>
+<h3>AllowResultUploadsToOpenBenchmarking</h3>
+<p>This option defines whether to allow/support result uploads to OpenBenchmarking.org. If set to <em>FALSE</em>, the user will not be prompted to allow uploading of test results to the public site.</p>
+
+<h1>General Options</h1>
+<h3>DefaultBrowser</h3>
+<p>The Phoronix Test Suite will automatically attempt to launch the system's default web browser when needed. This is done first by checking for x-www-browser and then xdg-open. If neither command is available, the Phoronix Test Suite will fallback to checking for Firefox, Epiphany, Mozilla, or the open command. If you wish to override the default browser that the Phoronix Test Suite selects, set this tag to the command name of the browser you wish to use. Leaving this tag empty will have the Phoronix Test Suite determine the default web browser.</p>
+<h3>UsePhodeviCache</h3>
+<p>If this option is set to <em>TRUE</em>, the Phoronix Test Suite will use the Phodevi smart cache (if available). The Phodevi smart cache will automatically cache relevant system hardware/software attributes that can be safely stored and will be used until the system's software/hardware has changed or the system rebooted. Enabling this option will speed up the detection of installed hardware and software through the Phoronix Test Suite. If this option is set to <em>FALSE</em>, Phodevi will not generate a smart cache. The default value is <em>TRUE</em>.</p>
+<h3>DefaultDisplayMode</h3>
+<p>This option affects how text is displayed on the command-line interface during the testing process. If this option is set to <em>DEFAULT</em>, the text interface will be the traditional Phoronix Test Suite output. If this option is set to <em>CONCISE</em>, the display mode is shorter and more concise. This is the default mode used during batch testing. The default value is <em>DEFAULT</em>.</p>
+<h3>PhoromaticServers</h3>
+<p>This option can be used to specify the IP address(es) and port(s) of any Phoromatic Servers you wish to connect to for obtaining cached data, connecting to Phoromatic as a client test system, etc. The Phoronix Test Suite will attempt zero-conf network discovery but if that fails you can add the <em>IP:port</em> (the Phoromatic Server's HTTP port) to this element for targeted probing by the Phoronix Test Suite. Multiple Phoromatic Servers can be added if delimited by a comma; e.g. <em>IP:port,IP:port, IP:port</em>.</p>
+
+<h1>Modules Options</h1>
+<h3>AutoLoadModules</h3>
+<p>This tag contains a string of the names of the Phoronix Test Suite modules to load by default when running the Phoronix Test Suite. Multiple modules can be listed when delimited by a comma. Modules that load via setting an environment variable can also be specified here (i.e. <em>FORCE_AA=8</em> as an option in this string to load the <em>graphics_override</em> module with the 8x forced anti-aliasing). The default value is <em>toggle_screensaver, update_checker</em>.</p>
+
+<h1>Installation Options</h1>
+<h3>RemoveDownloadFiles</h3>
+<p>If this option is set to <em>TRUE</em>, once a test has been installed the downloaded files will be removed. Enabling this option will conserve disk space and in nearly all circumstances will not result in any problems. However, if a test profile directly depends upon a file that was downloaded (as opposed to something extracted from a downloaded file during the installation process), enabling this option will cause issues. If this option is set to <em>FALSE</em>, the downloaded files will not be removed unless the test is uninstalled. The default value is <em>FALSE</em>.</p>
+<h3>SearchMediaForCache</h3>
+<p>If this option is set to <em>TRUE</em>, when installing a test it will automatically look for a Phoronix Test Suite download cache on removable media that is attached and mounted on the system. On the Linux operating system, the Phoronix Test Suite looks for devices mounted within the <em>/media/</em> or <em>/Volumes/</em> directories. If a download cache is found (a <em>download-cache/</em> folder within the drive's root directory) and a file it is looking for with matching MD5/SHA256 check-sum, the file will be automatically copied. Otherwise the standard download cache is checked. If this option is set to <em>FALSE</em>, removable media devices are not checked. The default value is <em>TRUE</em>.</p>
+<h3>SymLinkFilesFromCache</h3>
+<p>If this option is set to <em>TRUE</em>, during the test installation process when a file is found in a Phoronix Test Suite download cache, instead of copying the file just provide a symbolic link to the file. Enabling this option will conserve disk space and in nearly all circumstances will not result in any issues, permitting the download cache files are always mounted during testing and are not located on removable media. If this option is set to <em>FALSE</em>, the files will be copied from the download cache. The default value is <em>FALSE</em>.</p>
+<h3>PromptForDownloadMirror</h3>
+<p>If this option is set to <em>TRUE</em>, when downloading a test file the user will be prompted to select a mirror when multiple mirrors available. This option is targeted for those in remote regions or where their download speed may be greatly affected depending upon the server. If this option is set to <em>FALSE</em>, the Phoronix Test Suite will randomly pick a mirror. The default value is <em>FALSE</em>.</p>
+<h3>EnvironmentDirectory</h3>
+<p>This option sets the directory where tests will be installed to by the Phoronix Test Suite. The full path to the directory on the local file-system should be specified, though <em>~</em> is a valid character for denoting the user's home directory. The default value is <em>~/.phoronix-test-suite/installed-tests/</em>.</p>
+<h3>CacheDirectory</h3>
+<p>This option sets the directory for the main download cache. The download cache is checked when installing a test while attempting to locate a needed test file. If the file is found in the download cache, it will not be downloaded from there instead of an Internet mirror. When running <em>phoronix-test-suite make-download-cache</em>, files are automatically copied to this directory. The full path to the directory should be specified, though <em>~</em> is a valid character for denoting the user's home directory. Specifying an HTTP or FTP URL is valid. The default value is <em>~/.phoronix-test-suite/download-cache/</em>. Multiple cache directories can be specified as of Phoronix Test Suite 2.2 with each directory being delimited by a colon.</p>
+
+<h1>Testing Options</h1>
+<h3>SleepTimeBetweenTests</h3>
+<p>This option sets the time (in seconds) to sleep between running tests. The default value is <em>6</em>.</p>
+<h3>SaveSystemLogs</h3>
+<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will also save various system details and logs to a sub-directory of the result file's location. Among the logs that will be archived include the X.Org log, dmesg, and lspci outputs. These system details may also be saved if a test suite explicitly requests this information be saved. If this option is set to <em>FALSE</em>, the system details / logs will not be saved by default. The default value is <em>FALSE</em>. When running in batch mode or using a Phoronix Certification and Qualification Suite, the logs will be saved regardless of this user setting.</p>
+<h3>SaveInstallationLogs</h3>
+<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will archive the complete output generated by the test during its earlier installation process. The log(s) are then saved to a sub-directory of the result file's location. If this option is set to <em>FALSE</em>, the full test logs will not be saved. The default value is <em>FALSE</em>. When running in batch mode or using a Phoronix Certification and Qualification Suite, the logs will be saved regardless of this user setting.</p>
+<h3>RemoveTestInstallOnCompletion</h3>
+<p>If this option is set to <em>TRUE</em>, after a test has been completed, if that test profile is no longer present later in the test queue, the test installation will be removed from the disk. If the test is to be run at a later time, it will need to be re-installed. This is useful for embedded environments or Live CD/DVDs where the available memory (RAM) for storage may be limited.</p>
+<h3>SaveTestLogs</h3>
+<p>If this option is set to <em>TRUE</em>, when saving the results from a test it will archive the complete output of each test's run generated by the application under test itself. The default value is <em>FALSE</em>.</p>
+<h3>ResultsDirectory</h3>
+<p>This option sets the directory where test results will be saved by the Phoronix Test Suite. The full path to the directory on the local file-system should be specified, though <em>~</em> is a valid character for denoting the user's home directory. The default value is <em>~/.phoronix-test-suite/test-results/</em>.</p>
+<h3>AlwaysUploadResultsToOpenBenchmarking</h3>
+<p>This option defines whether test results should always be uploaded to OpenBenchmarking.org upon their completion. If this value is set to <em>FALSE</em>, the user will be prompted each time whether the results should be uploaded to OpenBenchmarking.org, unless running in batch mode where the value is pre-defined. The default value is <em>FALSE</em>.</p>
+<h3>AutoSortRunQueue</h3>
+<p>This option defines whether the Phoronix Test Suite should sort the queue of tests to run based upon their title and category of tests. If <em>FALSE</em>, the run queue won't be sorted and they will be run in the order they were added.</p>
+<h3>ShowPostRunStatistics</h3>
+<p>If <em>TRUE</em>, the Phoronix Test Suite will show various test run statistics / comparison data based upon the test results / result file being tested after the testing has finished.</p>
+
+<h1>TestResultValidation Options</h1>
+<h3>DynamicRunCount</h3>
+<p>If this option is set to <em>TRUE</em>, the Phoronix Test Suite will automatically increase the number of times a test is to be run if the standard deviation of the test results exceeds a predefined threshold. This option is set to <em>TRUE</em> by default and is designed to ensure the statistical signifiance of the test results. The run count will increase until the standard deviation falls below the threshold or when the total number of run counts exceeds twice the amount that is set to run by default from the given test profile. Under certain conditions the run count may also increase further.</p>
+<h3>LimitDynamicToTestLength</h3>
+<p>If <em>DynamicRunCount</em> is set to <em>TRUE</em>, this option sets a limit on the maximum length per trial run that a test can execute (in minutes) for the run count to be adjusted. This option is to prevent tests that take a very long amount of time to run from consuming too much time. By default this value is set to <em>20</em> minutes.</p>
+<h3>StandardDeviationThreshold</h3>
+<p>This option defines the overall standard deviation threshold (as a percent) for the Phoronix Test Suite to dynamically increase the run count of a test if this limit is exceeded. The default value is <em>3.50</em>.</p>
+<h3>ExportResultsTo</h3>
+<p>This option can specify a file (either the absolute path or relative if contained within <em>~/.phoronix-test-suite/</em> where a set of test results will be passed as the first argument as a string with each of the test results being delimited by a colon. If the executed script returns an exit status of <em>0</em> the results are considered valid, if the script returns an exit status of <em>1</em> the Phoronix Test Suite will request the test be run again.</p>
+
+<h1>ResultViewer Options</h1>
+<h3>WebPort</h3>
+<p>The default HTTP web port to use for launching the web-based result viewer. If the value is set to <em>RANDOM</em>, a random open web port will be used.</p>
+<h3>LimitAccessToLocalHost</h3>
+<p>If this value is set to <em>TRUE</em> (default), the web-based result viewer is only accessible by the local host. If the value is <em>FALSE</em>, anyone with access to the IP/port can access the result viewer.</p>
+<h3>AccessKey</h3>
+<p>An access key / password can be optionally supplied as a basic precaution particularly for web-accessible result viewers that aren't limited to the local host. Set the string value here of the desired key/password that the user will be prompted to enter when trying to access the result viewer.</p>
+<h3>AllowSavingResultChanges</h3>
+<p>This allows saving result file changes (notes, modifying result files, etc) of result files from the web-based result viewer. Besides needing to be set to <em>TRUE</em>, the result file directory must also be write-enabled.</p>
+<h3>AllowDeletingResults</h3>
+<p>This option is similar to <em>AllowSavingResultChanges</em> but controls the behavior of whether results can be permanently removed. Besides needing to be set to <em>TRUE</em>, the result file directory must also be write-enabled.</p>
+
+<h1>Batch Mode Options</h1>
+<p>The batch mode options are only used when using either the <em>batch-run</em> or <em>batch-benchmark</em> options with the Phoronix Test Suite. This mode is designed to fully automate the operation of the Phoronix Test Suite except for areas where the user would like to be prompted. To configure the batch mode options, it is recommended to run <em>phoronix-test-suite batch-setup</em> instead of modifying these values by hand.</p>
+<h3>SaveResults</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the test results will be automatically saved.</p>
+<h3>OpenBrowser</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the web-browser will automatically open when displaying test results. If this option is set to <em>FALSE</em>, the web-browser will not be opened.</p>
+<h3>UploadResults</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the test results will be automatically uploaded to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a>.</p>
+<h3>PromptForTestIdentifier</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test identifier. If this option is set to <em>FALSE</em>, a test identifier will be automatically generated.</p>
+<h3>PromptForTestDescription</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test description. If this option is set to <em>FALSE</em>, the default test description will be used.</p>
+<h3>PromptSaveName</h3>
+<p>If this option is set to <em>TRUE</em>, when running in batch mode the user will be prompted to enter a test name. If this option is set to <em>FALSE</em>, a test name will be automatically generated.</p>
+
+<h1>Networking Options</h1>
+<h3>NoInternetCommunication</h3>
+<p>If you wish to disable Internet communication within the Phoronix Test Suite by default, set this option to <em>TRUE</em>. The default value is <em>FALSE</em>. Setting this to <em>FALSE</em> will still allow Phoromatic to communicate with network servers such as for intranet-based download caches or a Phoromatic Server. Internet support is generally required for downloading test profiles from OpenBenchmarking.org, acquiring necessary test files from their respective sources, etc.</p>
+<h3>NoNetworkCommunication</h3>
+<p>If you wish to disable network support (including Internet access) entirely within the Phoronix Test Suite, set this option to <em>TRUE</em>. The default value is <em>FALSE</em>.</p>
+<h3>Timeout</h3>
+<p>This is the read timeout (in seconds) for network connections. The default value is <em>20</em>.</p>
+<h3>ProxyAddress</h3>
+<p>If you wish to use a HTTP proxy server to allow the Phoronix Test Suite to communicate with OpenBenchmarking.org and other web services, enter the IP address / server name of the proxy server in this tag. If the proxy address and port tags are left empty but the <em>http_proxy</em> environment variable is set, the Phoronix Test Suite will attempt to use that as the proxy information.</p>
+<h3>ProxyPort</h3>
+<p>If using a proxy server, enter the TCP port in this tag.</p>
+
+<h1>Server Options</h1>
+<h3>RemoteAccessPort</h3>
+<p>If you wish to allow remote access to the built-in web-based interface to the Phoronix Test Suite when running its built-in web server, set the port number for remote access here. Port 80 is the common HTTP port but the Phoronix Test Suite web-interface can be easily set to other port numbers. If you do not wish to allow remote access, use the default value of <em>FALSE</em> or <em>-1</em>. If the value is set to <em>RANDOM</em>, a random port number will be chosen.</p>
+<h3>Password</h3>
+<p>If you wish to require a password when entering the web-based interface to the Phoronix Test Suite -- either locally or remotely -- specify the password here using the password's SHA256 sum as the value.</p>
+<h3>WebSocketPort</h3>
+<p>The default port to use when running a WebSocket server. If no port is assigned or <em>RANDOM</em> is set, a random port will be chosen.</p>
+<h3>AdvertiseServiceZeroConf</h3>
+<p>If this option is set to <em>TRUE</em> when starting a Phoromatic Server instance, the software will attempt to broadcast its service using zeroconf networking (Avahi on Linux assuming <em>avahi-publish</em> is present).</p>
+<h3>AdvertiseServiceOpenBenchmarkRelay</h3>
+<p>If this option is set to <em>TRUE</em> when starting a Phoromatic Server instance, the software will broadcast the local IP/port of the server to a private OpenBenchmarking.org service so that if any other user on the local IP block from the same global IP address is in search of a Phoromatic Server, the IP address will be relayed. This is an alternative or complementary to the zero-conf/Avahi option above to help systems running the Phoronix Test Suite client on a LAN discover a Phoromatic Server for easy setup and/or download cache support for faster test setup/installation.</p>
+<h3>PhoromaticStorage</h3>
+<p>The location for the Phoromatic Server to store test results of connected systems, account information, etc. The default location is <em>~/.phoronix-test-suite/phoromatic/</em>.</p>
+</body>
+</html>

documentation/stubs/42_env_vars.html

@@ -0,0 +1,291 @@
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>Environment Variables</title>
+</head>
+<body>
+
+<h2>DONT_BALANCE_TESTS_FOR_SUBSYSTEMS</h2>
+<p><em>If this value is true, the Phoronix Test Suite stress-run manager will not attempt to distribute the selected test(s) among available hardware subsystems. For stress runs with tests covering multiple subsystems (e.g. CPU, GPU, RAM), the default behavior is try to ensure the tests to run concurrently are as balanced across the tested subsystems as possible.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: stress-run mode.
+</p>
+<h2>DONT_TRY_TO_ENSURE_TESTS_ARE_UNIQUE</h2>
+<p><em>When running in the stress-run mode, the default behavior will try to ensure when tests are running concurrently that as many unique tests as possible are being run. Setting this value to try will avoid that check and just attempt to truly randomize the tests being run concurrently without regard for trying to avoid duplicates.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: stress-run mode.
+</p>
+<h2>FORCE_ABSOLUTE_MIN_TIMES_TO_RUN</h2>
+<p><em>This option is similar to FORCE_MIN_TIMES_TO_RUN but is *absolute* in ensuring each test will run at least that number of times and not subject to change of any timed cut-offs or other factors.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>FORCE_MIN_DURATION_PER_TEST</h2>
+<p><em>This option can be used to specify the minimum number of times to run a given benchmark. Rather than relying on a static times-to-run count, the test will keep looping until the time has exceeded this number (in minutes).</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>FORCE_MIN_TIMES_TO_RUN</h2>
+<p><em>This option is similar to FORCE_TIMES_TO_RUN but is used for specifying the minimum possible number of times to run. Unlike FORCE_TIMES_TO_RUN, the run count can still exceed this value if the deviation between results or other factors are too high.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>FORCE_MIN_TIMES_TO_RUN_CUTOFF</h2>
+<p><em>Used in conjunction with the FORCE_MIN_TIMES_TO_RUN, the FORCE_MIN_TIMES_TO_RUN_CUTOFF can be used for specifyingg the amount of time (in minutes) before foregoing additional runs. This allows cutting off the testing early if this time threshold has been reached.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>FORCE_TIMES_TO_RUN</h2>
+<p><em>This option can be used to override the default number of times a given test is run. Rather than being specified by the individual test profile, FORCE_TIMES_TO_RUN allows for specifying the number of times to run each benchmark.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>FORCE_TIMES_TO_RUN_MULTIPLE</h2>
+<p><em>This option is similar to FORCE_TIMES_TO_RUN but the value is a multiple for how many times the test profile should be run respective to its default value. If the value is set to 2 and a given test profile by default is set to run 3 times, it would now instead be run a total of 6 times. This can be used for increasing the statistical significance of test results by using a multiple of the default rather than a static number as is the case with FORCE_TIMES_TO_RUN.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>GRAPH_HIGHLIGHT</h2>
+<p><em>If automatically generating an HTML or PDF result file from the command-line and wanting to highlight desired result identifier(s), GRAPH_HIGHLIGHT can be set to a comma delimited list of result identifiers to highlight / color differently than the rest.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: result output generation.
+</p>
+<h2>IGNORE_RUNS</h2>
+<p><em>This option can be used if wanting the Phoronix Test Suite to automatically toss out a specified result position when running a test profile multiple times. E.g. setting this value to 1 will toss out automatically the first run of each test profile or a value of 3 will toss out the third run of a given test. This overrides the IgnoreRuns option also available to individual test profiles. Multiple values for runs to ignore can be specified by delimiting with a comma.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>LIMIT_ELAPSED_TEST_TIME</h2>
+<p><em>This option can be used for limiting the amount of time the benchmarking process runs. The value specified is the number of minutes to allow for benchmarking. After a test finishes if that number of minutes has been exceeded, the testing process will abort early and not run any remaining tests.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>LINUX_PERF</h2>
+<p><em>This option allows providing additional complementary per-test graphs looking at various Linux perf subsystem metrics such as cache usage, instructions executed, and other metrics. This requires you to have Linux's perf user-space utility already installed and performance counter access.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: linux_perf.
+</p>
+<h2>LOG_CLI_OUTPUT</h2>
+<p><em>[EXPERIMENTAL] When this option is enabled, the Phoronix Test Suite standard output from the terminal will be logged to any relevant Phoronix Test Suite / Phoromatic log file. This is mainly useful for debugging purposes and if wishing to always archive the standard output as part of Phoronix Test Suite logs.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+</p>
+<h2>MONITOR</h2>
+<p><em>This option can be used for system sensor monitoring during test execution. The Phoronix Test Suite system_monitor module can monitor various exposed sensors and record them as part of the result file and present them as additional graphs / metrics in the result viewer. The exposed sensors varies by platform hardware/software. This functionality also requires PHP PCNTL support and thus is not available for some platforms (i.e. Windows).</em></p>
+<p>The value can be of type: enumeration (all, cpu.peak-freq, cpu.temp, cpu.power, cpu.usage, gpu.freq, gpu.power, gpu.temp, hdd.temp, memory.usage, swap.usage, sys.power, sys.temp)
+Multiple options can be supplied when delimited by a comma..
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: system_monitor.
+</p>
+<h2>NO_COLOR</h2>
+<p><em>This option when enabled will force-disable the CLI/TUI text coloring. By default the Phoronix Test Suite will attempt to use CLI/TUI text colors and bolding of text for supported terminals.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+</p>
+<h2>NO_COMPILER_MASK</h2>
+<p><em>By default the Phoronix Test Suite attempts to determine the intended system code compilers (namely C / C++ / Fortran) and to intercept the arguments being passed to them during test installation in order to record the prominent compiler flags being used. If this behavior causes problems for your system, NO_COMPILER_MASK can be enabled for debugging purposes to avoid this compiler intercepting/symlinking behavior.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test installation.
+</p>
+<h2>NO_DOWNLOAD_CACHE</h2>
+<p><em>Enable this option if the Phoronix Test Suite should not attempt to discover and use any local/remote Phoronix Test Suite download cache when installing tests and attempting to find those files locally or on a LAN resource.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test installation.
+</p>
+<h2>NO_EXTERNAL_DEPENDENCIES</h2>
+<p><em>Enabling this option will have the Phoronix Test Suite skip over attempting to detect and install any system/external dependencies needed to run desired test profiles. This should just be used in case of testing/evaluation purposes and may leave some tests unable to successfully build/install.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test installation.
+</p>
+<h2>NO_FILE_HASH_CHECKS</h2>
+<p><em>Enable this option if you want to skip the MD5 / SHA256 file hash checks after downloading files with known MD5/SHA256 hashsums for verification. This is namely useful for select debugging scenarios and other situations where a file may have been trivially changed / re-packaged and wishing to still install a test even though the hash no longer matches until the test profile has been updated.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test installation.
+</p>
+<h2>NO_HTTPS</h2>
+<p><em>Enable this option if wanting the Phoronix Test Suite when downloading resources to attempt to only use HTTP without any HTTPS connections. Note: some downloads may fail for servers that only support HTTPS.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+</p>
+<h2>NO_PHODEVI_CACHE</h2>
+<p><em>This option will disable use of the built-in Phodevi (Phoronix Device Interface) cache of system software/hardware details. When enabled, the information is not cached and will be re-computed on each query. This is mainly useful for debugging purposes.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+</p>
+<h2>OUTPUT_DIR</h2>
+<p><em>When exporting a result file, this option can be used for specifying the writable directory path where the exported result files should be saved to. The file-name will be automatically generated.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: result output generation.
+</p>
+<h2>OUTPUT_FILE</h2>
+<p><em>When exporting a result file, this option can be used for specifying the file name / file path and name of where to save the exported result file to rather than assuming the user home directory.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: result output generation.
+</p>
+<h2>PHODEVI_SANITIZE</h2>
+<p><em>This option can be used for stripping out part of a string on Phodevi (Phoronix Device Interface) hardware/software properties. Namely around the reported hardware/software information in result files if wanting any values / portions of strings stripped out from that information, such as for confidential hardware strings or other privacy concerns, PHODEVI_SANITIZE can be set. The value will be removed from read Phodevi hardware/software properties if set. Multiple strings to search for can be set by delimiting with a comma. If wanting to limit the sanitization to a particular property, the property value can be specified such as [property]=[value] to sanitisze like a value of "motherboard=ABCVENDOR" or CPU=ENGINEERING-SAMPLE to delete those strings rather than simply the string to remove that will look for matches in any property."</em></p>
+<p>The value can be of type: string.
+</p>
+<h2>PRESET_OPTIONS</h2>
+<p><em>PRESET_OPTIONS can be used for seeding the values of test profile run options from the environment (though the preferred approach for pre-configuring tests in an automated manner would be by constructing your own local test suite).  For setting any test option(s) from an environment variable rather than being prompted for the options when running a test. Example: "PRESET_OPTIONS='stream.run-type=Add' phoronix-test-suite benchmark stream".</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>PRESET_OPTIONS_VALUES</h2>
+<p><em>This option is similar to PRESET_OPTIONS and uses the same syntax but rather than seeding the selected run option it uses the value verbatim as for what is passed to the test profile run option.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>PTS_CONCURRENT_TEST_RUNS</h2>
+<p><em>This option is used in the stress run/benchmarking mode to indicate the number of tests to run concurrently as part of the stress run process.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: stress-run mode.
+</p>
+<h2>PTS_DISPLAY_MODE</h2>
+<p><em>If you wish to load a non-default display mode for a single instance, specify the mode in this variable as an alternative to adjusting the user configuration file.</em></p>
+<p>The value can be of type: enumeration (BASIC, BATCH, CONCISE, SHORT, DEFAULT).
+</p>
+<h2>PTS_DOWNLOAD_CACHE</h2>
+<p><em>PTS_DOWNLOAD_CACHE can be used for setting a path to a directory on the system containing a Phoronix Test Suite download cache if located outside one of the default locations.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation.
+</p>
+<h2>PTS_EXTRA_SYSTEM_LOGS_DIR</h2>
+<p><em>By default the Phoronix Test Suite collects common system logs (cpuinfo, lscpu, dmesg) during the benchmarking process when saving test results. If wanting to collect additional, arbitrary system log files specific to your operating environment or for other niche system information, this option can be set as a path to a directory containing such log files. Prior to running the Phoronix Test Suite simply set PTS_EXTRA_SYSTEM_LOGS_DIR to the directory where any files should be captured from following test completion.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>PTS_IGNORE_MODULES</h2>
+<p><em>Enabling this option can be used for temporarily disabling Phoronix Test Suite modules from being loaded on a given run. This is primarily for debugging purposes.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: modules.
+</p>
+<h2>PTS_MODULES</h2>
+<p><em>This option can be used for specifying a comma-separated list of Phoronix Test Suite modules to load at start-time, complementary to the modules specified in the user configuration file. PTS_MODULES is namely used for development purposes or wanting to temporarily enable a given module.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: modules.
+</p>
+<h2>PTS_MODULE_SETUP</h2>
+<p><em>This option can be used for seeding a module's settings when running the phoronix-test-suite module-setup command. An example would be: "PTS_MODULE_SETUP='phoromatic.remote_host=http://www.phoromatic.com/; phoromatic.remote_account=123456; phoromatic.remote_verifier=ABCD' phoronix-test-suite module-setup phoromatic".</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: modules.
+</p>
+<h2>PTS_SILENT_MODE</h2>
+<p><em>This option when enabled will yield slightly less verbose Phoronix Test Suite terminal output by silencing unnecessary messages / prompts.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+</p>
+<h2>PTS_TEST_INSTALL_ROOT_PATH</h2>
+<p><em>This option can be used for overriding where tests are installed to on the system. An absolute writable directory path can be the value if wanting to override the default (or user configuration file specified) test installation directory path.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation, test execution / benchmarking, stress-run mode.
+</p>
+<h2>REMOVE_TESTS_OLDER_THAN</h2>
+<p><em>This option with the cleanup module can be used for automatically un-installing/removing installed tests if they have not been run in a period of time. The value for REMOVE_TESTS_OLDER_THAN is the number of days the test can be installed without running until this module will clean-up/remove older tests.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: cleanup.
+</p>
+<h2>REMOVE_TESTS_ON_COMPLETION</h2>
+<p><em>When this option is set to true, installed test profiles will be automatically removed/uninstalled when they are no longer in the current test execution queue. This is used for saving disk space / resources by automatically removing installed tests after they have been executed. For more persistent behavior is the RemoveTestInstallOnCompletion option within the Phoronix Test Suite user configuration file.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>SKIP_EXTERNAL_DEPENDENCIES</h2>
+<p><em>Rather than NO_EXTERNAL_DEPENDENCIES to outright disable the Phoronix Test Suite external dependency handling, SKIP_EXTERNAL_DEPENDENCIES can be used with a value of a comma separated list of specific external dependencies to avoid. This is mostly useful for any external dependencies that may be out of date or fail to install on your platform.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation.
+</p>
+<h2>SKIP_TESTING_SUBSYSTEMS</h2>
+<p><em>This option is similar to SKIP_TESTS but allows for specifying hardware subsystems (e.g. Graphics) to skip from installing/running any test profiles beloning to that subsystem type. Multiple subsystems can be specified when delimited by a comma.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation, test execution / benchmarking.
+</p>
+<h2>SKIP_TESTS</h2>
+<p><em>SKIP_TESTS will skip the test installation and execution of any test identifiers specified by this option. Multiple test identifiers can be specified, delimited by a comma.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation, test execution / benchmarking.
+</p>
+<h2>SKIP_TESTS_HAVING_ARGS</h2>
+<p><em>SKIP_TESTS_HAVING_ARGS will skip the test installation and execution of any tests where the specified test arguments match the given string. E.g. if wanting to skip all Vulkan tests in a result file but run just the OpenGL tests or similar where wanting to limit the tests being run from within a result file. Multiple values can be specified when delimited by a comma.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test installation, test execution / benchmarking.
+</p>
+<h2>SKIP_TEST_SUPPORT_CHECKS</h2>
+<p><em>This debugging/validation option will have the Phoronix Test Suite skip any test support checks for a test profile (architecture compatibility, OS compatibility, etc) and just assume all tests are supported.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test installation, test execution / benchmarking.
+</p>
+<h2>SORT_BY</h2>
+<p><em>This option can be used for specifying the sort order for commands like auto-sort-result-file whether to sort by identifier name, test length, etc.</em></p>
+<p><strong>Default Value:</strong> identifier</p>
+<p>The value can be of type: enumeration (date, date-asc, date-desc, identifier).
+</p>
+<h2>TERMINAL_WIDTH</h2>
+<p><em>This option is used for overriding the detected default of the terminal width for the CLI/TUI interface.</em></p>
+<p>The value can be of type: positive integer.
+</p>
+<h2>TEST_EXECUTION_SORT</h2>
+<p><em>This option can be used for controlling the sort order that the test profiles / benchmarks are run in, whether sorted or not and in what manner.</em></p>
+<p>The value can be of type: enumeration (none, random, dependencies, test-estimated-time, test-estimated-time-desc, test, default).
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>TEST_EXEC_PREPEND</h2>
+<p><em>This option can be used if wanting to specify a binary (e.g. sudo, cgroup or other resource limiting binaries or performance counters) to be called as the binary pre-pended prior to running a test profile binary/script. This option is namely used for specialized use-cases.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>TEST_RESULTS_DESCRIPTION</h2>
+<p><em>This option can be used for specifying the result file description for saving that string and not be prompted for providing a description during the test execution process.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking, stress-run mode.
+</p>
+<h2>TEST_RESULTS_IDENTIFIER</h2>
+<p><em>This option can be used for specifying the result identifier for distinguishing this run within the saved result file.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking, stress-run mode.
+</p>
+<h2>TEST_RESULTS_NAME</h2>
+<p><em>This option can be used for specifying the result file name for saving the test/benchmark results automatically to the given name.</em></p>
+<p>The value can be of type: string.
+The variable is relevant for: test execution / benchmarking, stress-run mode.
+</p>
+<h2>TEST_TIMEOUT_AFTER</h2>
+<p><em>When this variable is set, the value will can be set to "auto" or a positive integer. The value indicates the number of minutes until a test run should be aborted, such as for a safeguard against hung/deadlocked processes or other issues. Setting this to a high number as a backup would be recommended for fending off possible hangs / stalls in the testing process if the test does not quit. If the value is "auto", it will quit if the time of a test run exceeds 3x the average time it normally takes the particular test to complete its run. In the future, auto might be enabled by default in a future PTS release. This functionality requires system PHP PCNTL support (i.e. no Windows support).</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: test_timeout.
+</p>
+<h2>TOTAL_LOOP_COUNT</h2>
+<p><em>This option is used to specify a multiple if wishing to run each test multiple times rather than just once per saved result file.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+</p>
+<h2>TOTAL_LOOP_TIME</h2>
+<p><em>This option is used to specify the amount of time (in minutes) to loop the testing during the Phoronix Test Suite stress run or normal benchmarking process.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: stress-run mode, test execution / benchmarking.
+</p>
+<h2>TURBOSTAT_LOG</h2>
+<p><em>This option allows attaching "turbostat" outputs to the end of archived benchmark/test log files if interested in the Linux TurboStat information. This assumes you have turbostat available on the Linux system(s) and have permissions (root) for running turbostat.</em></p>
+<p>The value can be of type: boolean (TRUE / FALSE).
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: turbostat.
+</p>
+<h2>WATCHDOG_MAXIMUM_WAIT</h2>
+<p><em>Used in conjunction with the WATCHDOG_SENSOR option, this is the maximum amount of time to potentially wait when the watchdog is triggered for surpassing the threshold value. The value is the maximum number of minutes to wait being above the threshold.</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: watchdog.
+</p>
+<h2>WATCHDOG_SENSOR</h2>
+<p><em>This option will enable the watchdog module that checks system sensor values pre/interim/post benchmark execution. If the selected sensor(s) exceed the static threshold level, testing will be paused before continuing to any additional tests so that the system can sleep. Ideally this will allow the system to return to a more suitable state before resuming testing after the sensor value is back below the threshold or after a pre-defined maximum time limit to spend sleeping. This module is mostly focused on pausing testing should system core temperatures become too elevated to allow time for heat dissipation.</em></p>
+<p>The value can be of type: enumeration (cpu.temp, gpu.temp, hdd.temp, sys.temp)
+Multiple options can be supplied when delimited by a comma..
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: watchdog.
+</p>
+<h2>WATCHDOG_SENSOR_THRESHOLD</h2>
+<p><em>Used in conjunction with the WATCHDOG_SENSOR option, the WATCHDOG_SENSOR_THRESHOLD specifies the threshold for the sensor reading when the testing should be paused (e.g. the Celsius cut-off temperature).</em></p>
+<p>The value can be of type: positive integer.
+The variable is relevant for: test execution / benchmarking.
+The variable depends upon functionality provided by the Phoronix Test Suite module: watchdog.
+</p>
+</body>
+</html>

documentation/stubs/50_general_information.html

@@ -5,19 +5,18 @@
 <body>
 <h1>Frequently Asked Questions</h1>
 <p><strong><em>Q:</em> May I use the Phoronix Test Suite when running benchmarks for my own publication or blog? Are there any publishing restrictions?</strong></p>
-<p><strong>A:</strong> Anyone is more than welcome to use the Phoronix Test Suite for their own publication or purpose. While the Phoronix Test Suite came out of our internal test tools for carrying out Linux hardware reviews at <a href="http://www.phoronix.com/">Phoronix.com</a>, we invite other hardware review web-sites, technology journals, and independent publications to use our software too. While not required, we would just kindly ask that you mention in your review/article that the <em>Phoronix Test Suite</em> was used for carrying out your testing, and ideally to link to <a href="http://www.phoronix-test-suite.com/">www.phoronix-test-suite.com</a> so that your readers will know where to obtain the software if they are interested in running the tests. You are also more than welcome to upload your results to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> so that others may compare their results against yours in an easy manner.</p>
+<p><strong>A:</strong> Anyone is more than welcome to use the Phoronix Test Suite for their own publication or purpose. While the Phoronix Test Suite came out of our internal test tools for carrying out Linux hardware reviews at <a href="https://www.phoronix.com/">Phoronix.com</a>, we invite other hardware review web-sites, technology journals, and independent publications to use our software too. While not required, we would just kindly ask that you mention in your review/article that the <em>Phoronix Test Suite</em> was used for carrying out your testing, and ideally to link to <a href="http://www.phoronix-test-suite.com/">www.phoronix-test-suite.com</a> so that your readers will know where to obtain the software if they are interested in running the tests. You are also more than welcome to upload your results to <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> so that others may compare their results against yours in an easy manner.</p>
 <p>We also try to make the Phoronix Test Suite easy-to-use by independent publications. For example, if you would like to watermark your web-site's URL into the graphs containing your test results, that can be easily modified in <em>~/.phoronix-test-suite/graph-config.json</em>. The colors and other graph settings are also stored in this XML file. If you are a publication and run into any issues with the Phoronix Test Suite or have a feature request, please let us know.</p>
-<p>A partial list of other publications using the Phoronix Test Suite can be found in <a href="http://www.phoronix.com/forums/showthread.php?t=12925">this forum thread</a>.</p>
 <p><strong><em>Q:</em> Why does the Phoronix Test Suite not use my distribution's package management system for acquiring all needed packages?</strong></p>
 <p><strong>A:</strong> The tests themselves are generally downloaded from source and built locally on the machine, rather than fetching any distribution-specific packages. This is done to ensure more comparable results across operating systems / releases, etc. The distribution packager could be applying a number of unknown patches to the software, building the software with unique build options, or making other changes to the software that could skew the results.</p>
 <p><strong><em>Q:</em> Besides being a developer, documentation writer, or having any other unique technical abilities, how else can I contribute to the Phoronix Test Suite?</strong></p>
-<p><strong>A:</strong> Independent code contributions are very welcome as well as creating your own test profiles and suites. We also appreciate any feedback, comments, or other ideas either by emailing us, posting on the forums, or sending a message to the mailing list. You can also see this (outdated) <a href="http://www.phoronix.com/forums/showthread.php?t=16180">forum thread</a>.</p>
+<p><strong>A:</strong> Independent code contributions are very welcome as well as creating your own test profiles and suites. We also appreciate any feedback, comments, or other ideas either by emailing us, posting on GitHub, or sending a message to the mailing list.</p>
 <p><strong><em>Q:</em> Do you offer technical support for the Phoronix Test Suite</strong></p>
-<p><strong>A:</strong> Paid, professional support is available and is done via <a href="http://commercial.phoronix-test-suite.com/">our commercial services</a>. We also offer Phoromatic licenses for use within a corporate intranet and other custom services. Free, community support is offered via our <a href="http://phoronix-test-suite.com/mailman/listinfo/trondheim-pts_phoronix-test-suite.com">mailing list</a>, IRC channel (<em>#phoronix</em> on <em>FreeNode.net</em>, and the <a href="http://www.phoronix.com/forums/">Phoronix Forums</a>.</p>
+<p><strong>A:</strong> Paid, professional support is available and is done via <a href="http://commercial.phoronix-test-suite.com/">our commercial services</a>. Free, community support is offered via <a href="https://github.com/phoronix-test-suite">GitHub</a>.</p>
 <p><strong><em>Q:</em> May I put the Phoronix Test Suite logo on my company's web-site or on my product packaging?</strong></p>
 <p><strong>A:</strong> <a href="http://www.phoronix-test-suite.com/?k=contact">Contact us</a> for licensing information and details regarding the Phoronix Certification &amp; Qualification Suite.</p>
 <p><strong><em>Q:</em> How often is the Phoronix Test Suite updated?</strong></p>
-<p><strong>A:</strong> We provide major updates on a quarterly basis with an occasional point release to fix outstanding bugs or address other issues. The latest work going into the Phoronix Test Suite is accessible via our Git repository at <a href="https://github.com/phoronix-test-suite/phoronix-test-suite/">GitHub.com/phoronix-test-suite</a>.</p>
+<p><strong>A:</strong> We provide major updates on a quarterly basis with an occasional point release to fix outstanding bugs or address other issues. The latest work going into the Phoronix Test Suite is accessible via our Git repository at <a href="https://github.com/phoronix-test-suite/phoronix-test-suite/">github.com/phoronix-test-suite</a>.</p>
 <h1>Tips &amp; Tricks</h1>
 <h2>General</h2>
 <p>- The desktop's screensaver will automatically be shutdown when a test is running and will be restored to its previous state upon the test's completion. This is supported for GNOME, KDE, and other XDG-supportive desktop environments.</p>

documentation/stubs/55_virtual_suites.html

@@ -6,22 +6,31 @@
 <body>
 <p>Virtual test suites are not like a traditional test suite defined by the XML suite specification. Virtual test suites are dynamically generated in real-time by the Phoronix Test Suite client based upon the specified test critera. Virtual test suites can automatically consist of all test profiles that are compatible with a particular operating system or test profiles that meet other critera. When running a virtual suite, the OpenBenchmarking.org repository of the test profiles to use for generating the dynamic suite must be prefixed. </p>
 <p>Virtual test suites can be installed and run just like a normal XML test suite and shares nearly all of the same capabilities. However, when running a virtual suite, the user will be prompted to input any user-configuration options for needed test profiles just as they would need to do if running the test individually. When running a virtual suite, the user also has the ability to select individual tests within the suite to run or to run all of the contained test profiles. Virtual test suites are also only supported for an OpenBenchmarking.org repository if there is no test profile or test suite of the same name in the repository. Below is a list of common virtual test suites for the main Phoronix Test Suite repository, but the dynamic list of available virtual test suites based upon the enabled repositories is available by running <em>phoronix-test-suite list-available-virtual-suites.</em><hr></p>
+<h3>64-bit Arm / AArch64 Tests In pts<em>  pts/aarch64</em>
+</h3>
+<p>This is a collection of test profiles where there have been successful benchmark results submitted to OpenBenchmarking.org from 64-bit Arm / AArch64 CPU architecture hardware, i.e. these tests are proven to be 64-bit Arm / AArch64 compatible though not necessarily all compatible test profiles for the given architecture - just those with submitted public results previously on OpenBenchmarking.org.</p>
 <h3>All Tests in pts<em>  pts/all</em>
 </h3>
 <p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>benchmark Tests<em>  pts/benchmark</em>
+<h3>Application Tests<em>  pts/application</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  pts/benchmark</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
-<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  pts/blas</em>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Tests<em>  pts/blas</em>
 </h3>
-<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
-<h3>Boost Development Libraries Tests<em>  pts/boost</em>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine)</p>
+<h3>C++ Boost Tests<em>  pts/boost</em>
 </h3>
-<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
-<h3>bsd Operating System Tests<em>  pts/bsd</em>
+<p>This is a collection of test profiles having an external dependency on C++ Boost</p>
+<h3>BSD Operating System Tests<em>  pts/bsd</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
-<h3>disk Subsystem Tests<em>  pts/disk</em>
+<h3>C/C++ Compiler Benchmark Workloads In pts<em>  pts/compiler</em>
+</h3>
+<p>This is a collection of test profiles often useful for C/C++ compiler benchmarks and where the test profiles will respect CFLAGS/CXXFLAGS environment variables.</p>
+<h3>Disk Subsystem Tests<em>  pts/disk</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
 <h3>Everything in pts<em>  pts/everything</em>
@@ -30,34 +39,37 @@
 <h3>Fortran Tests<em>  pts/fortran</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Fortran</p>
-<h3>game Tests<em>  pts/game</em>
+<h3>Game Tests<em>  pts/game</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
 <h3>Go Language Tests<em>  pts/golang</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Go Language</p>
-<h3>graphics Subsystem Tests<em>  pts/graphics</em>
+<h3>Graphics Subsystem Tests<em>  pts/graphics</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
 <h3>Installed Tests<em>  pts/installed</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>java Tests<em>  pts/java</em>
+<h3>Java Tests<em>  pts/java</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing java.</p>
 <h3>LAPACK (Linear Algebra Pack) Tests<em>  pts/lapack</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on LAPACK (Linear Algebra Pack)</p>
-<h3>linux Operating System Tests<em>  pts/linux</em>
+<h3>Linux Operating System Tests<em>  pts/linux</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
-<h3>macosx Operating System Tests<em>  pts/macosx</em>
+<h3>Macosx Operating System Tests<em>  pts/macosx</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
-<h3>memory Subsystem Tests<em>  pts/memory</em>
+<h3>Memory Subsystem Tests<em>  pts/memory</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the memory sub-system.</p>
-<h3>network Subsystem Tests<em>  pts/network</em>
+<h3>Multi-Core/Multi-Threaded Workloads In pts<em>  pts/multicore</em>
+</h3>
+<p>This is a collection of test profiles that have been detected to be CPU multi-threaded capable.</p>
+<h3>Network Subsystem Tests<em>  pts/network</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
 <h3>Node.js + NPM Tests<em>  pts/node-npm</em>
@@ -66,194 +78,212 @@
 <h3>OpenCV Tests<em>  pts/opencv</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on OpenCV</p>
-<h3>openmpi Tests<em>  pts/openmpi</em>
+<h3>Openmpi Tests<em>  pts/openmpi</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.</p>
-<h3>processor Subsystem Tests<em>  pts/processor</em>
+<h3>OS Subsystem Tests<em>  pts/os</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the os sub-system.</p>
+<h3>Processor Subsystem Tests<em>  pts/processor</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
-<h3>python Tests<em>  pts/python</em>
+<h3>Python Tests<em>  pts/python</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.</p>
+<h3>RISC-V Tests In pts<em>  pts/riscv</em>
+</h3>
+<p>This is a collection of test profiles where there have been successful benchmark results submitted to OpenBenchmarking.org from RISC-V CPU architecture hardware, i.e. these tests are proven to be RISC-V compatible though not necessarily all compatible test profiles for the given architecture - just those with submitted public results previously on OpenBenchmarking.org.</p>
 <h3>Ruby Tests<em>  pts/ruby</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Ruby</p>
 <h3>Rust Tests<em>  pts/rust</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Rust</p>
-<h3>scientific Tests<em>  pts/scientific</em>
+<h3>Scientific Tests<em>  pts/scientific</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
-<h3>simulator Tests<em>  pts/simulator</em>
+<h3>Simulator Tests<em>  pts/simulator</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a simulator software test.</p>
-<h3>solaris Operating System Tests<em>  pts/solaris</em>
+<h3>Single-Threaded Workloads In pts<em>  pts/single-threaded</em>
+</h3>
+<p>This is a collection of test profiles that have been detected to be single-threaded or only very poorly CPU threaded.</p>
+<h3>Solaris Operating System Tests<em>  pts/solaris</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
-<h3>system Subsystem Tests<em>  pts/system</em>
+<h3>Steam Tests<em>  pts/steam</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on Steam</p>
+<h3>System Subsystem Tests<em>  pts/system</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
-<h3>utility Tests<em>  pts/utility</em>
+<h3>Utility Tests<em>  pts/utility</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
-<h3>windows Operating System Tests<em>  pts/windows</em>
+<h3>Windows Operating System Tests<em>  pts/windows</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
-<h3>smp Tests<em>  pts/smp</em>
+<h3>Smp Tests<em>  pts/smp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
-<h3>opencl Tests<em>  pts/opencl</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
-<h3>cuda Tests<em>  pts/cuda</em>
+<h3>Cuda Tests<em>  pts/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
-<h3>mpi Tests<em>  pts/mpi</em>
+<h3>Mpi Tests<em>  pts/mpi</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing mpi.</p>
-<h3>openmp Tests<em>  pts/openmp</em>
+<h3>Openmp Tests<em>  pts/openmp</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.</p>
-<h3>cloud Tests<em>  pts/cloud</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cloud.</p>
-<h3>docker Tests<em>  pts/docker</em>
+<h3>Docker Tests<em>  pts/docker</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing docker.</p>
-<h3>go Tests<em>  pts/go</em>
+<h3>Ai Tests<em>  pts/ai</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing ai.</p>
+<h3>Opencl Tests<em>  pts/opencl</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
+<h3>Cloud Tests<em>  pts/cloud</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cloud.</p>
+<h3>Go Tests<em>  pts/go</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.</p>
-<h3>vdpau Tests<em>  pts/vdpau</em>
+<h3>Optix Tests<em>  pts/optix</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing optix.</p>
+<h3>Vdpau Tests<em>  pts/vdpau</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing vdpau.</p>
-<h3>video Tests<em>  pts/video</em>
+<h3>Video Tests<em>  pts/video</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing video.</p>
-<h3>responsiveness Tests<em>  pts/responsiveness</em>
+<h3>Responsiveness Tests<em>  pts/responsiveness</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.</p>
+<h3>64-bit Arm / AArch64 Tests In system<em>  system/aarch64</em>
+</h3>
+<p>This is a collection of test profiles where there have been successful benchmark results submitted to OpenBenchmarking.org from 64-bit Arm / AArch64 CPU architecture hardware, i.e. these tests are proven to be 64-bit Arm / AArch64 compatible though not necessarily all compatible test profiles for the given architecture - just those with submitted public results previously on OpenBenchmarking.org.</p>
 <h3>All Tests in system<em>  system/all</em>
 </h3>
 <p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>benchmark Tests<em>  system/benchmark</em>
+<h3>Application Tests<em>  system/application</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>Benchmark Tests<em>  system/benchmark</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
-<h3>BLAS (Basic Linear Algebra Sub-Routine) Library Tests<em>  system/blas</em>
+<h3>BLAS (Basic Linear Algebra Sub-Routine) Tests<em>  system/blas</em>
 </h3>
-<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine) Library</p>
-<h3>Boost Development Libraries Tests<em>  system/boost</em>
+<p>This is a collection of test profiles having an external dependency on BLAS (Basic Linear Algebra Sub-Routine)</p>
+<h3>C++ Boost Tests<em>  system/boost</em>
 </h3>
-<p>This is a collection of test profiles having an external dependency on Boost Development Libraries</p>
-<h3>bsd Operating System Tests<em>  system/bsd</em>
+<p>This is a collection of test profiles having an external dependency on C++ Boost</p>
+<h3>BSD Operating System Tests<em>  system/bsd</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
-<h3>disk Subsystem Tests<em>  system/disk</em>
+<h3>Disk Subsystem Tests<em>  system/disk</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the disk sub-system.</p>
 <h3>Everything in system<em>  system/everything</em>
 </h3>
 <p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
-<h3>game Tests<em>  system/game</em>
+<h3>Game Tests<em>  system/game</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a game software test.</p>
-<h3>graphics Subsystem Tests<em>  system/graphics</em>
+<h3>Graphics Subsystem Tests<em>  system/graphics</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the graphics sub-system.</p>
-<h3>Installed Tests<em>  system/installed</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.</p>
-<h3>linux Operating System Tests<em>  system/linux</em>
+<h3>Linux Operating System Tests<em>  system/linux</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
-<h3>macosx Operating System Tests<em>  system/macosx</em>
+<h3>Macosx Operating System Tests<em>  system/macosx</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
-<h3>network Subsystem Tests<em>  system/network</em>
+<h3>Network Subsystem Tests<em>  system/network</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the network sub-system.</p>
-<h3>processor Subsystem Tests<em>  system/processor</em>
+<h3>OpenCV Tests<em>  system/opencv</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on OpenCV</p>
+<h3>OpenMPI Tests<em>  system/openmpi</em>
+</h3>
+<p>This is a collection of test profiles having an external dependency on OpenMPI</p>
+<h3>Processor Subsystem Tests<em>  system/processor</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
 <h3>Python Tests<em>  system/python</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Python</p>
-<h3>solaris Operating System Tests<em>  system/solaris</em>
+<h3>RISC-V Tests In system<em>  system/riscv</em>
+</h3>
+<p>This is a collection of test profiles where there have been successful benchmark results submitted to OpenBenchmarking.org from RISC-V CPU architecture hardware, i.e. these tests are proven to be RISC-V compatible though not necessarily all compatible test profiles for the given architecture - just those with submitted public results previously on OpenBenchmarking.org.</p>
+<h3>Scientific Tests<em>  system/scientific</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a scientific software test.</p>
+<h3>Solaris Operating System Tests<em>  system/solaris</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
-<h3>system Subsystem Tests<em>  system/system</em>
+<h3>System Subsystem Tests<em>  system/system</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
-<h3>utility Tests<em>  system/utility</em>
+<h3>Utility Tests<em>  system/utility</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
-<h3>windows Operating System Tests<em>  system/windows</em>
+<h3>Windows Operating System Tests<em>  system/windows</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
-<h3>opencl Tests<em>  system/opencl</em>
+<h3>Opencl Tests<em>  system/opencl</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.</p>
-<h3>cuda Tests<em>  system/cuda</em>
+<h3>Cuda Tests<em>  system/cuda</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.</p>
+<h3>Smp Tests<em>  system/smp</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
+<h3>Openmp Tests<em>  system/openmp</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.</p>
+<h3>Mpi Tests<em>  system/mpi</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing mpi.</p>
+<h3>64-bit Arm / AArch64 Tests In git<em>  git/aarch64</em>
+</h3>
+<p>This is a collection of test profiles where there have been successful benchmark results submitted to OpenBenchmarking.org from 64-bit Arm / AArch64 CPU architecture hardware, i.e. these tests are proven to be 64-bit Arm / AArch64 compatible though not necessarily all compatible test profiles for the given architecture - just those with submitted public results previously on OpenBenchmarking.org.</p>
 <h3>All Tests in git<em>  git/all</em>
 </h3>
 <p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>bsd Operating System Tests<em>  git/bsd</em>
+<h3>Application Tests<em>  git/application</em>
+</h3>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a application software test.</p>
+<h3>BSD Operating System Tests<em>  git/bsd</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
 <h3>Everything in git<em>  git/everything</em>
 </h3>
 <p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
-<h3>linux Operating System Tests<em>  git/linux</em>
+<h3>Linux Operating System Tests<em>  git/linux</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
-<h3>macosx Operating System Tests<em>  git/macosx</em>
+<h3>Macosx Operating System Tests<em>  git/macosx</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
-<h3>processor Subsystem Tests<em>  git/processor</em>
+<h3>Processor Subsystem Tests<em>  git/processor</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
 <h3>Rust Tests<em>  git/rust</em>
 </h3>
 <p>This is a collection of test profiles having an external dependency on Rust</p>
-<h3>utility Tests<em>  git/utility</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
-<h3>smp Tests<em>  git/smp</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
-<h3>All Tests in windows<em>  windows/all</em>
-</h3>
-<p>This is a collection of all supported test profiles found within the specified OpenBenchmarking.org repository.</p>
-<h3>benchmark Tests<em>  windows/benchmark</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a benchmark software test.</p>
-<h3>bsd Operating System Tests<em>  windows/bsd</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the bsd Operating System.</p>
-<h3>Everything in windows<em>  windows/everything</em>
-</h3>
-<p>This is a collection of all test profiles found within the specified OpenBenchmarking.org repository, including unsupported tests, etc.</p>
-<h3>linux Operating System Tests<em>  windows/linux</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the linux Operating System.</p>
-<h3>macosx Operating System Tests<em>  windows/macosx</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the macosx Operating System.</p>
-<h3>processor Subsystem Tests<em>  windows/processor</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the processor sub-system.</p>
-<h3>solaris Operating System Tests<em>  windows/solaris</em>
-</h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the solaris Operating System.</p>
-<h3>system Subsystem Tests<em>  windows/system</em>
+<h3>System Subsystem Tests<em>  git/system</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the system sub-system.</p>
-<h3>utility Tests<em>  windows/utility</em>
+<h3>Utility Tests<em>  git/utility</em>
 </h3>
 <p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a utility software test.</p>
-<h3>windows Operating System Tests<em>  windows/windows</em>
+<h3>Smp Tests<em>  git/smp</em>
 </h3>
-<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the windows Operating System.</p>
+<p>This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.</p>
 </body>
 </html>

documentation/stubs/59_component_testing.html

@@ -5,10 +5,9 @@
 <body>
 <h1>Compiler Testing &amp; Masking</h1>
 <p>A majority of the test profiles provided by OpenBenchmarking.org to the Phoronix Test Suite are source-based tests. Relying upon the upstream source-code for each program under test allows for the tests to be easily brought to new platforms and architectures, avoids any out-of-tree / non-default packaging differences by different distributions and operating systems, and to allow the Phoronix Test Suite user to easily test new compilers and/or compiler options. For the source-based tests, the Phoronix Test Suite relies upon a compiler (e.g. GCC, LLVM/Clang, Sun Studio, Open64, et al) being present on the system under test. The Phoronix Test Suite does respect <em>CC/CXX</em> environment variables and test profiles are expected to honor <em>CFLAGS/CXXFLAGS</em> and other compiler settings.</p>
-<p>As of Phoronix Test Suite 3.8, a compiler masking method is utilized for logging compiler options and other settings with each test profile installation. Prior to installing an open-source test, the Phoronix Test Suite determines the intended compiler to be used based upon the pre-set environment variables or the pre-set compiler(s) within the <em>PATH</em> environment variable. The Phoronix Test Suite then masks the compiler to ensure that any options/flags submitted to the compiler are first passed through pts-core so that they can be logged for later use, then is linked to the original, intended compiler. Additionally, other compiler binary names of the same type are blacklisted to prevent their un-intended use (i.e. if a test profile has hard-coded <em>gcc</em> in its build script, but <em>clang</em> is set as the compiler via <em>CC</em>, a sym-link will automatically be created from <em>gcc</em> to the masked <em>clang</em> for the duration of its test installation).</p>
+<p>The Phoronix Test Suite utilizes a compiler masking method for logging compiler options and other settings with each test profile installation. Prior to installing an open-source test, the Phoronix Test Suite determines the intended compiler to be used based upon the pre-set environment variables or the pre-set compiler(s) within the <em>PATH</em> environment variable. The Phoronix Test Suite then masks the compiler to ensure that any options/flags submitted to the compiler are first passed through pts-core so that they can be logged for later use, then is linked to the original, intended compiler. Additionally, other compiler binary names of the same type are blacklisted to prevent their un-intended use (i.e. if a test profile has hard-coded <em>gcc</em> in its build script, but <em>clang</em> is set as the compiler via <em>CC</em>, a sym-link will automatically be created from <em>gcc</em> to the masked <em>clang</em> for the duration of its test installation).</p>
 <p>The logged compiler data is then used by the Phoronix Test Suite following the test execution process for automated result analysis. If there is a detected change in compiler settings, the differences are automatically reported to the test result graphs. Additionally, key compiler information (e.g. the compiler optimization level and key libraries that were linked at compile-time) is reported as a footnote on result graphs. The Phoronix Test Suite handles all of this in a fully automated manner; test profiles require no changes to take advantage of these compiler-reporting capabilities.</p>
 <p>Separately, the Phoronix Test Suite attempts to automatically log the system compiler's build configuration (i.e. GCC's <em>gcc -v</em> &quot;Configured with&quot;) output. If the Phoronix Test Suite detects there is a compiler build configuration change between test runs in a result file, it will report each compiler's build configuration to the system information table within the results viewer. Reported to the table is a reduced view of the build configuration options, with less relevant items being stripped away from this view to reduce verbosity. Upon clicking the text, the raw compiler information output can be viewed in full.</p>
-
 <h1>Disk / File-System Testing</h1>
 <p>By default tests are installed to <em>~/.phoronix-test-suite/installed-tests/</em>. However, the location can be updated from <em>~/.phoronix-test-suite/user-config.xml</em> or dynamically via the <em>PTS_TEST_INSTALL_ROOT_PATH</em> environment variable.</p>
 <p>When any disk tests are executed, the Phoronix Test Suite will attempt to log the mount options and scheduler of the disk/partition being used for testing. This information is subsequently displayed within the system information table. If the scheduler / mount options are maintained the same throughout all test runs, only a single line is displayed otherwise the options for each test run will be automatically displayed. The file-system in use is always captured and shown in the system information table.</p>

documentation/stubs/60_phoromatic.html

@@ -1,9 +1,88 @@
-<html>
+<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
-<Title>Phoromatic</Title>
+<Title>Phoronix Test Suite Phoromatic</Title>
 </head>
 <body>
-<h1>Overview</h1>
-<p>The Phoronix Test Suite now features its own built-in Phoromatic server and client implementations. See the <em>phoromatic.pdf</em> / <em>phoromatic.html</em> documentation for more information on this advanced capability.</p>
+<h1>Phoromatic Server</h1>
+<h2>Introduction</h2>
+<p>Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name <em>Phoro-matic</em>) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.</p>
+<h2>Features</h2>
+<p>Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:</p>
+<h3>Automated Scheduling</h3>
+<p>Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.</p>
+<h3>Extensible</h3>
+<p>Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).</p>
+<h3>Remote Testing</h3>
+<p>Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.</p>
+<h3>Multi-System Support</h3>
+<p>A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.</p>
+<h3>Turn-Key Deployment</h3>
+<p>No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.</p>
+<h3>Result Management</h3>
+<p>Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.</p>
+<h3>Decentralized / Offline Testing</h3>
+<p>Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.</p>
+<h3>Fully Open-Source</h3>
+<p>Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.</p>
+<h2>Phoromatic Server Setup</h2>
+<p>Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the <strong>phoronix-test-suite</strong>. Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within <em>~/.phoronix-test-suite/user-config.xml</em>. The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.</p>
+<p>On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as <u>avahi-tools</u>), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.</p>
+<p>Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the <em>~/.phoronix-test-suite/user-config.xml</em> file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.</p>
+<p>The Phoromatic Server utilizes PHP's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.</p>
+<p>Systemd service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP is recommended for offering the best support.</p>
+<h2>Example Deployments</h2>
+<h3>Use Case A: Unrestricted Internet Access, Local Result Storage</h3>
+<p>Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, simply run:</p>
+<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
+<p>Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.</p>
+<p>From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running <strong>phoronix-test-suite phoromatic.connect</strong> with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.</p>
+<h3>Use Case B: No Internet Available To Client Systems</h3>
+<p>It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, a few commands from the system while having an Internet connection will be able to cache the needed data:</p>
+<p><strong>$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg</strong></p>
+<p>This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to <em>~/.phoronix-test-suite/download-cache/</em> or <em>/usr/share/phoronix-test-suite/download-cache/</em> depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.</p>
+<p><strong>$ phoronix-test-suite make-openbenchmarking-cache</strong></p>
+<p>This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.</p>
+<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
+<p>Proceed to start the Phoromatic Server and operate as normal.</p>
+<p>For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.</p>
+<h3>Use Case C: Phoromatic Across The Internet</h3>
+<p>If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's <em>PhoromaticServers</em> element within the <em>user-config.xml</em>. Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.</p>
+<h2>Client Setup</h2>
+<p>From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:</p>
+<p><strong>$ phoronix-test-suite phoromatic.explore</strong></p>
+<p>With the following example output on finding one successful server:</p>
+<p><em>IP: 192.168.1.211<br />
+HTTP PORT: 5447<br />
+WEBSOCKET PORT: 5427<br />
+SERVER: PHP 5.5.9-1ubuntu4.4 Development Server<br />
+PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]<br />
+DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE<br />
+SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:<br />
+      pts - Last Generated: 05 Oct 2014 07:16</em></p>
+<p>Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's <em>~/.phoronix-test-suite/user-config.xml</em> with the <em>PhoromaticServers</em> element. Adding the <em>IP:port</em> (the Phoromatic Server's HTTP port) to the PhoromaticServers <em>user-config.xml</em> element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.</p>
+<p>To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. <em>phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY</em>. When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the <strong>phoronix-test-suite phoromatic.connect</strong> command running on the system (after the initial account has been synced, simply running <strong>phoronix-test-suite phoromatic.connect</strong> is enough for the system to find the server and its account).</p>
+<h2>Root Administrator</h2>
+<p>The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.</p>
+<p>To enable the root administrator log-in, first from the server's command-line interface run <strong>phoronix-test-suite phoromatic.set-root-admin-password</strong> to set the password. Following that, you can log into the root administrator account via the web interface via the <em>rootadmin</em> user-name and the set password.</p>
+<h2>Other Advice</h2>
+<h3>Disable Internet Precaution</h3>
+<p>If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the <em>~/.phoronix-test-suite/user-config.xml</em> you can set <em>NoInternetCommunication</em> to <em>TRUE</em>. There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.</p>
+<h3>Ports / Services</h3>
+<p>The Phoromatic Server process currently relies upon a PHP built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.</p>
+<h3>Systemd</h3>
+<p>Packaged with the Phoronix Test Suite are basic <em>phoromatic-client</em> and <em>phoromatic-server</em> configurations for systemd. The <em>phoromatic-server</em> configuration will launch the Phoronix Test Suite's Phoromatic Server and the <em>phoromatic-client</em> service will attempt to connect to a <u>pre-configured</u> Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite <em>install-sh</em> process.</p>
+<h3>Cache Verification</h3>
+<p>To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the <em>settings</em> page followed by the <em>cache settings</em> link to view information about the download and OpenBenchmarking.org caches. From the client systems, running <strong>phoronix-test-suite phoromatic.explore</strong> will also supply cache statistics.</p>
+<h3>Log Files</h3>
+<p>The Phoromatic Server will produce a log file of events / debugging information to <em>~/.phoronix-test-suite/phoromatic.log</em> or <em>/var/log/phoromatic.log</em> depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in <em>phoronix-test-suite.log</em>.</p>
+<h3>Multi-User Accounts</h3>
+<p>For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the <em>Users</em> tab of their account. The main administrator can make an additional administrator account or a &quot;viewer&quot; account that can consume the account's data but not create/modify the schedules, systems, or other account details.</p>
+<h3>
+<h3>File Locations</h3>
+<p>When running the Phoronix Test Suite Phoromatic Server as root, rather than using the <em>~/.phoronix-test-suite/</em> directory, the standard Linux file-system hierarchy standard is honored. The main storage path is <em>/var/lib/phoronix-test-suite/</em>, the user configuration file is <em>/etc/phoronix-test-suite.xml</em>, and <em>/var/cache/phoronix-test-suite/</em> for cache files.</p>
+<h3>Uploading Other Test Results</h3>
+<p>Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the <em>phoronix-test-suite phoromatic.upload-result &lt;result file identifier &gt;</em> sub-command.</p>
+<h3>User Context File Logging</h3>
+<p>For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the <em>PHOROMATIC_LOG_FILE</em> environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include <em>PHOROMATIC_TRIGGER</em>, <em>PHOROMATIC_SCHEDULE_ID</em>, and <em>PHOROMATIC_SCHEDULE_PROCESS</em>.</p>
 </body>
 </html>

documentation/stubs/67_offline.html

@@ -0,0 +1,14 @@
+<html>
+<head>
+<Title>Offline Testing</Title>
+</head>
+<body>
+<h1>Offline Testing/Benchmarking For Single System Environments</h1>
+<p>The Phoronix Test Suite ships with a cache (up to date as of release time) of the available OpenBenchmarking.org test profile / test suite metadata but external download files are necessary for the test profiles to function. Due to hundreds of different test profiles and consisting of software under test that is of varying software licenses, there is no centralized archive of all possible test files.</p>
+<p>To obtain a cache of the files needed for the desired test(s), from a machine with a working Internet connection, run the <strong>phoronix-test-suite make-download-cache</strong> sub-command and pass the name of the tests/suites you wish to download. The make-download-cache will download the files for the desired test profiles so you can then transfer them for use on individual computer(s) lacking an Internet connection.</p>
+<p>By default the files will be cached to <em>~/.phoronix-test-suite/download-cache</em> and when transferred to an offline system in the same location it should then be automatically utilized by the Phoronix Test Suite when going to install the test(s). The individual download-cache directory can be copied to the offline system or otherwise more broadly the <em>~/.phoronix-test-suite</em> directory can also be copied to the offline system(s). If running as root, the default download cache directory is <strong>/var/cache/phoronix-test-suite/download-cache/</strong>.</p>
+<p>When the Phoronix Test Suite download cache is transferred to the offline system, the Phoronix Test Suite should begin automatically making use of those files when detected in the appropriate directory and having a matching hash-sum for the given file.</p>
+<p>If not able to run the Phoronix Test Suite on a machine with a working Internet connection, manually downloading the files referenced within the test profiles XML metadata and placing them within the respective download-cache directory on a system is another manual alternative.</p>
+<p>Note that the make-download-cache will only cache the files downloaded by the Phoronix Test Suite. Depending upon the test(s) and the state of your operating system, you may also need packages obtained from your package manager / distribution (e.g. compiler, dependency libraries, etc) that are not cached by the Phoronix Test Suite due to the diverse nature of different supported operating systems.</p>
+</body>
+</html>

documentation/stubs/68_offline_additions.html

@@ -1,16 +1,16 @@
 <html>
 <head>
-<Title>Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads</Title>
+<Title>Confidential Testing / Avoiding Accidental Result Uploads</Title>
 </head>
 <body>
 <h1>Offline Enhancement Via Local Cache</h1>
-<p>Beginning with Phoronix Test Suite 9.0, there are improvements to improve the out-of-the-box experience if running the Phoronix Test Suite in a strictly offline environment / behind-the-firewall without access to OpenBenchmarking.org for being able to obtain test profiles / test suites. From Phoronix Test Suite 3.0 when OpenBenchmarking.org was introduced until Phoronix Test Suite 9.0, Internet connectivity was initially required for obtaining the test profiles/suites as the cloud/repository. OpenBenchmarking.org allows for tests to be updated independently of the Phoronix Test Suite releases as well as allowing new tests to be introduced on-demand. Aside from when new tests require explicit new PTS features, this allows tests/suites to be seamlessly used by older versions of the Phoronix Test Suite without any upgrade process required, assuming Internet connectivity is available.</p>
-<p>Beginning with Phoronix Test Suite 9.0, a static snapshot of the official tests/suites is included as part of the Phoronix Test Suite package. The intention with this is to provide a static snapshot with all tests/suites as of release time, similar to the behavior with pre-3.0 releases. The benefit to including this static snapshot is helping those that are running strictly offline/isolated to be able to have at least recent tests/suites available without first needing to query OpenBenchmarking.org for this data. But Internet support is certainly desired in order to be able to obtain updated and new test profiles.</p>
+<p>Since Phoronix Test Suite 9.0, there have been improvements to improve the out-of-the-box experience if running the Phoronix Test Suite in a strictly offline environment / behind-the-firewall without access to OpenBenchmarking.org for being able to obtain test profiles / test suites. From Phoronix Test Suite 3.0 when OpenBenchmarking.org was introduced until Phoronix Test Suite 9.0, Internet connectivity was initially required for obtaining the test profiles/suites as the cloud/repository. OpenBenchmarking.org allows for tests to be updated independently of the Phoronix Test Suite releases as well as allowing new tests to be introduced on-demand. Aside from when new tests require explicit new PTS features, this allows tests/suites to be seamlessly used by older versions of the Phoronix Test Suite without any upgrade process required, assuming Internet connectivity is available.</p>
+<p>A static snapshot of the official tests/suites is included as part of the Phoronix Test Suite package. The intention with this is to provide a static snapshot with all tests/suites as of release time, similar to the behavior with pre-3.0 releases. The benefit to including this static snapshot is helping those that are running strictly offline/isolated to be able to have at least recent tests/suites available without first needing to query OpenBenchmarking.org for this data. But Internet support is certainly desired in order to be able to obtain updated and new test profiles.</p>
 <p>This static snapshot is provided in the <em>ob-cache/</em> Phoronix Test Suite folder. If this cache is not needed or wish to customize/extend it, it can be safely removed and or altered without causing issues. When the Phoronix Test Suite has Internet connectivity, it will continue to query OpenBenchmarking.org for new/updated tests and suites.</p>
 <p>This local cache does provide current and previous versions of test profiles to allow users to continue running older versions of tests/results even when upgrading their Phoronix Test Suite offline copy.</p>
 <p>Even with the local cache, there still is the need for obtaining any necessary files needed to run the selected test(s). For those wishing to optimize that workflow for offline usage, see the existing <em>phoronix-test-suite make-download-cache</em> sub-command documentation. The <em>phoronix-test-suite make-openbenchmarking-cache</em> sub-command may also be desirable depending upon setup.</p>
 <h1>Disabling OpenBenchmarking.org Result Upload Functionality</h1>
-<p>Phoronix Test Suite 9.0 also improved the workflow around disabling OpenBenchmarking.org result uploading functionality for those carrying out confidential tests or otherwise wish to provide safeguards for ensuring no results may be accidentally uploaded publicly.</p>
+<p>Phoronix Test Suite 9.0 improved the workflow around disabling OpenBenchmarking.org result uploading functionality for those carrying out confidential tests or otherwise wish to provide safeguards for ensuring no results may be accidentally uploaded publicly.</p>
 <p>Removal of OpenBenchmarking.org upload support can be done by deleting <em>phoronix-test-suite/pts-core/objects/pts_openbenchmarking_upload.php</em>. If that file is removed, the Phoronix Test Suite should respond gracefully and not prompt users about any upload and within that file is the only logic for actually uploading the results to Openbenchmarking. So simply by removing that file you should be covered from any accidental uploading of results. Removal/disabling of this file also prevents any anonymous usage reporting.</p>
 <p>For those without the ability to remove that file from their Phoronix Test Suite installation or as a secondary safeguard, from the Phoronix Test Suite user configuration file (<em>/etc/phoronix-test-suite.xml<em> as root or <em>~/.phoronix-test-suite/user-config.xml<em> for most users) is a "<em>AllowResultUploadsToOpenBenchmarking</em>" option. If setting that value to <em>FALSE</em>, it should apply the same behavior as if deleting the pts_openbenchmarking_upload file.</p>
 <p>If distributing a customized/local copy of the Phoronix Test Suite, the default behavior of the configuration file (in addition to deleting the pts_openbenchmarking_upload file) can be done via the user configuration defaults defined within <em>pts-core/static/user-config-defaults.xml</em>.</p>

documentation/stubs/99_development_credits.html

@@ -3,4 +3,4 @@
 <Title>Development Credits</Title>
 </head>
 <body>
-<p>The Phoronix Test Suite is based upon the extensive testing and internal tools developed by <a href="http://www.phoronix.com/">Phoronix.com</a> since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are <a href="http://www.michaellarabel.com/">Michael Larabel</a> and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</p>
+<p>The Phoronix Test Suite is based upon the extensive testing and internal tools developed by <a href="http://www.phoronix.com/">Phoronix.com</a> since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are <a href="http://www.michaellarabel.com/">Michael Larabel</a> and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, and nye_Xml are some of the related open-source projects provided by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</p>

documentation/stubs/readme-basics.txt

@@ -1,25 +1,26 @@
-The Phoronix Test Suite itself is an open-source framework for conducting automated tests along with reporting of test results, detection of installed system software/hardware, and other features. Modules for the Phoronix Test Suite also allow for integration with git-bisect and other revision control systems for per-commit regression testing, system sensor monitoring, and other extras.
 
-This framework is designed to be an extensible architecture so that new test profiles and suites can be easily added to represent performance benchmarks, unit tests, and other quantitative and qualitative (e.g. image quality comparison) measurements. Available through OpenBenchmarking.org, a collaborative storage platform developed in conjunction with the Phoronix Test Suite, are more than 200 individual test profiles and more than 60 test suites available by default from the Phoronix Test Suite. Independent users are also able to upload their test results, profiles, and suites to OpenBenchmarking.org. A test profile is a single test that can be executed by the Phoronix Test Suite -- with a series of options possible within every test -- and a test suite is a seamless collection of test profiles and/or additional test suites. A test profile consists of a set of Bash/shell scripts and XML files while a test suite is a single XML file.
+[Phoronix Test Suite Documentation](https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite.md)
+
+The Phoronix Test Suite itself is an open-source framework for conducting automated tests along with reporting of test results, detection of installed system software/hardware, and other features. This framework is designed to be an extensible architecture so that new test profiles and suites can be easily added to represent performance benchmarks, unit tests, and other quantitative and qualitative (e.g. image quality comparison and pass/fail) measurements. Available through OpenBenchmarking.org, a collaborative storage platform developed in conjunction with the Phoronix Test Suite, are more than 600 individual test profiles and more than 200 test suites available by default from the Phoronix Test Suite. Independent users are also able to upload their test results, profiles, and suites to OpenBenchmarking.org.
+
+A test profile is a single test that can be executed by the Phoronix Test Suite -- with a series of options possible within every test -- and a test suite is a seamless collection of test profiles and/or additional test suites. A test profile consists of a set of Bash/shell scripts and XML files while a test suite is a single XML file. Modules for the Phoronix Test Suite also allow for integration with git-bisect and other revision control systems for per-commit regression testing, system sensor monitoring, and other extras.
 
 [OpenBenchmarking.org](https://www.openbenchmarking.org/) also allows for conducting side-by-side result comparisons, a central location for storing and sharing test results, and collaborating over test data. [Phoromatic](https://www.phoromatic.com/) is a complementary platform to OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s) to automatically execute test runs on a timed, per-commit, or other trigger-driven basis. Phoromatic is designed for enterprise and allows for the easy management of multiple networked systems running Phoronix Test Suite clients via a single web-based interface.
 
 Professional support and custom engineering for the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org is available by contacting <https://www.phoronix-test-suite.com/>.
 
-Full details on the Phoronix Test Suite setup and usage is available from the included HTML/PDF documentation within the phoronix-test-suite package and from the Phoronix Test Suite website.
+Full details on the Phoronix Test Suite setup and usage is available from the included HTML/PDF documentation within the phoronix-test-suite package and from the [Phoronix Test Suite documentation directory](documentation/).
 
 ## Installation & Setup
 
-The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, RISC-V, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
+The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM / AArch64, RISC-V, POWER 64-bit), but not all of the test profiles/suites are compatible with all architectures.
 
-The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php7-cli`, `php5-cli`, or `php7`.
+The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php7-cli`, `php5-cli`, `php7` or `php8`.
 
 ## Usage
 
-The process to download, install/setup, execute, and report the results of a benchmark can be as simple as a command such as `phoronix-test-suite benchmark smallpt` to run a simple CPU test profile. If wishing to simply install a test, it's a matter of running `phoronix-test-suite install <test or suite name>` and to run it's `phoronix-test-suite run <test or suite name>`. There's also a batch mode for non-interactive benchmarking by first running `phoronix-test-suite batch-setup` and then using the `batch-run` sub-command rather than `run`.
+The process to download, install/setup, execute, and report the results of a benchmark can be as simple as a command such as `phoronix-test-suite benchmark smallpt` to run a simple CPU test profile. If wishing to simply install a test, it's a matter of running `phoronix-test-suite install <test or suite name>` and to run it's `phoronix-test-suite run <test or suite name>`. There's also a batch mode for non-interactive benchmarking by first running `phoronix-test-suite batch-setup` and then using the `batch-run` sub-command rather than `run`. For more guidance see the [Phoronix Test Suite documentation](documentation/phoronix-test-suite.md).
 
 Viewing installed system hardware and software is available via `phoronix-test-suite system-info`.
 
 Facilitating a result comparison from OpenBenchmarking.org can be done by running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if wishing to compare the results of the `https://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
-
-Additional information is available from the Phoronix Test Suite website <https://www.phoronix-test-suite.com/> and the material bundled within the `phoronix-test-suite/documentation/` directory or via <https://www.phoronix-test-suite.com/documentation/>. A man page is also bundled with the phoronix-test-suite software.

documentation/test-profile-creation.md

@@ -0,0 +1,40 @@
+
+# Creating Custom Tests / Benchmarks (Test Profiles)
+
+## Overview
+
+*This test profile creation documentation is a work in progress.*
+
+*Note: For some of the sample/debug test profiles mentioned on this page you may also need to first run the `phoronix-test-suite enable-repo debug` command to enable access to the tests on your local system.*
+
+The easiest way to get started with creating a test at this time would be by modifying an existing test profile, such as:
+
+> cp -va ~/.phoronix-test-suite/test-profiles/pts/build-llvm-1.3.1/ ~/.phoronix-test-suite/test-profiles/local/build-llvm/
+> (Edit the test contents within ~/.phoronix-test-suite/test-profiles/local/build-llvm/ )
+> ./phoronix-test-suite benchmark local/build-llvm # to try out the modified test profile
+
+Some built-in Phoronix Test Suite commands that may help in test profile creation are `phoronix-test-suite create-test-profile` to help in the generation of the standard XML metadata for test profiles and other boilerplate code, `phoronix-test-suite debug-install [test]` to see the output of the install process, `phoronix-test-suite debug-run [test]` to debug the test run-time behavior, `phoronix-test-suite debug-result-parser [test]` if trying to debug the result parser XML handling, and `phoronix-test-suite inspect-test-profile [test]` to view the parsed Phoronix Test Suite test profile.
+
+The `phoronix-test-suite diagnostics` output can also be beneficial for seeing the environment variables that by default are exported to all test profiles if needing to query certain software/hardware information.
+
+## Pass/Fail Tests
+
+The Phoronix Test Suite is primarily focused on quantitative tests, but does support pass/fail type testing (e.g. success / failure). See `phoronix-test-suite benchmark debug/pass-fail` as an example test profile for how to setup a pass/fail test. Similarly, there is `phoronix-test-suite benchmark debug/multi-pass-fail` for the multi-pass/fail of multiple pass/failures as part of a single result.
+
+## Tests With Single Run, Multiple Results Output
+
+See test profiles such as *pts/hpcc* and *pts/fio* for test profiles having a *results-definition.xml* where multiple results are generated from a single run.
+
+## Sensor-Based Monitoring As A Test Result
+
+See *pts/video-cpu-usage* as an (old) example of a working test centered around CPU usage tracking during video playback.
+
+## Triggering Reboot During Test Install Or Test Run-Time
+
+Since Phoronix Test Suite 10.6 is a built-in helper if a test profile during installation or at run-time needs to trigger a system reboot. This can be done by having the test install script or run script write to *~/reboot-needed*. If *~/reboot-needed* is created by the test profile, the Phoronix Test Suite will attempt to reboot the system in a cross-platform compatible manner.
+
+The default behavior if reboot-needed is present is to reboot as soon as the test script execution finishes. This is the "immediate" mode and the default. Alternatively, if "queued" is written to the reboot-needed file, the Phoronix Test Suite will wait until after all other tests are either installed or run before triggering the reboot (or until running a test requesting an immediate reboot). The queued mode is intended to cut-down on the possible number of reboots needed, depending upon the constraints and intentions of the test profile rebooting.
+
+When the test is recovering/re-run after a reboot, the Phoronix Test Suite will set the *$TEST_RECOVERING_FROM_REBOOT* environment variable to let the test profile know a reboot happened. Note that the $TEST_RECOVERING_FROM_REBOOT behavior will only report if it's the first subsequent run of the Phoronix Test Suite since the reboot was initiated, i.e. if there was an interruption and running PTS later on past that point of the test in question, the $TEST_RECOVERING_FROM_REBOOT would not be set. The $TEST_RECOVERING_FROM_REBOOT will also not be set if the Phoronix Test Suite client version was upgraded/changed during the reboot process.
+
+Running `phoronix-test-suite benchmark debug/reboot-now` is a sample test profile demonstrating the reboot interface. Similarly, `phoronix-test-suite benchmark debug/reboot-during-install` demonstrates the reboot-needed activity on test installation.

install-sh

@@ -50,7 +50,7 @@ rm -rf $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
 rm -rf $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
 
 mkdir -p $DESTDIR$INSTALL_PREFIX/bin/
-mkdir -p $DESTDIR$INSTALL_PREFIX/share/appdata/
+mkdir -p $DESTDIR$INSTALL_PREFIX/share/metainfo/
 mkdir -p $DESTDIR$INSTALL_PREFIX/share/applications/
 mkdir -p $DESTDIR$INSTALL_PREFIX/share/icons/hicolor/48x48/apps/
 mkdir -p $DESTDIR$INSTALL_PREFIX/share/man/man1/
@@ -74,7 +74,7 @@ cp pts-core/static/bash_completion $DESTDIR$INSTALL_PREFIX/../etc/bash_completio
 cp pts-core/static/images/phoronix-test-suite.png $DESTDIR$INSTALL_PREFIX/share/icons/hicolor/48x48/apps/phoronix-test-suite.png
 cp pts-core/static/phoronix-test-suite.desktop $DESTDIR$INSTALL_PREFIX/share/applications/
 cp pts-core/static/phoronix-test-suite-launcher.desktop $DESTDIR$INSTALL_PREFIX/share/applications/
-cp pts-core/static/phoronix-test-suite.appdata.xml $DESTDIR$INSTALL_PREFIX/share/appdata/
+cp pts-core/static/com.phoronix_test_suite.phoronix_test_suite.metainfo.xml $DESTDIR$INSTALL_PREFIX/share/metainfo/
 
 mkdir -p $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
 cp deploy/*-systemd/*.service $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/

install.bat

@@ -2,7 +2,7 @@
 
 :: Phoronix Test Suite
 :: URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-:: Copyright (C) 2018, Phoronix Media
+:: Copyright (C) 2018 - 2020, Phoronix Media
 ::
 :: This program is free software; you can redistribute it and/or modify
 :: it under the terms of the GNU General Public License as published by
@@ -29,5 +29,5 @@ md %destination%
 :: cd /d %destination%
 :: for /F "delims=" %%i in ('dir /b') do (rmdir "%%i" /s/q || del "%%i" /s/q)
 
-xcopy %cd% %destination% /E
+xcopy "%cd%" %destination% /E
 echo Phoronix Test Suite installed to %destination%

install_macos

@@ -0,0 +1,125 @@
+#!/bin/sh
+
+# Phoronix Test Suite
+# URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+# Copyright (C) 2008 - 2019, Phoronix Media
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Generic Phoronix Test Suite installer
+
+# To install into a chroot, set $DESTDIR to the corresponding directory.
+
+die() {
+	printf '%s\n' "$1" >&2
+	exit "${2:-1}"
+}
+
+if [ "X$1" = "X" ]
+then
+	INSTALL_PREFIX="/usr"
+else
+	INSTALL_PREFIX="$1"
+fi
+
+# Ensure the user is in the correct directory
+if [ ! -f pts-core/phoronix-test-suite.php ]
+then
+	die '
+	To install the Phoronix Test Suite you must first change directories to phoronix-test-suite/. For support visit: http://www.phoronix-test-suite.com/
+'
+fi
+
+mkdir -p "$DESTDIR$INSTALL_PREFIX"
+if [ ! -w "$DESTDIR$INSTALL_PREFIX" ]
+then
+	die "ERROR: $DESTDIR$INSTALL_PREFIX is not writable. Run this installer as root or specify a different directory prefix as the first argument sent to this script."
+fi
+
+rm -rf "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/
+rm -rf "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/bin/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/applications/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/icons/hicolor/48x48/apps/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/man/man1/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/../etc/bash_completion.d/
+#mkdir -p $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
+#mkdir -p $DESTDIR$INSTALL_PREFIX/../etc/init/
+
+cp ChangeLog "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+cp COPYING "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+cp AUTHORS "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+
+cd documentation/ || exit
+cp -r * "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/
+cd ..
+rm -rf "$DESTDIR$INSTALL_PREFIX"/share/doc/phoronix-test-suite/man-pages/
+
+cp documentation/man-pages/*.1 "$DESTDIR$INSTALL_PREFIX"/share/man/man1/
+cp pts-core/static/bash_completion "$DESTDIR$INSTALL_PREFIX"/../etc/bash_completion.d/phoronix-test-suite
+cp pts-core/static/images/phoronix-test-suite.png "$DESTDIR$INSTALL_PREFIX"/share/icons/hicolor/48x48/apps/phoronix-test-suite.png
+cp pts-core/static/phoronix-test-suite.desktop "$DESTDIR$INSTALL_PREFIX"/share/applications/
+cp pts-core/static/phoronix-test-suite-launcher.desktop "$DESTDIR$INSTALL_PREFIX"/share/applications/
+
+mkdir -p "$DESTDIR$INSTALL_PREFIX"/../usr/lib/systemd/system/
+cp deploy/*-systemd/*.service "$DESTDIR$INSTALL_PREFIX"/../usr/lib/systemd/system/
+
+# mkdir -p $DESTDIR$INSTALL_PREFIX/../etc/init/
+# cp pts-core/static/upstart/*.conf $DESTDIR$INSTALL_PREFIX/../etc/init/
+
+rm -rf "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core
+cp -r pts-core "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/
+cp -r ob-cache "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/
+cp -r deploy "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/static/phoronix-test-suite.desktop
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/static/phoronix-test-suite-launcher.desktop
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/openbenchmarking.org/openbenchmarking-mime.xml
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/static/bash_completion
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/static/images/openbenchmarking.png
+rm -f "$DESTDIR$INSTALL_PREFIX"/share/phoronix-test-suite/pts-core/static/images/%phoronix-test-suite.png
+
+
+sed -e 's:export PTS_DIR=$(actualpath "$(dirname "$0")"):export PTS_DIR='"$INSTALL_PREFIX"'\/share\/phoronix-test-suite:g' phoronix-test-suite > $DESTDIR$INSTALL_PREFIX/bin/phoronix-test-suite
+chmod +x "$DESTDIR$INSTALL_PREFIX"/bin/phoronix-test-suite
+
+# sed 's:\$url = PTS_PATH . \"documentation\/index.html\";:\$url = \"'"$INSTALL_PREFIX"'\/share\/doc\/packages\/phoronix-test-suite\/index.html\";:g' pts-core/commands/gui_gtk.php > $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/commands/gui_gtk.php
+
+# XDG MIME OpenBenchmarking support
+if [ "X$DESTDIR" = "X" ] && which xdg-mime >/dev/null && which xdg-icon-resource >/dev/null
+then
+	#No chroot
+	xdg-mime install pts-core/openbenchmarking.org/openbenchmarking-mime.xml
+	xdg-icon-resource install --context mimetypes --size 64 pts-core/static/images/openbenchmarking.png application-x-openbenchmarking
+else
+	#chroot
+	mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/mime/packages/
+	mkdir -p "$DESTDIR$INSTALL_PREFIX"/share/icons/hicolor/64x64/mimetypes/
+	cp pts-core/openbenchmarking.org/openbenchmarking-mime.xml "$DESTDIR$INSTALL_PREFIX"/share/mime/packages/
+	cp pts-core/static/images/openbenchmarking.png "$DESTDIR$INSTALL_PREFIX"/share/icons/hicolor/64x64/mimetypes/application-x-openbenchmarking.png
+
+fi
+
+printf '%b\n' "\nPhoronix Test Suite Installation Completed\n
+Executable File: $INSTALL_PREFIX/bin/phoronix-test-suite
+Documentation: $INSTALL_PREFIX/share/doc/phoronix-test-suite/
+Phoronix Test Suite Files: $INSTALL_PREFIX/share/phoronix-test-suite/\n"
+
+if [ "X$DESTDIR" != "X" ]
+then
+	echo "Installed to chroot: $DESTDIR"
+	echo "Please update your desktop and mime-database manually"
+fi

ob-cache/test-profiles/git/aom-av1-1.1.0/changelog.json

@@ -0,0 +1 @@
+{"1.1.0":{"last_updated":"1552063091","commit_description":"Git version of AOM-AV1."}}

ob-cache/test-profiles/git/aom-av1-1.1.0/generated.json

@@ -0,0 +1 @@
+{"overview":{"244a0e9a456f6a8561ce62689c1c96b783484955":{"description":"AV1 Video Encoding","test_version":"1.1.x","app_version":"Git","unit":"Frames Per Second","samples":48,"timing_samples":47,"hib":1,"first_appeared":1552068371,"last_appeared":1681404639,"run_time_avg":1344,"stddev_avg":0.45,"percentiles":[0.01,0.01,0.01,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.02,0.03,0.03,0.03,0.03,0.05,0.05,0.05,0.05,0.07,0.07,0.07,0.07,0.07,0.07,0.07,0.07,0.07,0.07,0.08,0.08,0.08,0.13,0.13,0.13,0.13,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.14,0.15,0.15,0.15,0.15,0.23,0.23,0.23,0.3,0.3,3.56,3.56,4.55,4.55,5.87,5.87,16.14,16.14,69.08,69.08,70.89,70.89,71.24,71.24,71.41,71.41,73.11,73.11,136.37,136.37],"run_time_percentiles":[2.5100000000000002,2.5100000000000002,2.5100000000000002,4.449999999999999,4.449999999999999,4.56,4.56,4.58,4.58,4.640000000000001,4.640000000000001,4.719999999999999,4.719999999999999,10,10,13,13,13,15,15,17,17,200,200,263,263,390,390,396,396,438,438,438,438,438,439,439,439,439,440,440,442,442,442,442,779,779,822,822,847,847,847,851,851,852,852,893,893,1162,1162,1173,1173,1761,1761,1791,1791,1802,1802,1802,2300,2300,2643,2643,3111,3111,3113,3113,3116,3116,3121,3121,3123,3123,3123,3123,3123,3133,3133,3140,3140,3173,3173,3207,3207,3225,3225,3236,3236,3262,3262],"tested_archs":["x86_64","aarch64"]}}}

ob-cache/test-profiles/git/clickhouse-1.0.0/changelog.json

@@ -0,0 +1 @@
+{"1.0.0":{"last_updated":"1673526602","commit_description":"Git-based ClickHouse test profile for QPL testing."}}

ob-cache/test-profiles/git/clickhouse-1.0.0/downloads.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>https://datasets.clickhouse.com/hits_compatible/hits.tsv.gz</URL>
+      <MD5>de2f86030d1c86fd39d03468bd90a911</MD5>
+      <SHA256>bd500f777ef0eb55dc9d4f0eb245870351cc44a44d72a56395048d5c15f09fe2</SHA256>
+      <FileName>hits.tsv.gz</FileName>
+      <FileSize>16298506510</FileSize>
+    </Package>
+    <Package>
+      <URL>https://github.com/ClickHouse/ClickBench/archive/d9a1281ca7d2dd6c5144bd801a5ce493c0fe6fa0.zip</URL>
+      <MD5>c9e554e017403f399f3175258e12befa</MD5>
+      <SHA256>6ab8955c8fc0af8946f2b6d5dfdef708eafd89826e3ffa7204f664f256240238</SHA256>
+      <FileName>ClickBench-d9a1281ca7d2dd6c5144bd801a5ce493c0fe6fa0.zip</FileName>
+      <FileSize>627752</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/clickhouse-1.0.0/generated.json

@@ -0,0 +1 @@
+{"overview":[]}

ob-cache/test-profiles/git/clickhouse-1.0.0/install.sh

@@ -0,0 +1,91 @@
+#!/bin/bash
+git clone --recursive https://github.com/ClickHouse/ClickHouse.git clickhouse-git
+mkdir build
+cd build
+EXTRA_FLAGS="-DCMAKE_BUILD_TYPE=Release "
+if grep avx /proc/cpuinfo > /dev/null
+then
+	EXTRA_FLAGS="$EXTRA_FLAGS -DENABLE_AVX=ON "
+fi
+if grep bmi /proc/cpuinfo > /dev/null
+then
+	EXTRA_FLAGS="$EXTRA_FLAGS -DENABLE_BMI=ON "
+fi
+if grep avx2 /proc/cpuinfo > /dev/null
+then
+	EXTRA_FLAGS="$EXTRA_FLAGS -DENABLE_AVX2=ON "
+	if grep avx512 /proc/cpuinfo > /dev/null
+	then
+		EXTRA_FLAGS="$EXTRA_FLAGS -DENABLE_AVX512=ON -DENABLE_AVX512_FOR_SPEC_OP=ON "
+		if type accel-config > /dev/null
+		then
+			EXTRA_FLAGS="$EXTRA_FLAGS -DENABLE_QPL=ON "
+		fi
+	fi
+fi
+echo "Clickhouse CMake build configuration: $EXTRA_FLAGS"
+cmake ../clickhouse-git $EXTRA_FLAGS
+ninja
+echo $? > ~/install-exit-status
+cd ~
+./build/programs/clickhouse server --version > ~/install-footnote 2>&1
+unzip -o ClickBench-d9a1281ca7d2dd6c5144bd801a5ce493c0fe6fa0.zip
+gzip -d -k hits.tsv.gz
+cp ClickBench-d9a1281ca7d2dd6c5144bd801a5ce493c0fe6fa0/clickhouse/queries.sql queries.sql
+mkdir config.d
+echo "<clickhouse>
+<profiles>
+<default>
+	<allow_experimental_codecs>1</allow_experimental_codecs>
+</default>
+</profiles>
+<compression>
+<case>
+	<method>deflate_qpl</method>
+</case>
+</compression>
+</clickhouse>" > config.d/qpl.xml
+CLICKHOUSE_CLIENT=build/programs/clickhouse
+echo "#!/bin/bash
+rm -rf d*
+rm -rf f*
+rm -rf m*
+rm -rf n*
+rm -rf preprocessed_configs
+rm -rf s*
+rm -rf tmp
+rm -rf u*
+
+TRIES=3
+./$CLICKHOUSE_CLIENT server 2>/dev/null &
+SERVER_PID=\$!
+sleep 5
+./$CLICKHOUSE_CLIENT client < ClickBench-d9a1281ca7d2dd6c5144bd801a5ce493c0fe6fa0/clickhouse/create-tuned.sql
+./$CLICKHOUSE_CLIENT client --time --query \"INSERT INTO hits FORMAT TSV\" < hits.tsv
+echo \$? > ~/test-exit-status
+cat queries.sql | while read query; do
+    sync
+    echo \"QUERY: \$query\" >> \$LOG_FILE
+    for i in \$(seq 1 \$TRIES); do
+    	echo -n \"Clickhouse Query Time \$i: \" >> \$LOG_FILE
+    	./$CLICKHOUSE_CLIENT client --time --format=Null --max_memory_usage=100G --max_threads=\$NUM_CPU_CORES --query=\"\$query\" --progress 0 >> \$LOG_FILE 2>&1
+    	retval=\$?
+	if [ \$retval -ne 0 ]; then
+	    echo \$retval > ~/test-exit-status
+	    kill -9 \$SERVER_PID
+	    sleep 3
+	    exit
+	fi
+    done
+done
+kill -9 \$SERVER_PID
+sleep 2
+rm -rf d*
+rm -rf f*
+rm -rf m*
+rm -rf n*
+rm -rf preprocessed_configs
+rm -rf s*
+rm -rf tmp
+rm -rf u*" > clickhouse
+chmod +x clickhouse

ob-cache/test-profiles/git/clickhouse-1.0.0/results-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Clickhouse Query Time 1: #_RESULT_#</OutputTemplate>
+    <LineHint>Clickhouse Query Time 1:</LineHint>
+    <MultiMatch>GEOMETRIC_MEAN</MultiMatch>
+    <DivideResultDivisor>60</DivideResultDivisor>
+    <ArgumentsDescription>100M Rows Hits Dataset, First Run / Cold Cache</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Clickhouse Query Time 2: #_RESULT_#</OutputTemplate>
+    <LineHint>Clickhouse Query Time 2:</LineHint>
+    <MultiMatch>GEOMETRIC_MEAN</MultiMatch>
+    <DivideResultDivisor>60</DivideResultDivisor>
+    <ArgumentsDescription>100M Rows Hits Dataset, Second Run</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Clickhouse Query Time 3: #_RESULT_#</OutputTemplate>
+    <LineHint>Clickhouse Query Time 3:</LineHint>
+    <MultiMatch>GEOMETRIC_MEAN</MultiMatch>
+    <DivideResultDivisor>60</DivideResultDivisor>
+    <ArgumentsDescription>100M Rows Hits Dataset, Third Run</ArgumentsDescription>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/clickhouse-1.0.0/test-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>ClickHouse</Title>
+    <Description>ClickHouse is an open-source, high performance OLAP data management system. This test profile uses ClickHouse's standard benchmark recommendations per https://clickhouse.com/docs/en/operations/performance-test/ / https://github.com/ClickHouse/ClickBench/tree/main/clickhouse with the 100 million rows web analytics dataset. The reported value is the query processing time using the geometric mean of all separate queries performed as an aggregate. This test profile is currently an experimental build of ClickHouse from Git and setup for Intel QPL accelerator use.</Description>
+    <ResultScale>Queries Per Minute, Geo Mean</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Application</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Experimental</Status>
+    <ExternalDependencies>build-utilities, cmake, nasm, git, clang</ExternalDependencies>
+    <EnvironmentSize>91900</EnvironmentSize>
+    <ProjectURL>https://clickhouse.com/</ProjectURL>
+    <RepositoryURL>https://github.com/ClickHouse/ClickHouse</RepositoryURL>
+    <Maintainer>Michael Larabel</Maintainer>
+    <SystemDependencies>ninja, lld</SystemDependencies>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/dav1d-1.0.1/changelog.json

@@ -0,0 +1 @@
+{"1.0.1":{"last_updated":"1579721829","commit_description":"Ensure dav1d-git build folder is cleaned on reinstall."},"1.0.0":{"last_updated":"1579697045","commit_description":"Add a Git version of dav1d benchmark."}}

ob-cache/test-profiles/git/dav1d-1.1.0/changelog.json

@@ -0,0 +1 @@
+{"1.1.0":{"last_updated":"1647888021","commit_description":"Update download links, adjust for v1.0+ that have changed CLI options, add threads\/framedelay options."},"1.0.1":{"last_updated":"1579721829","commit_description":"Ensure dav1d-git build folder is cleaned on reinstall."},"1.0.0":{"last_updated":"1579697045","commit_description":"Add a Git version of dav1d benchmark."}}

ob-cache/test-profiles/git/dav1d-1.1.0/downloads.xml

@@ -0,0 +1,42 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ffmpeg.org/releases/ffmpeg-4.4.tar.xz</URL>
+      <MD5>7b9d5b652d20e8c5405304ad72636d4a</MD5>
+      <SHA256>06b10a183ce5371f915c6bb15b7b1fffbe046e8275099c96affc29e17645d909</SHA256>
+      <FileName>ffmpeg-4.4.tar.xz</FileName>
+      <FileSize>9557868</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/Stream2_AV1_HD_6.8mbps.webm</URL>
+      <MD5>5e4c83cf494cbe6dc1668cbbd7ff774c</MD5>
+      <SHA256>2f23d29750a0663a6df656e8137cf934bddfc96b31e5088db2c3624f19ed14d4</SHA256>
+      <FileName>Stream2_AV1_HD_6.8mbps.webm</FileName>
+      <FileSize>122378926</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/Stream2_AV1_4K_22.7mbps.webm</URL>
+      <MD5>8acc9c60c10a37bf4e568b465b03e35a</MD5>
+      <SHA256>52f3aa1d4b4487af62d37b0f295aabbc4b57f03fdc4c76402c6358193e4aa490</SHA256>
+      <FileName>Stream2_AV1_4K_22.7mbps.webm</FileName>
+      <FileSize>409458359</FileSize>
+    </Package>
+    <Package>
+      <URL>http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/Old/Chimera-AV1-8bit-1920x1080-6736kbps.mp4</URL>
+      <MD5>d444e11f8b049de2a9739899a9c0b4f6</MD5>
+      <SHA256>d566d294e2c18bb274a54aad03352c92312a62c393656d38e1f7dda10c0bf10c</SHA256>
+      <FileName>Chimera-AV1-8bit-1920x1080-6736kbps.mp4</FileName>
+      <FileSize>313490236</FileSize>
+    </Package>
+    <Package>
+      <URL>http://download.opencontent.netflix.com.s3.amazonaws.com/AV1/Chimera/Old/Chimera-AV1-10bit-1920x1080-6191kbps.mp4</URL>
+      <MD5>4c0f24bac42ddcda3d46087a36492ce4</MD5>
+      <SHA256>df2080fd77e0dbd9138bd4f172bf008d2ade17da7ab4532fba54ceccf40a9439</SHA256>
+      <FileName>Chimera-AV1-10bit-1920x1080-6191kbps.mp4</FileName>
+      <FileSize>288122748</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/dav1d-1.1.0/install.sh

@@ -0,0 +1,36 @@
+#!/bin/sh
+
+# FFmpeg install to demux AV1 WebM to IVF that can then be consumed by dav1d...
+tar -xf ffmpeg-4.4.tar.xz
+mkdir ffmpeg_/
+
+cd ffmpeg-4.4
+./configure --disable-zlib --disable-doc --prefix=$HOME/ffmpeg_/
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+make install
+cd ~/
+
+./ffmpeg_/bin/ffmpeg -i Stream2_AV1_HD_6.8mbps.webm -vcodec copy -an -f ivf summer_nature_1080p.ivf
+./ffmpeg_/bin/ffmpeg -i Stream2_AV1_4K_22.7mbps.webm -vcodec copy -an -f ivf summer_nature_4k.ivf
+./ffmpeg_/bin/ffmpeg -i Chimera-AV1-8bit-1920x1080-6736kbps.mp4 -vcodec copy -an -f ivf chimera_8b_1080p.ivf
+./ffmpeg_/bin/ffmpeg -i Chimera-AV1-10bit-1920x1080-6191kbps.mp4 -vcodec copy -an -f ivf chimera_10b_1080p.ivf
+
+rm -rf ffmpeg-4.2.1
+rm -rf ffmpeg_
+
+# Build Dav1d
+rm -rf dav1d-git
+git clone https://code.videolan.org/videolan/dav1d.git dav1d-git
+cd dav1d-git
+git rev-parse --short HEAD > ~/install-footnote
+mkdir build
+meson build --buildtype release
+ninja -C build
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+./dav1d-git/build/tools/dav1d \$@ > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > dav1d
+chmod +x dav1d

ob-cache/test-profiles/git/dav1d-1.1.0/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Decoded 8929 8929 frames (100.0%) - #_RESULT_# 25 fps (9.20x)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>/</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/dav1d-1.1.0/test-definition.xml

@@ -0,0 +1,140 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>dav1d</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>Dav1d is an open-source, speedy AV1 video decoder. This test profile times how long it takes to decode sample AV1 video content. The git/dav1d test profile builds against current dav1d Git, for the static version use the pts/dav1d test profile.</Description>
+    <ResultScale>FPS</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.0</Version>
+    <SupportedPlatforms>Linux, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, yasm, meson, git</ExternalDependencies>
+    <EnvironmentSize>1200</EnvironmentSize>
+    <ProjectURL>https://www.videolan.org/projects/dav1d.html</ProjectURL>
+    <RepositoryURL>http://code.videolan.org/videolan/dav1d</RepositoryURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>--muxer null  </Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Video Input</DisplayName>
+      <Identifier>video</Identifier>
+      <ArgumentPrefix>-i </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>Summer Nature 1080p</Name>
+          <Value>summer_nature_1080p.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Summer Nature 4K</Name>
+          <Value>summer_nature_4k.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Chimera 1080p</Name>
+          <Value>chimera_8b_1080p.ivf</Value>
+        </Entry>
+        <Entry>
+          <Name>Chimera 1080p 10-bit</Name>
+          <Value>chimera_10b_1080p.ivf</Value>
+        </Entry>
+      </Menu>
+    </Option>
+    <Option>
+      <DisplayName>Threads</DisplayName>
+      <Identifier>threads</Identifier>
+      <ArgumentPrefix>--threads </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>1</Name>
+          <Value>1</Value>
+        </Entry>
+        <Entry>
+          <Name>4</Name>
+          <Value>4</Value>
+        </Entry>
+        <Entry>
+          <Name>8</Name>
+          <Value>8</Value>
+        </Entry>
+        <Entry>
+          <Name>16</Name>
+          <Value>16</Value>
+        </Entry>
+        <Entry>
+          <Name>32</Name>
+          <Value>32</Value>
+        </Entry>
+        <Entry>
+          <Name>64</Name>
+          <Value>64</Value>
+        </Entry>
+        <Entry>
+          <Name>128</Name>
+          <Value>128</Value>
+        </Entry>
+        <Entry>
+          <Name>160</Name>
+          <Value>160</Value>
+        </Entry>
+        <Entry>
+          <Name>256</Name>
+          <Value>256</Value>
+        </Entry>
+        <Entry>
+          <Name>384</Name>
+          <Value>384</Value>
+        </Entry>
+      </Menu>
+    </Option>
+    <Option>
+      <DisplayName>Frame Delay</DisplayName>
+      <Identifier>frame-delay</Identifier>
+      <ArgumentPrefix>--framedelay </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>1</Name>
+          <Value>1</Value>
+        </Entry>
+        <Entry>
+          <Name>4</Name>
+          <Value>4</Value>
+        </Entry>
+        <Entry>
+          <Name>8</Name>
+          <Value>8</Value>
+        </Entry>
+        <Entry>
+          <Name>12</Name>
+          <Value>12</Value>
+        </Entry>
+        <Entry>
+          <Name>16</Name>
+          <Value>16</Value>
+        </Entry>
+        <Entry>
+          <Name>24</Name>
+          <Value>24</Value>
+        </Entry>
+        <Entry>
+          <Name>32</Name>
+          <Value>32</Value>
+        </Entry>
+        <Entry>
+          <Name>64</Name>
+          <Value>64</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/rav1e-1.0.0/changelog.json

@@ -0,0 +1 @@
+{"1.0.0":{"last_updated":"1578155433","commit_description":"Add initial rav1e Git test https:\/\/github.com\/phoronix-test-suite\/test-profiles\/pull\/117"}}

ob-cache/test-profiles/git/svt-av1-2.3.0/changelog.json

@@ -0,0 +1 @@
+{"2.3.0":{"last_updated":"1623277644","commit_description":"Update test profile for changed GitLab repository from GitHub, update against pts\/svt-av1 version with 4K option, etc. Pull per https:\/\/github.com\/phoronix-test-suite\/test-profiles\/pull\/200"},"1.2.0":{"last_updated":"1552059151","commit_description":"Commit of svt-av1 test to git\/ that uses Git sources of encoder for benchmark."}}

ob-cache/test-profiles/git/svt-av1-2.3.0/downloads.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.4.0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</URL>
+      <MD5>db7c7ff09acd5d7820cc4d1eb0939cf9</MD5>
+      <SHA256>e73a54088e88e6465f578625d185933e85c3209ab7105deb755a8b8918b78cab</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</FileName>
+      <FileSize>680772328</FileSize>
+    </Package>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_3840x2160_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>815ca5830c8ea2f95545429817b96b1f</MD5>
+      <SHA256>d0fcb7d8cfc6e51c47ce8b427dab41ce38300c4b1a51a1903ddd5d86067c4c79</SHA256>
+      <FileName>Bosphorus_3840x2160_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>2804470345</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-av1-2.3.0/install.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z -aoa
+7z x Bosphorus_3840x2160_120fps_420_8bit_YUV_Y4M.7z -aoa
+
+rm -rf SVT-AV1-master
+git clone https://gitlab.com/AOMediaCodec/SVT-AV1.git SVT-AV1-master
+./SVT-AV1-master/Build/linux/build.sh release
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+./SVT-AV1-master/Bin/Release/SvtAv1EncApp \$@ > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > svt-av1
+chmod +x svt-av1
+

ob-cache/test-profiles/git/svt-av1-2.3.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.4.0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Average Speed:		#_RESULT_# fps</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-av1-2.3.0/test-definition.xml

@@ -0,0 +1,58 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.4.0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>SVT-AV1</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a benchmark of the SVT-AV1 open-source video encoder/decoder. SVT-AV1 was originally developed by Intel as part of their Open Visual Cloud / Scalable Video Technology (SVT). Development of SVT-AV1 has since moved to the Alliance for Open Media as part of upstream AV1 development. SVT-AV1 is a CPU-based multi-threaded video encoder for the AV1 video format with a sample YUV video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>2.3.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, yasm, cmake, git</ExternalDependencies>
+    <EnvironmentSize>2900</EnvironmentSize>
+    <ProjectURL>https://gitlab.com/AOMediaCodec/SVT-AV1</ProjectURL>
+    <RepositoryURL>https://gitlab.com/AOMediaCodec/SVT-AV1</RepositoryURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Option>
+      <DisplayName>Encoder Mode</DisplayName>
+      <Identifier>enc-mode</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Preset 8</Name>
+          <Value>--preset 8</Value>
+          <Message>Fastest - Default</Message>
+        </Entry>
+        <Entry>
+          <Name>Preset 4</Name>
+          <Value>--preset 4 -n 160</Value>
+          <Message>Mid-Speed</Message>
+        </Entry>
+      </Menu>
+    </Option>
+    <Option>
+      <DisplayName>Input</DisplayName>
+      <Identifier>input</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Bosphorus 1080p</Name>
+          <Value>-i Bosphorus_1920x1080_120fps_420_8bit_YUV.yuv -w 1920 -h 1080</Value>
+        </Entry>
+        <Entry>
+          <Name>Bosphorus 4K</Name>
+          <Value>-i Bosphorus_3840x2160.y4m -w 3840 -h 2160</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-hevc-1.0.0/changelog.json

@@ -0,0 +1 @@
+{"1.0.0":{"last_updated":"1552060068","commit_description":"SVT-HEVC built from Git."}}

ob-cache/test-profiles/git/svt-hevc-1.0.0/generated.json

@@ -0,0 +1 @@
+{"overview":{"622f5f75d068059e83e0f1d74060a7bd53ef41a5":{"description":"1080p 8-bit YUV To HEVC Video Encode","test_version":"1.0.x","app_version":"Git","unit":"Frames Per Second","samples":85,"timing_samples":84,"hib":1,"first_appeared":1552068371,"last_appeared":1689666386,"run_time_avg":167,"stddev_avg":2.37,"percentiles":[0.25,0.25,0.38,1.14,1.54,4.24,4.65,4.65,5.3,5.6,7.38,7.39,40.35,40.5,40.5,51.32,51.33,54.7,76.93,77.43,77.81,77.81,81.94,83.56,84.12,87.12,87.4,87.4,95.85,96.04,97.06,97.57,97.6,97.69,97.69,98.56,98.75,98.79,98.94,99.35,99.44,99.44,99.71,99.92,100.38,100.8,100.85,100.85,101.11,101.36,101.53,102.89,104.36,106.75,106.75,107.56,108.22,108.71,115.3,120.2,125.26,125.26,125.66,126.55,137.89,142.12,146.55,146.55,147.65,147.88,149.46,150.38,150.6,151,151,151.06,152.13,174.42,184.3,185.29,196.71,196.71,203.28,210.13,228.72,249.59,253,253,257.61,257.9,261,261.08,265.85,277.69,277.69,292.81,298.36,343.67,363,363.18],"run_time_percentiles":[7.28,7.28,8.18,8.43,8.89,9.36,9.85,9.85,10,10,11,12,13,13,13,13,13,14,14,14,14,15,16,17,18,18,18,18,18,18,18,18,18,18,19,19,19,19,19,19,19,20,20,20,20,21,22,22,24,24,24,24,25,25,25,26,28,28,31,34,37,37,39,39,41,43,43,43,45,45,45,45,46,46,51,51,51,60,65,80,92,94,94,95,96,96,97,98,98,126,247,247,325,342,342,391,430,1577,3211,4760],"tested_archs":["x86_64"]}}}

ob-cache/test-profiles/git/svt-vp9-1.0.0/changelog.json

@@ -0,0 +1 @@
+{"1.0.0":{"last_updated":"1552059681","commit_description":"Test profile allows testing SVT-VP9 from Git."}}

ob-cache/test-profiles/git/svt-vp9-1.0.0/generated.json

@@ -0,0 +1 @@
+{"overview":{"a487e881b17105bc9db144d42e2af56494f3fa0e":{"description":"1080p 8-bit YUV To VP9 Video Encode","test_version":"1.0.x","app_version":"Git","unit":"Frames Per Second","samples":50,"timing_samples":49,"hib":1,"first_appeared":1552068371,"last_appeared":1636495768,"run_time_avg":19,"stddev_avg":1.47,"percentiles":[21.63,21.63,43.58,43.58,48.32,48.32,49.12,49.12,49.75,49.75,53.01,53.01,58.52,58.52,63.8,63.8,78.17,78.17,81.51,81.51,87.44,87.44,92.32,92.32,99.86,99.86,103.22,103.22,107.8,107.8,109.33,109.33,151.68,151.68,155.38,155.38,157.56,157.56,158.13,158.13,158.22,158.22,158.73,158.73,158.84,158.84,158.95,158.95,158.99,158.99,159.41,159.41,160.47,160.47,160.71,160.71,160.74,160.74,160.74,161.52,161.56,161.56,162.52,162.52,162.57,162.57,162.69,162.69,162.69,162.69,162.74,162.74,162.9,162.9,163.35,163.35,163.48,163.48,163.88,163.88,163.97,163.97,164.46,164.46,231.12,231.12,268.97,268.97,269,269,275.54,275.54,278,278,334.79,334.79,403.98,403.98,404,404],"run_time_percentiles":[7.99,7.99,7.99,7.99,7.99,8.76,8.76,10,10,11,11,11,11,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,13,13,16,16,16,16,16,16,16,16,17,17,18,18,20,20,20,20,21,21,22,22,24,24,28,28,28,28,29,29,32,32,35,35,37,37,38,38,38,38,42,42,55,55,63,63],"tested_archs":["x86_64"]}}}

ob-cache/test-profiles/git/vpxenc-2.3.0/changelog.json

@@ -0,0 +1 @@
+{"2.3.0":{"last_updated":"1552061773","commit_description":"Git version of test."}}

ob-cache/test-profiles/git/vpxenc-2.3.0/generated.json

@@ -0,0 +1 @@
+{"overview":{"eb95fb2a256d8778cc2325337fbfab5b04610f4c":{"description":"vpxenc VP9 1080p Video Encode","test_version":"2.3.x","app_version":"Git","unit":"Frames Per Second","samples":56,"timing_samples":55,"hib":1,"first_appeared":1552068371,"last_appeared":1635727948,"run_time_avg":735,"stddev_avg":0.79,"percentiles":[6.97,6.97,11.14,11.14,11.85,11.85,12.69,12.69,13.01,13.39,13.39,15.71,15.71,17.98,17.98,18.69,18.69,19.21,19.41,19.41,20.04,20.04,20.49,20.49,20.61,23.07,23.07,23.08,23.08,23.8,23.8,23.81,23.81,23.84,23.89,23.89,23.9,23.9,24.04,24.04,24.26,24.26,25.6,25.62,25.62,26.74,26.74,27.18,27.18,27.25,27.3,27.3,27.32,27.32,27.32,27.32,27.32,27.32,27.36,27.36,27.36,27.39,27.39,27.41,27.41,27.45,27.45,27.45,27.47,27.47,27.5,27.5,27.52,27.52,27.52,27.53,27.53,27.53,27.53,27.53,27.53,27.55,27.55,27.55,27.56,27.56,27.56,27.56,27.57,27.57,27.58,27.58,27.61,27.65,27.65,28.89,28.89,28.95,28.95,29.07],"run_time_percentiles":[482,482,485,485,487,487,496,496,504,504,510,513,513,514,514,516,516,516,516,517,517,517,517,517,517,517,518,518,518,518,518,519,519,519,519,519,519,519,519,520,520,520,521,521,522,522,522,522,523,523,525,526,526,528,528,530,530,536,536,544,603,603,619,619,623,623,627,627,631,631,639,700,700,712,712,717,717,731,731,792,885,885,1053,1053,1075,1075,1098,1098,1128,1128,1182,1411,1411,1441,1441,1675,1675,2188,2188,2859],"tested_archs":["x86_64"]}}}

ob-cache/test-profiles/git/x265-1.1.0/changelog.json

@@ -0,0 +1 @@
+{"1.1.0":{"last_updated":"1552061034","commit_description":"Commit of git\/x265 test using its latest Git mirror snapshot."}}

ob-cache/test-profiles/git/x265-1.1.0/generated.json

@@ -0,0 +1 @@
+{"overview":{"63dbbf942bc9022f095e6888e3513b0c6dc0d8e7":{"description":"H.265 1080p Video Encoding","test_version":"1.1.x","app_version":"Git","unit":"Frames Per Second","samples":97,"timing_samples":95,"hib":1,"first_appeared":1552068371,"last_appeared":1644513045,"run_time_avg":273,"stddev_avg":1.68,"percentiles":[0.77,0.77,0.82,1.36,1.5,1.52,2.33,2.4,2.61,3.16,3.21,3.22,3.25,3.25,4.64,4.9,6.56,6.95,7.01,7.29,7.58,8.29,9.49,10.11,10.24,10.39,10.41,10.63,11.75,12.54,12.74,13.08,13.39,14.13,14.13,14.33,16.08,17.09,17.42,17.7,17.8,19.61,22.78,23.03,23.16,23.86,23.94,24,24.56,25.65,26.98,28.2,30.24,30.69,32.03,32.88,33.43,33.6,34.11,34.64,35.14,35.19,35.49,36,37.71,38.69,38.94,38.94,39.25,40.88,41.18,41.44,41.76,42.19,43.72,43.82,47.44,48.01,48.77,48.82,49.22,49.29,49.39,49.68,50.13,50.54,50.83,51.01,52.01,53.24,53.39,54.73,56.79,59.01,66.89,66.92,67.06,67.31,69.94,79.4],"run_time_percentiles":[23,23,26,27,27,27,31,33,34,34,35,36,36,36,36,37,37,37,37,37,38,38,38,41,41,43,44,45,45,46,46,47,47,48,50,51,51,52,52,54,54,54,56,59,60,64,70,75,78,78,79,92,92,102,106,112,126,128,130,135,138,138,142,144,153,170,173,173,176,178,183,190,218,238,247,257,259,265,274,295,299,299,335,368,378,380,389,520,555,555,561,569,724,1203,1379,1588,2196,2239,2265,2328],"tested_archs":["x86_64","aarch64","armv7l","i686"]}}}

ob-cache/test-profiles/pts/3dmark-1.0.0/changelog.json

@@ -0,0 +1 @@
+{"1.0.0":{"last_updated":"1683925273","commit_description":"Allow automating 3DMark benchmark if you have the necessary licensed files already on your system."}}

ob-cache/test-profiles/pts/3dmark-1.0.0/downloads.xml

@@ -0,0 +1,10 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <FileName>3dmark-attan-extreme-1.1.2.1-workload-bin.zip</FileName>
+      <FileSize>277469377</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/3dmark-1.0.0/install.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+unzip -o 3dmark-attan-extreme-1.1.2.1-workload-bin.zip
+chmod +x bin/linux/x64/workload
+echo "#!/bin/sh
+echo \"{
+    \\\"workload_name\\\": \\\"WildLifeGt1XC\\\",
+    \\\"rendering_resolution\\\": [\$1, \$2],
+    \\\"asset_root\\\": \\\"assets_desktop\\\",
+    \\\"timeline\\\": \\\"timelines_heavy/attan_gt1_heavy_timeline.txt\\\",
+    \\\"fullscreen\\\": true,
+    \\\"debug_api\\\": false,
+    \\\"enable_debug_log\\\": false,
+    \\\"base_log_path\\\": \\\"\\\",
+    \\\"threads\\\": 0,
+    \\\"vsync\\\": false,
+    \\\"loop\\\": false,
+    \\\"loop_count\\\": 0
+}\" > settings/pts.json
+rm -f result.json
+./bin/linux/x64/workload --in=settings/pts.json --out=result.json
+echo \$? > ~/test-exit-status
+cat result.json > \$LOG_FILE" > 3dmark
+chmod +x 3dmark

ob-cache/test-profiles/pts/3dmark-1.0.0/install_windows.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+unzip -o 3dmark-attan-extreme-1.1.2.1-workload-bin.zip
+chmod +x bin/windows/x64/workload.exe
+echo "#!/bin/sh
+echo \"{
+    \\\"workload_name\\\": \\\"WildLifeGt1XC\\\",
+    \\\"rendering_resolution\\\": [\$1, \$2],
+    \\\"asset_root\\\": \\\"assets_desktop\\\",
+    \\\"timeline\\\": \\\"timelines_heavy/attan_gt1_heavy_timeline.txt\\\",
+    \\\"fullscreen\\\": true,
+    \\\"debug_api\\\": false,
+    \\\"enable_debug_log\\\": false,
+    \\\"base_log_path\\\": \\\"\\\",
+    \\\"threads\\\": 0,
+    \\\"vsync\\\": false,
+    \\\"loop\\\": false,
+    \\\"loop_count\\\": 0
+}\" > settings/pts.json
+rm -f result.json
+./bin/windows/x64/workload.exe --in=settings/pts.json --out=result.json
+echo \$? > ~/test-exit-status
+cat result.json > \$LOG_FILE" > 3dmark
+chmod +x 3dmark

ob-cache/test-profiles/pts/3dmark-1.0.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>            "value": #_RESULT_#</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/3dmark-1.0.0/test-definition.xml

@@ -0,0 +1,38 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>3DMark Wild Life Extreme</Title>
+    <AppVersion>1.1.2.1</AppVersion>
+    <Description>This test profile only automates the vendor build of 3DMark with its command-line / JSON support. If you do not have a licensed copy of the necessary 3DMark binaries in your Phoronix Test Suite download cache on your system, this test profile will not do anything and simply fail. You must have already obtained the proper licensed binaries from UL for this test profile to work -- this test profile simply automates the firing of the 3DMark benchmark at your desired resolution and capturing the results within the Phoronix Test Suite while you must already have the necessary 3DMark files on your system.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+    <PreInstallMessage>This test profile only automates the vendor build of 3DMark with its command-line / JSON support. If you do not have a licensed copy of the necessary 3DMark binaries in your Phoronix Test Suite download cache on your system, this test profile will not do anything and simply fail. You must have already obtained the proper licensed binaries from UL for this test profile to work -- this test profile simply automates the firing of the 3DMark benchmark at your desired resolution and capturing the results within the Phoronix Test Suite while you must already have the necessary 3DMark files on your system.</PreInstallMessage>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux, Windows</SupportedPlatforms>
+    <SoftwareType>Game</SoftwareType>
+    <TestType>Graphics</TestType>
+    <License>Restricted</License>
+    <Status>Experimental</Status>
+    <SupportedArchitectures>x86_64</SupportedArchitectures>
+    <RequiresDisplay>TRUE</RequiresDisplay>
+    <EnvironmentSize>598</EnvironmentSize>
+    <ProjectURL>https://www.3dmark.com/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Option>
+      <DisplayName>Resolution</DisplayName>
+      <Identifier>auto-resolution</Identifier>
+      <Menu>
+        <Entry>
+          <Name>$VIDEO_WIDTH x $VIDEO_HEIGHT</Name>
+          <Value>$VIDEO_WIDTH $VIDEO_HEIGHT</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/install.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.2.0
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+	echo 2 > ~/install-exit-status
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/.local/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.8.0m2-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Device Inference Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Inference Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device Training Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Training Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device AI Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device AI Score</ArgumentsDescription>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.0/test-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v9.8.0m2-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AI Benchmark Alpha</Title>
+    <AppVersion>0.1.2</AppVersion>
+    <Description>AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library.</Description>
+    <ResultScale>Score</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>1</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>python</ExternalDependencies>
+    <RequiresInternet>TRUE</RequiresInternet>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>http://ai-benchmark.com/alpha</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/changelog.json

@@ -0,0 +1 @@
+{"1.0.1":{"last_updated":"1605138215","commit_description":"Fix for macOS support."},"1.0.0":{"last_updated":"1594232753","commit_description":"Initial commit of AI Benchmark Alpha."}}

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/generated.json

@@ -0,0 +1 @@
+{"overview":{"6ca1d493839e7acaaacdd66d2ed4166cbcc71052":{"description":"Device Inference Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":890,"timing_samples":877,"hib":1,"first_appeared":1298700000,"last_appeared":1643914610,"run_time_avg":1235,"stddev_avg":0,"percentiles":[105,261,394,419,465,532,546,570,584,628,635,656,663,676,700,730,740,744,755,769,813,836,912,943,956,972,1020,1029,1056,1068,1094,1100,1103,1120,1128,1146,1167,1174,1191,1265,1272,1284,1294,1303,1330,1348,1372,1392,1424,1445,1456,1464,1484,1502,1519,1537,1544,1582,1596,1610,1627,1631,1638,1655,1660,1670,1725,1746,1761,1786,1795,1811,1831,1842,1843,1854,1861,1879,1905,1908,1912,1926,1930,1936,1946,1958,1970,1989,2017,2032,2044,2050,2064,2079,2100,2122,2127,2171,2277,2452],"run_time_percentiles":[509,1000,1016,1033,1040,1049,1058,1061,1063,1067,1071,1072,1075,1078,1079,1083,1085,1088,1092,1096,1102,1103,1104,1108,1113,1118,1119,1121,1125,1129,1131,1132,1137,1141,1144,1150,1151,1153,1156,1159,1163,1166,1167,1168,1171,1176,1179,1184,1187,1188,1190,1193,1198,1204,1209,1212,1216,1218,1223,1226,1227,1230,1234,1236,1241,1249,1257,1265,1275,1279,1280,1292,1298,1301,1309,1315,1321,1331,1348,1359,1366,1376,1389,1395,1417,1426,1431,1440,1461,1477,1484,1503,1512,1518,1528,1536,1548,1604,1648,1754],"tested_archs":["x86_64"]},"5a3d50e3e91837d6f813a215ecd471122e41fbda":{"description":"Device Training Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":890,"timing_samples":877,"hib":1,"first_appeared":1298700000,"last_appeared":1643914610,"run_time_avg":1235,"stddev_avg":0,"percentiles":[138,322,470,518,569,589,598,603,611,623,637,664,685,687,690,709,731,736,772,778,788,809,835,852,857,861,868,881,887,902,910,920,937,965,974,984,989,996,1011,1034,1043,1046,1054,1057,1062,1064,1069,1074,1077,1084,1089,1093,1104,1108,1112,1118,1126,1130,1133,1134,1140,1147,1159,1165,1173,1181,1187,1200,1218,1226,1249,1251,1269,1279,1285,1301,1307,1319,1338,1354,1365,1402,1434,1439,1445,1458,1466,1475,1488,1491,1504,1518,1523,1546,1555,1578,1605,1634,1727,1841],"run_time_percentiles":[509,1000,1016,1033,1040,1049,1058,1061,1063,1067,1071,1072,1075,1078,1079,1083,1085,1088,1092,1096,1102,1103,1104,1108,1113,1118,1119,1121,1125,1129,1131,1132,1137,1141,1144,1150,1151,1153,1156,1159,1163,1166,1167,1168,1171,1176,1179,1184,1187,1188,1190,1193,1198,1204,1209,1212,1216,1218,1223,1226,1227,1230,1234,1236,1241,1249,1257,1265,1275,1279,1280,1292,1298,1301,1309,1315,1321,1331,1348,1359,1366,1376,1389,1395,1417,1426,1431,1440,1461,1477,1484,1503,1512,1518,1528,1536,1548,1604,1648,1754],"tested_archs":["x86_64"]},"18d0ce5cc395e59249172d7c8c5cd0ae47d30c7e":{"description":"Device AI Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":890,"timing_samples":877,"hib":1,"first_appeared":1298700000,"last_appeared":1643914610,"run_time_avg":1235,"stddev_avg":0,"percentiles":[243,575,878,937,1034,1130,1165,1200,1252,1271,1279,1315,1336,1367,1387,1403,1430,1544,1592,1634,1696,1722,1738,1814,1931,1945,1965,1987,2019,2061,2127,2150,2179,2206,2214,2217,2225,2232,2241,2284,2307,2338,2364,2377,2394,2416,2441,2456,2467,2487,2562,2575,2596,2603,2615,2627,2654,2673,2679,2698,2733,2741,2771,2791,2849,2888,2905,2917,2926,2967,3010,3020,3045,3065,3082,3089,3119,3131,3161,3181,3245,3270,3309,3335,3361,3397,3435,3472,3492,3507,3522,3540,3550,3568,3585,3615,3668,3766,3943,4100],"run_time_percentiles":[509,1000,1016,1033,1040,1049,1058,1061,1063,1067,1071,1072,1075,1078,1079,1083,1085,1088,1092,1096,1102,1103,1104,1108,1113,1118,1119,1121,1125,1129,1131,1132,1137,1141,1144,1150,1151,1153,1156,1159,1163,1166,1167,1168,1171,1176,1179,1184,1187,1188,1190,1193,1198,1204,1209,1212,1216,1218,1223,1226,1227,1230,1234,1236,1241,1249,1257,1265,1275,1279,1280,1292,1298,1301,1309,1315,1321,1331,1348,1359,1366,1376,1389,1395,1417,1426,1431,1440,1461,1477,1484,1503,1512,1518,1528,1536,1548,1604,1648,1754],"tested_archs":["x86_64"]}}}

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/install.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.2.0
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+	echo 2 > ~/install-exit-status
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/.local/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/install_macosx.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.2.0
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/Library/Python/3.*/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Device Inference Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Inference Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device Training Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Training Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device AI Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device AI Score</ArgumentsDescription>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.1/test-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AI Benchmark Alpha</Title>
+    <AppVersion>0.1.2</AppVersion>
+    <Description>AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library.</Description>
+    <ResultScale>Score</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>1</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.1</Version>
+    <SupportedPlatforms>Linux, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>python</ExternalDependencies>
+    <RequiresInternet>TRUE</RequiresInternet>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>http://ai-benchmark.com/alpha</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/changelog.json

@@ -0,0 +1 @@
+{"1.0.2":{"last_updated":"1658252311","commit_description":"Update tensorflow pip dependency so it works on newer distributuions."},"1.0.1":{"last_updated":"1605138215","commit_description":"Fix for macOS support."},"1.0.0":{"last_updated":"1594232753","commit_description":"Initial commit of AI Benchmark Alpha."}}

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/generated.json

@@ -0,0 +1 @@
+{"overview":{"6ca1d493839e7acaaacdd66d2ed4166cbcc71052":{"description":"Device Inference Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":1164,"timing_samples":1149,"hib":1,"first_appeared":1594236431,"last_appeared":1705438847,"run_time_avg":1206,"stddev_avg":0,"percentiles":[46,236,370,417,465,511,536,569,581,627,640,660,665,673,681,703,728,735,745,756,769,815,863,889,936,955,986,1020,1029,1041,1060,1077,1090,1102,1120,1128,1162,1173,1183,1245,1268,1284,1294,1306,1321,1352,1373,1392,1412,1428,1454,1464,1481,1502,1533,1569,1580,1605,1619,1638,1654,1660,1711,1735,1768,1788,1811,1825,1837,1843,1854,1861,1879,1894,1906,1912,1921,1928,1936,1952,1968,1983,2007,2026,2042,2054,2079,2100,2125,2159,2232,2288,2324,2343,2508,2857,3341,3995,4700,18696],"run_time_percentiles":[379,560,646,705,734,798,859,865,952,979,983,1007,1020,1033,1035,1042,1048,1056,1060,1063,1071,1074,1075,1079,1083,1085,1087,1091,1093,1098,1103,1105,1108,1114,1119,1121,1125,1129,1131,1133,1136,1141,1146,1147,1150,1154,1157,1159,1163,1167,1170,1174,1177,1182,1187,1188,1197,1201,1206,1211,1217,1223,1226,1230,1235,1241,1253,1259,1269,1279,1287,1295,1299,1307,1315,1322,1329,1338,1350,1360,1370,1382,1390,1403,1418,1428,1434,1447,1468,1478,1496,1512,1520,1528,1537,1582,1616,1658,1762,2199],"tested_archs":["x86_64","aarch64","arm64"]},"18d0ce5cc395e59249172d7c8c5cd0ae47d30c7e":{"description":"Device AI Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":1163,"timing_samples":1149,"hib":1,"first_appeared":1594236431,"last_appeared":1705438847,"run_time_avg":1206,"stddev_avg":0,"percentiles":[74,487,785,930,1034,1118,1152,1200,1256,1272,1288,1318,1349,1372,1391,1424,1512,1546,1598,1636,1689,1722,1737,1765,1825,1910,1943,1965,1982,1996,2053,2111,2150,2206,2215,2221,2236,2277,2303,2340,2374,2401,2445,2464,2486,2527,2562,2586,2612,2621,2654,2673,2686,2714,2734,2771,2792,2855,2898,2908,2925,2932,2998,3018,3045,3072,3113,3124,3153,3176,3205,3251,3279,3333,3352,3390,3411,3439,3472,3495,3518,3545,3572,3611,3672,3730,3781,3886,3992,4176,4189,4521,4778,5532,5574,6548,7522,9310,10315,38200],"run_time_percentiles":[379,560,646,705,734,798,859,865,952,979,983,1007,1020,1033,1035,1042,1048,1056,1060,1063,1071,1074,1075,1079,1083,1085,1087,1091,1093,1098,1103,1105,1108,1114,1119,1121,1125,1129,1131,1133,1136,1141,1146,1147,1150,1154,1157,1159,1163,1167,1170,1174,1177,1182,1187,1188,1197,1201,1206,1211,1217,1223,1226,1230,1235,1241,1253,1259,1269,1279,1287,1295,1299,1307,1315,1322,1329,1338,1350,1360,1370,1382,1390,1403,1418,1428,1434,1447,1468,1478,1496,1512,1520,1528,1537,1582,1616,1658,1762,2199],"tested_archs":["x86_64","aarch64","arm64"]},"5a3d50e3e91837d6f813a215ecd471122e41fbda":{"description":"Device Training Score","test_version":"1.0.x","app_version":"0.1.2","unit":"Score","samples":1162,"timing_samples":1148,"hib":1,"first_appeared":1594236431,"last_appeared":1705438847,"run_time_avg":1206,"stddev_avg":0,"percentiles":[28,260,421,491,548,574,596,603,611,625,638,677,687,691,706,731,751,776,785,790,818,839,853,861,870,883,897,910,924,949,964,974,986,992,1002,1025,1038,1044,1051,1059,1062,1066,1074,1079,1086,1094,1104,1109,1125,1130,1133,1135,1145,1160,1173,1179,1185,1197,1216,1226,1249,1263,1277,1282,1288,1307,1319,1338,1348,1362,1391,1407,1428,1446,1456,1466,1478,1490,1503,1518,1534,1553,1606,1634,1720,1790,1841,1878,2020,2306,2496,2612,2811,3222,3508,3691,4163,5090,5648,19122],"run_time_percentiles":[379,560,646,705,734,798,859,865,952,979,983,1007,1020,1033,1035,1042,1048,1056,1060,1063,1071,1074,1075,1079,1083,1085,1087,1091,1093,1097,1103,1105,1108,1113,1119,1121,1125,1129,1131,1133,1136,1141,1146,1147,1150,1154,1157,1159,1163,1167,1170,1174,1177,1182,1187,1188,1197,1201,1205,1211,1216,1223,1226,1230,1234,1241,1251,1259,1269,1279,1286,1295,1298,1307,1315,1322,1329,1337,1350,1360,1369,1381,1390,1401,1418,1428,1434,1447,1468,1478,1496,1512,1520,1528,1537,1582,1616,1658,1762,2199],"tested_archs":["x86_64","aarch64","arm64"]}}}

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/install.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.9.1
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+	echo 2 > ~/install-exit-status
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/.local/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/install_macosx.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+
+if which pip3 >/dev/null 2>&1 ;
+then
+	echo 0 > ~/install-exit-status
+else
+	echo "ERROR: Python pip3 is not found on the system! This test profile needs Python pip3 to proceed."
+	echo 2 > ~/install-exit-status
+fi
+
+pip3 install --user tensorflow==2.9.1
+pip3 install --user ai-benchmark==0.1.2
+
+if [[ ! -f "$HOME/.local/bin/ai-benchmark" ]]
+then
+	echo "ERROR: AI-Benchmark failed to install on the system!"
+	echo 2 > ~/install-exit-status
+fi
+
+echo "#!/bin/bash
+
+cd \$HOME/Library/Python/3.*/bin/
+python3 ./ai-benchmark > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > ai-benchmark
+chmod +x ai-benchmark

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Device Inference Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Inference Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device Training Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device Training Score</ArgumentsDescription>
+  </ResultsParser>
+  <ResultsParser>
+    <OutputTemplate>Device AI Score: #_RESULT_#</OutputTemplate>
+    <ArgumentsDescription>Device AI Score</ArgumentsDescription>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/ai-benchmark-1.0.2/test-definition.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.8.4-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AI Benchmark Alpha</Title>
+    <AppVersion>0.1.2</AppVersion>
+    <Description>AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library.</Description>
+    <ResultScale>Score</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>1</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.2</Version>
+    <SupportedPlatforms>Linux, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>python</ExternalDependencies>
+    <RequiresInternet>TRUE</RequiresInternet>
+    <EnvironmentSize>10</EnvironmentSize>
+    <ProjectURL>http://ai-benchmark.com/alpha</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.2/changelog.json

@@ -0,0 +1 @@
+{"1.1.2":{"last_updated":"1601214674","commit_description":"Deprecate aio-stress."},"1.1.1":{"last_updated":"1376422733","commit_description":"Fix aio-stress compilation."},"1.1.0":{"last_updated":"1291658419","commit_description":"Initial import into OpenBenchmarking.org"}}

ob-cache/test-profiles/pts/aio-stress-1.1.2/downloads.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v10.0.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://fsbench.filesystems.org/bench/aio-stress.c</URL>
+      <MD5>727e5fc3566837b3ea72f887d048769b</MD5>
+      <FileName>aio-stress.c</FileName>
+      <FileSize>39513</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.2/generated.json

@@ -0,0 +1 @@
+{"overview":{"2829059a4752865dce3ac12c01497347a3b14174":{"description":"Test: Random Write","test_version":"1.1.x","app_version":"0.21","unit":"MB\/s","samples":6579,"timing_samples":706,"hib":1,"first_appeared":1298700000,"last_appeared":1641236010,"run_time_avg":152,"stddev_avg":9.53,"percentiles":[0.11,2.49,4.82,6.73,8.25,9.54,11,11.88,12.74,13.54,14.85,16.37,17.85,18.89,20.68,22.06,24.01,25.66,27.99,29.6,30.92,32.81,35.63,38.97,42.3,47.24,51.23,55.23,58.7,62.22,68.54,75.05,82.1,91.82,99.86,110.81,120.9,133.83,138.95,144.63,157,174.44,188.49,203.5,222.5,243.22,261.31,282.37,311.07,337.95,368.97,394.33,423.73,455.71,493.78,536.16,581.94,632.11,679.94,715.19,766.79,833.63,890.31,932.29,988.19,1029.58,1074.99,1123.23,1183.34,1266.03,1329.48,1382.69,1431.36,1489.73,1546.3,1603.33,1644.21,1697.23,1756.83,1804.22,1871.76,1935,2018.88,2113.86,2160.79,2171.08,2238.74,2318.45,2418,2529.46,2638.1,2772.22,2920.87,3019.25,3156.21,3402.54,3539.65,3625.44,3823.4,4246.73],"run_time_percentiles":[1.7200000000000002,3.34,3.82,3.88,4.22,4.48,4.67,5.09,5.41,5.72,5.99,6.26,6.5,7.43,9.05,9.75,10,11,11,11,12,12,13,14,15,16,17,17,17,17,20,22,23,24,24,26,29,30,31,31,32,33,33,34,34,36,37,40,43,47,49,51,53,55,57,59,61,62,63,65,70,72,73,77,79,81,83,86,88,93,100,105,111,115,122,128,131,138,151,167,193,208,220,234,243,256,271,294,352,372,434,448,489,532,605,746,925,1060,1218,1525],"tested_archs":["x86_64","i686","ppc64le","armv7l","mips64","aarch64","armv6l","ppc64","mips64el"]},"f4f9a92cea869281661787dedbabbf000d252bea":{"description":"Random Write","test_version":"1.1.x","app_version":"0.21","unit":"MB\/s","samples":3176,"timing_samples":259,"hib":1,"first_appeared":1298700000,"last_appeared":1637276912,"run_time_avg":108,"stddev_avg":6.4,"percentiles":[0.53,3.84,6.47,8.7,10.53,12.63,13.94,16.33,18.67,21.8,24.05,25.89,30.67,34.61,38.04,43.31,46.86,50.1,56.82,63.81,71.21,80.99,88.27,101.11,112.37,128.22,144.39,165.35,187,201.27,221.1,250.86,282.55,318.68,365.64,410.34,462.16,521.45,563.55,610.1,657.29,686,730.39,779,811.37,835.95,861.81,900.52,935.48,981.22,1037.28,1058.07,1107.42,1145.43,1191.29,1230.18,1294.87,1351.34,1382.49,1412.52,1453.78,1478.1,1509,1545.11,1578.06,1614.15,1667.61,1700.9,1746.45,1791.89,1827.33,1872.4,1905.46,1930.47,1956.02,1985.8,2023.27,2048,2075.72,2112.58,2154.1,2196.24,2227.69,2264.07,2308.59,2347.98,2379.57,2437.8,2489.19,2519.5,2566.75,2629.01,2693,2777.03,2880.37,3022.41,3133.74,3330.25,3591.66,3898.26],"run_time_percentiles":[2.07,3.54,4.34,4.38,4.53,4.5600000000000005,5.65,5.89,6.17,6.21,6.369999999999999,6.640000000000001,7.039999999999999,7.4399999999999995,7.550000000000001,8.07,8.15,8.4,8.59,8.809999999999999,8.93,9.02,9.16,9.51,9.65,9.69,9.74,9.959999999999999,10,10,11,11,11,12,12,13,13,14,17,18,18,19,20,21,21,24,25,26,27,27,28,30,30,31,31,31,33,35,38,39,42,44,46,48,48,50,53,56,58,61,63,63,69,72,73,79,79,82,90,94,108,117,129,137,143,164,168,193,210,243,249,274,325,396,448,489,546,808,1169,1518],"tested_archs":["x86_64","i686","armv7l","ppc","armv5tel","armv6l","sparc64","aarch64","ppc64","ppc64le"]},"f741d1daf21b100843b4f58c3dfcd4ef82391acd":{"description":"","test_version":"1.1.x","app_version":"0.21","unit":"MB\/s","samples":20,"timing_samples":13,"hib":1,"first_appeared":1519443538,"last_appeared":1558024404,"run_time_avg":112,"stddev_avg":1.68,"percentiles":[874.72,874.72,874.72,874.72,874.72,1145.51,1145.51,1145.51,1145.51,1145.51,1726.03,1726.03,1726.03,1726.03,1726.03,1779.25,1779.25,1779.25,1779.25,1779.25,2137.06,2137.06,2137.06,2137.06,2137.06,2168.91,2168.91,2168.91,2168.91,2168.91,2417.96,2417.96,2417.96,2417.96,2417.96,3259.43,3259.43,3259.43,3259.43,3259.43,3529.37,3529.37,3529.37,3529.37,3529.37,4029.93,4029.93,4029.93,4029.93,4029.93,4053,4053,4053,4053,4053,4254,4254,4254,4254,4254,4315.17,4315.17,4315.17,4315.17,4315.17,4342,4342,4342,4342,4342,4454.4,4454.4,4454.4,4454.4,4454.4,4502.18,4502.18,4502.18,4502.18,4502.18,4511.63,4511.63,4511.63,4511.63,4511.63,4516.85,4516.85,4516.85,4516.85,4516.85,4675.08,4675.08,4675.08,4675.08,4675.08,5109.61,5109.61,5109.61,5109.61,5109.61],"run_time_percentiles":[74,74,74,74,74,74,74,74,76,76,76,76,76,76,76,76,81,81,81,81,81,81,81,81,104,104,104,104,104,104,104,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,105,115,115,115,115,115,115,115,127,127,127,127,127,127,127,127,129,129,129,129,129,129,129,129,130,130,130,130,130,130,130,134,134,134,134,134,134,134,134,136,136,136,136,136,136,136,136,137,137,137,137,137,137,137],"tested_archs":["x86_64"]}}}