Phoronix Test Suite 8.8 Milestone 2 prep

phodevi: Check more specific ipmitool cpu sensor names first

AUTHORS

@@ -1,4 +1,4 @@
-Copyright 2008 - 2018 by Phoronix Media.
+Copyright 2008 - 2019 by Phoronix Media.
 
 Lead Architects:
 

ChangeLog

@@ -1,5 +1,210 @@
 PHORONIX TEST SUITE CHANGE-LOG
 
+Phoronix Test Suite 8.8.0 Milestone 2
+27 March 2019
+
+pts-core: On new installs set 3.00% as deviation for increasing dynamic run count rather than 3.50%
+pts-core: Add FORCE_MIN_TIMES_TO_RUN_CUTOFF= environment variable to toggle cutoff for FORCE_MIN_TIMES_TO_RUN behavior
+pts-core: Add FORCE_TIMES_TO_RUN_MULTIPLE
+pts-core: Add strict-benchmark option for enabling various options to ensure stricter behavior about the number of times to run each benchmark
+pts-core: Add cpu-physical-threads / cpu-threads / auto-executable auto test run options
+phodevi: CPU-therm reporting via sysfs for CPU core temp
+phodevi: GPU-therm reporting via sysfs for GPU core temp
+
+Phoronix Test Suite 8.8.0 Milestone 1
+16 March 2019
+
+pts-core: Allow FORCE_TIMES_TO_RUN to override debug-run/debug-benchmark single run behavior
+pts-core: Various fallbacks/workarounds for those wanting to (unofficially) use PTS on Windows 8.1
+pts-core: Other small tweaks and optimizations
+pts-core: Fixes an odd scenario where on Windows after 2 runs, further runs of that test profile for current run could fail in select (random) instances
+pts-core: Write out system notes for each system when generating PDF result output
+pts-core: CSV output improvements
+pts-core: Add result-file-raw-to-csv for dumping the raw/individual benchmark result run data
+pts_Graph: Condensing improvements for large graphs
+pts_Graph: Various formatting improvements
+phodevi: Intel GPU reporting improvements
+phodevi: BIOS version reporting support extended to Windows
+phodevi: ipmitool DMCI power sensor reporting for at least some POWER servers
+
+Phoronix Test Suite 8.6.1-Spydeberg
+21 February 2019
+
+pts-core: Dynamically honoring the CLI terminal width (no longer cache the value on Linux/BSD/macOS)
+pts-core: BSD external dependency updates
+ob_auto_compare: Minor formatting updates
+
+Phoronix Test Suite 8.6.0-Spydeberg
+5 February 2019
+
+pts-core: Other small tweaks and fixes
+pts-core: Fix for "/installed" virtual suite not including the proprietary/commercial tests that are installed
+ob_auto_compare: Allow showing more results in the embedded percentile graphs
+
+Phoronix Test Suite 8.6.0 Milestone 3
+27 January 2019
+
+pts-core: Better handling on Windows when there are space(s) in the paths
+pts-core: Add __event_post_run_stats module hook
+pts-core: Inline display of result distributions from OpenBenchmarking.org on applicable results
+pts-core: Don't try to run tests from result file(s) or suite(s) that were not installed
+pts-core: More accurate time reporting when result files are passed to `phoronix-test-suite estimate-run-time`
+phodevi: Better reporting of NVMe drive strings on BSDs
+
+Phoronix Test Suite 8.6.0 Milestone 2
+19 January 2019
+
+pts-core: Update Mac OS X to macOS references
+pts-core: Add result-file-stats sub-command to show more test result statistics
+pts-core: Show geometric mean and harmonic mean data at end of benchmarking process when relevant as part of post-run stats
+pts-core: Improved formatting of file download text user interface for downloading of dependencies on Microsoft Windows
+pts-core: Fix Loop 'Test i of n' Reporting
+pts-core: A few warning fixes for PHP 7.3+
+pts-core: External dependency handling support for OpenMandriva Lx 4.0 (transitioned from urpmi to DNF)
+phodevi: Report Windows Subsystem for Linux (WSL) to the system layer string
+
+Phoronix Test Suite 8.6.0 Milestone 1
+7 January 2019
+
+pts-core: No longer clear test profile identifiers on secondary saved results
+pts-core: Add compare-results-to-baseline sub-command for comparing one result file (baseline) against a set of results for text look at relative perf
+pts-core: Minor fixes
+pts-core: Show wins and losses summary after testing on large result files
+pts-core: On two-way result files, after test completion show comparison of current result against previous result (baseline)
+pts-core: Add ShowPostRunStatistics user configuration XML option to toggle whether various test statistics are printed at the end of testing
+pts-core: Add recently-added-tests sub-command to show newly added test profiles / alternative to openbenchmarking-changes/recently-updated-tests
+pts-core: Allow outputting line graphs to terminal interface as basic box plot graphs
+phodevi: Improve Linux multiple network device reporting
+phodevi: LXQt desktop version reporting
+phodevi: Update for Microsoft Windows Server 2019
+system_monitor: Allow outputting system monitor graphs to terminal interface inline
+
+Phoronix Test Suite 8.4.1-Skiptvet
+9 December 2018
+
+pts-core: Workaround for RHEL 7.6 PHP having buggy CURL that is causing seg faults
+pts-core: Ensure "." doesn't get set in PATH when launching phoronix-test-suite
+pts-core: Allow PRESET_OPTIONS env var to be used when in batch mode
+pts-core: FreeBSD updates
+pts-core: Add PRESET_OPTIONS_VALUES environment variable that is similar to PRESET_OPTIONS but for explicit values
+phodevi: Improved Radeon AMDGPU frequency detection/reporting
+phodevi: Add support for reading GPU usage sensor with AMDGPU gpu_busy_percent sysfs interface
+phodevi: Updated gpu_voltage sensor for AMDGPU driver
+phodevi: Updated gpu_power sensor for AMDGPU driver
+phodevi: Updated gpu_fanspeed sensor for AMDGPU driver
+phodevi: Updated gpu_freq sensor for AMDGPU driver
+phodevi: Improved physical CPU core detection on DragonFlyBSD 5.4
+phodevi: Improved Spectre detection on DragonFlyBSD 5.4
+phodevi: Allow using NUM_CPU_CORES to override the number of exposed CPU threads/cores to tests (alternative to PTS_NPROC / NUMBER_OF_PROCESSORS)
+phodevi: Improved multi-disk reporting for non-Linux operating systems
+phodevi: Minor GPU string reporting improvement
+
+Phoronix Test Suite 8.4.0-Skiptvet
+26 November 2018
+
+phodevi: Various minor tweaks
+
+Phoronix Test Suite 8.4.0 Milestone 3
+20 November 2018
+
+pts-core: Various external dependency updates
+pts-core: Expose NUM_CPU_PHYSICAL_CORES to test profiles as alternative to NUM_CPU_CORES (logical count)
+pts-core: Fix possible situation of openbenchmarking-refresh not updating when IndexCacheTTL is set to 0
+pts-core: Allow branded colors to work in the terminal/CLI text graphs
+pts-core: Continued POWER9 support improvements
+pts-core: Add RemoveInstallDirectoryOnReinstall to toggle test re-installation behavior
+pts_Graph: Fix for sometimes tick marker text overlapping
+phodevi: Improved physical CPU core detection for architectures like POWER
+phodevi: Improved CPU cache size detection
+
+Phoronix Test Suite 8.4.0 Milestone 2
+5 November 2018
+
+pts-core: Allow passing PTS environment variables as arguments to PTS and then just have PTS set them in the environment
+pts-core: Add dry-run test option
+pts-core: Various result parser additions to help parsing of CSV frame-time data for more test profiles (games)
+phodevi: Vulkan driver reporting fix
+
+Phoronix Test Suite 8.4.0 Milestone 1
+8 October 2018
+
+phodevi: Report vRAM size on GPU model string in GB when over 1GB
+pts-core: At run-time also report if test is deprecated/broken
+pts-core: Add "node-npm" as an external dependency for Node.js + NPM
+phodevi: Drop information reporting support for the long ago Catalyst/fglrx driver stack that hasn't been supported/maintained in many years
+
+Phoronix Test Suite 8.2.0-Rakkestad
+11 September 2018
+
+pts-core: Add RequiresDisplay test profile property to indicate if the test profile explicitly requires a monitor / display server to work
+pts-core: Add RequiresNetwork test profile property to indicate if the test profile needs a working network connection
+pts-core: Add RequiresInternet test profile property to indicate if the test profile needs a working Internet connection
+pts-core: Don't install tests where the current run-time checks will fail
+phodevi: Improve CPU thread count detection when running under LXC
+phodevi: More robust DMI decode ignore value handling
+phodevi: Add extra fallback detection for LXC in system-layer
+
+Phoronix Test Suite 8.2.0 Milestone 4
+7 September 2018
+
+pts-core: Fix behavior of "Would you like to stop and install these tests now"
+pts-core: Add reinstall / re-install aliases to force-install
+ob_auto_compare: Default first to trying to do an automated local result comparison
+phodevi: Amlogic SoC detection based on meson_cpufreq driver
+phodevi: Rockchip SoC fallback detection
+phodevi: Embedded ARM Cortex / ARM part model detection
+phodevi: Expanded sensor coverage using Intel RAPL
+
+Phoronix Test Suite 8.2.0 Milestone 3
+22 August 2018
+
+pts-core: Minor updates to logic for determining how many times to dynamically increase run count on high variance results
+pts-core: Show deprecated tests too when running list-all-tests
+pts-core: Add DropNoisyResults option if wishing to not save results that have too high of deviation between runs
+pts-core: Add PTS_DOWNLOAD_CACHE_OVERRIDE environment variable option on make-download-cache
+pts-core: Add PTS_USER_PATH_OVERRIDE environment variable to override PTS_USER_PATH on initialization
+pts-core: Add PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT environment variable for make-download-cache
+pts-core: Add list-cached-tests option
+
+Phoronix Test Suite 8.2.0 Milestone 2
+15 August 2018
+
+pts-core: Add DivideResultDivisor to result parser
+pts-core: Add libevent as possible external dependency
+pts-core: Alert the user when the test install directory is on a file-system mounted with 'noexec'
+pts-core: Optional support with PHP 7.3+ to also obtain network info using net_get_interfaces()
+pts-core: Fix possible alignment issue with value/spacer on outputting text-based result graphs
+pts-core: When Internet is not available and running list-tests, only show tests that are available via caches unless `list-all-tests` is run
+pts-core: Rework `list-recommended-tests` to make use of download count information exposed via OB index repository data rather than needing to make fresh web query
+pts-core: Show "popular tests" in the invalid command helper
+pts-core: Allow stress-run to be triggered from interactive mode
+pts-core: Allow comparing dynamic in-line result comparisons against local results when no Internet results matching or Internet not available
+phodevi: Reporting of L1TF / Level 1 Terminal Fault vulnerability
+pts_Graph: Tweaks to text sizing on vertical bar graphs
+
+Phoronix Test Suite 8.2.0 Milestone 1
+21 July 2018
+
+pts-core: Add readline-based autocompletion support for text-based menu selection lists
+pts-core: Add readline-based autocompletion support for the result file save name for trying to match existing result file identifiers
+pts-core: Add autocompletion support for result file identifier based on installed software/hardware
+pts-core: Various Alpine Linux external dependency updates
+pts-core: Various improvements to the Shell sub-command functionality, including auto-completion support
+phoromatic: From rootadmin allow the rootadmin to override/reset a user account password
+phodevi: Windows battery reporting discharge rate support
+phodevi: Improved multi CPU socket reporting on Windows
+phodevi: Various hardware/software detection improvements when running within Docker/containers
+pgo: New module to provide for automated compiler PGO (Profile Guided Optimizations) testing on GCC/Clang
+
+Phoronix Test Suite 8.0.1-Aremark
+26 June 2018
+
+pts-core: Add "wine" external dependency support
+pts-core: Fix skip-test behavior
+pts-core: Web browser handling improvements on Windows, other Windows updates
+phoromatic: Various minor fixes and enhancements
+phodevi: Add USE_WINE environment variable check, point it towards the wine binary you wish to test
+
 Phoronix Test Suite 8.0.0-Aremark
 5 June 2018
 

README.md

@@ -1,13 +1,13 @@
-# Phoronix Test Suite 8.0.0
-http://www.phoronix-test-suite.com/
+# Phoronix Test Suite 8.8.0m2
+https://www.phoronix-test-suite.com/
 
 The **Phoronix Test Suite** is the most comprehensive testing and benchmarking
-platform available for Linux, Solaris, OS X, and BSD operating systems. The
-Phoronix Test Suite allows for carrying out tests in a fully automated manner
-from test installation to execution and reporting. All tests are meant to be
-easily reproducible, easy-to-use, and support fully automated execution. The
-Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed
-by Phoronix Media in cooperation with partners.
+platform available for Linux, Solaris, macOS, Windows, and BSD operating
+systems. The Phoronix Test Suite allows for carrying out tests in a fully
+automated manner from test installation to execution and reporting. All tests
+are meant to be easily reproducible, easy-to-use, and support fully automated
+execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license
+and is developed by Phoronix Media in cooperation with partners.
 
 The Phoronix Test Suite itself is an open-source framework for conducting
 automated tests along with reporting of test results, detection of installed
@@ -50,13 +50,12 @@ the Phoronix Test Suite website.
 
 ## Installation & Setup
 
-The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and
-Windows systems. However, the most full-featured and well supported operating
-system for conducting the tests is Linux with some non-basic functionality not
-being available under all platforms. The Phoronix Test Suite software/framework
-is compatible with all major CPU architectures (e.g. i686, x86_64, ARM,
-PowerPC), but not all of the test profiles/suites are compatible with all
-architectures.
+The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows
+systems. However, the most full-featured and well supported operating system for
+conducting the tests is Linux with some non-basic functionality not being
+available under all platforms. The Phoronix Test Suite software/framework is
+compatible with all major CPU architectures (e.g. i686, x86_64, ARM, PowerPC),
+but not all of the test profiles/suites are compatible with all architectures.
 
 The Phoronix Test Suite can be installed for system-wide usage or run locally
 without installation from the extracted tar.gz/zip package. The only hard

deploy/deb-package/build-package-deb.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2016, Phoronix Media
-	Copyright (C) 2008 - 2016, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by

documentation/TODO

@@ -13,55 +13,18 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - Optimus / DRI_PRIME=1 / xrandr --listproviders better GPU detection
 - NVIDIA make use of UsedDedicatedGPUMemory as new GPU sensor
 - Clean up all the WebSocket code and make it fit to spec
-- Incremental result uploads support
+- Incremental/streaming result uploads support
 - Check with Phodevi's parsers what other tools support JSON outputs - e.g. http://karelzak.blogspot.com/2015/06/json-output-for-basic-tools.html for cleaner parsing
-
-- Figure out why TestClientVersion in result file XML is missing after adding multiple systems, the previous ones are then blank
-- Restore the ReferenceID support in the results XML
 - Improve titles on tables like in: http://openbenchmarking.org/result/1603172-GA-AMDTONGA973
 
 ### PTS-GRAPH ###
 
-- Rotated text alignment for some PNG graphixs is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
+- Rotated text alignment for some PNG graphics is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
 
 ### OPENBENCHMARKING.ORG ###
 
 - Shorten long change-logs on test profile pages - e.g. http://openbenchmarking.org/test/pts/hpcc
 
-### POSSIBLE NEW TEST PROFILES ###
-
-- Hadoop Benchmarks / TeraSort
-- LATT Latency Tests [git://git.kernel.dk/latt.git]
-- QMC Beaver [http://qmcbeaver.sourceforge.net/]
-- Some potential interesting GPGPU tests [http://www.bealto.com/gpu-benchmarks.html]
-- WebGL / Chrome Test [http://phoronix.com/forums/showthread.php?73167-Some-Benchmarks-to-add-to-PTS-WebGL-and-Google-Chrome]
-- GtkPerf improvements [http://phoronix.com/forums/showthread.php?73047-gtkperf-I-m-maintaining-feature-requests-new-tests]
-- SIPP [http://sipp.sourceforge.net/]
-- oclGaussCrack [https://hashcat.net/oclGaussCrack/]
-- http://hashcat.net/oclhashcat/
-- Handbrake [http://handbrake.fr/]
-- libMicro [https://github.com/portante/libMicro]
-- http://markjstock.org/pages/rad_bench.html
-- https://code.google.com/p/pyrit/
-- https://github.com/acangiano/ruby-benchmark-suite
-- http://www.nwchem-sw.org/index.php/Compiling_NWChem
-- http://sourceforge.net/projects/ffsb/
-- http://www.ratgpu.com/
-- http://panthema.net/2013/pmbw/
-- http://parsec.cs.princeton.edu/parsec3-doc.htm
-- http://www.capsl.udel.edu/splash/Download.html
-- stephentu/silo
-- https://code.google.com/p/smhasher/wiki/SMHasher
-- http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fasta / http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gcc&id=1#sourcecode
-- https://openfoamwiki.net/index.php/Benchmarks_standard_v1
-- Cassandra 1.2 stress - http://docs.datastax.com/en/cassandra/1.2/cassandra/tools/toolsCStress_t.html
-- http://gcc.opensuse.org/c++bench-frescobaldi/
-- https://gooseberry.blender.org/gooseberry-production-benchmark-file/
-- https://github.com/gormanm/mmtests
-- https://github.com/soumith/convnet-benchmarks
-- http://apt.cs.manchester.ac.uk/projects/PAMELA/tools/SLAMBench/
-- https://haasn.xyz/posts/2017-10-05-how-to-benchmark-mpvs-raw-throughput.html
-
 ### WEB SOCKET / HTML5 USER INTERFACE WORK ###
 
 - Implement WebSocket security - add the password prompts from ~/.phoronix-test-suite/user-config.xml, etc
@@ -75,4 +38,4 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - Implement the complementary WebSocket communication to HTTP pulls
 - Ensure sane validating of more user input / resource uploads
 - Hook up TriggerTarget for allowing event.php to optionally specify a sub-set of systems/groups to target for a given trigger
-- Clean up the code and modularize more of the code into functions
+- Clean up the code and modularize more of the code

documentation/man-pages/phoronix-test-suite.1

@@ -1,4 +1,4 @@
-.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "8.0.0"
+.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "8.8.0m2"
 .SH NAME
 phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.
 .SH SYNOPSIS
@@ -6,12 +6,31 @@ phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source plat
 .br
 .B phoronix-test-suite benchmark [test | suite]
 .SH DESCRIPTION
-The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
+The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
 .SH OPTIONS
 .TP
+.SH SYSTEM
+.B diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B interactive
+A simple text-driven interactive interface to the Phoronix Test Suite.
+.TP
+.B php-conf
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B shell
+A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
+.TP
+.B system-info
+Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
+.TP
+.B system-sensors
+Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
+.TP
 .SH TEST INSTALLATION
 .B force-install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
-This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.
+This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.
 .TP
 .B install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
@@ -50,6 +69,9 @@ This option can be used if you wish to run all of the tests found in a supplied
 .B stress-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.
 .TP
+.B strict-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.
+.TP
 .SH BATCH TESTING
 .B batch-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments are equivalent to the benchmark option, but the process will be run in the Phoronix Test Suite batch mode.
@@ -69,6 +91,9 @@ This option will install the selected test(s) (if needed) and will proceed to ru
 .B default-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.
 .TP
+.B dry-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.
+.TP
 .B internal-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.
 .TP
@@ -97,6 +122,9 @@ This option will list the OpenBenchmarking.org repositories currently linked to
 .B openbenchmarking-uploads
 This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.
 .TP
+.B recently-added-tests
+This option will list the most recently added (newest) test profiles.
+.TP
 .B upload-result [Test Result]
 This option is used for uploading a test result to OpenBenchmarking.org.
 .TP
@@ -106,38 +134,25 @@ This option can be used for uploading a test profile to your account on OpenBenc
 .B upload-test-suite
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 .TP
-.SH SYSTEM
-.B diagnostics
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
-.B interactive
-A simple text-driven interactive interface to the Phoronix Test Suite.
-.TP
-.B php-conf
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
-.B shell
-A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
-.TP
-.B system-info
-Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
-.TP
-.B system-sensors
-Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
-.TP
 .SH INFORMATION
 .B info [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
 .TP
+.B list-all-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
+.TP
 .B list-available-suites
 This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
 .TP
 .B list-available-tests
-This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
 .TP
 .B list-available-virtual-suites
 This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
 .TP
+.B list-cached-tests
+This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.
+.TP
 .B list-installed-dependencies
 This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.
 .TP
@@ -212,6 +227,9 @@ This option will generate a candlestick graph showing the distribution of result
 .B auto-sort-result-file [Test Result]
 This option is used if you wish to automatically attempt to sort the results by their result identifier string.
 .TP
+.B compare-results-to-baseline [Test Result] [Test Result]
+This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.
+.TP
 .B edit-result-file [Test Result]
 This option is used if you wish to edit the title and description of an existing result file.
 .TP
@@ -239,6 +257,12 @@ This option is used if you wish to change the name of the saved name of a result
 .B reorder-result-file [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 .TP
+.B result-file-raw-to-csv [Test Result]
+This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
+.TP
+.B result-file-stats [Test Result]
+This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
+.TP
 .B result-file-to-csv [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 .TP
@@ -254,7 +278,7 @@ This option will read a saved test results file and output the system hardware a
 .B show-result [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 .TP
-.B winners-and-losers [Test Result]
+.B wins-and-losses [Test Result]
 This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
 .TP
 .SH OTHER
@@ -311,10 +335,6 @@ This option can be used for resetting the Phoronix Test Suite user configuration
 .B user-config-set
 This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
 .TP
-.SH WEB / GUI SUPPORT
-.B gui
-Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.
-.TP
 .SH PHOROMATIC
 .B n
 _
@@ -322,15 +342,15 @@ _
 .SH SEE ALSO
 .B Websites:
 .br
-http://www.phoronix-test-suite.com/
+https://www.phoronix-test-suite.com/
 .br
-http://commercial.phoronix-test-suite.com/
+https://commercial.phoronix-test-suite.com/
 .br
-http://www.openbenchmarking.org/
+https://www.openbenchmarking.org/
 .br
-http://www.phoronix.com/
+https://www.phoronix.com/
 .br
-http://www.phoronix.com/forums/
+https://www.phoronix.com/forums/
 .SH AUTHORS
-Copyright 2008 - 2018 by Phoronix Media, Michael Larabel.
+Copyright 2008 - 2019 by Phoronix Media, Michael Larabel.
 .TP

documentation/phoronix-test-suite.md

@@ -2,7 +2,7 @@
 # Getting Started
 
 ## Overview
-The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.
+The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.
 The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via [OpenBenchmarking.org](http://openbenchmarking.org/) integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.
 Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as *phoronix-test-suite benchmark c-ray* , which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.
 
@@ -20,9 +20,23 @@ The following options are currently supported by the Phoronix Test Suite client.
 
 ---
 
+## System
+#### diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+#### interactive
+A simple text-driven interactive interface to the Phoronix Test Suite.
+#### php-conf
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+#### shell
+A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
+#### system-info
+Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
+#### system-sensors
+Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
+
 ## Test Installation
 #### force-install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
-This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.
+This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.
 #### install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
 #### install-dependencies  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
@@ -49,6 +63,8 @@ This option will query OpenBenchmarking.org to run random benchmarks and result
 This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.
 #### stress-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.
+#### strict-benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.
 
 ## Batch Testing
 #### batch-benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
@@ -63,6 +79,8 @@ This option is used to configure the batch mode options for the Phoronix Test Su
 This option will install the selected test(s) (if needed) and will proceed to run the test(s) in the defaults mode. This option is equivalent to running phoronix-test-suite with the install option followed by the default-run option.
 #### default-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.
+#### dry-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.
 #### internal-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.
 
@@ -83,6 +101,8 @@ This option is used for refreshing the stored OpenBenchmarking.org repostory inf
 This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.
 #### openbenchmarking-uploads
 This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.
+#### recently-added-tests
+This option will list the most recently added (newest) test profiles.
 #### upload-result  [Test Result]
 This option is used for uploading a test result to OpenBenchmarking.org.
 #### upload-test-profile
@@ -90,29 +110,19 @@ This option can be used for uploading a test profile to your account on OpenBenc
 #### upload-test-suite
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 
-## System
-#### diagnostics
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-#### interactive
-A simple text-driven interactive interface to the Phoronix Test Suite.
-#### php-conf
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-#### shell
-A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
-#### system-info
-Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
-#### system-sensors
-Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
-
 ## Information
 #### info  [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
+#### list-all-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
 #### list-available-suites
 This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
 #### list-available-tests
-This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
 #### list-available-virtual-suites
 This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
+#### list-cached-tests
+This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.
 #### list-installed-dependencies
 This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.
 #### list-installed-suites
@@ -165,6 +175,8 @@ This option can be used for validating a Phoronix Test Suite test suite as being
 This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
 #### auto-sort-result-file  [Test Result]
 This option is used if you wish to automatically attempt to sort the results by their result identifier string.
+#### compare-results-to-baseline  [Test Result] [Test Result]
+This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.
 #### edit-result-file  [Test Result]
 This option is used if you wish to edit the title and description of an existing result file.
 #### extract-from-result-file  [Test Result]
@@ -183,6 +195,10 @@ This option is used if you wish to change the name of the identifier in a test r
 This option is used if you wish to change the name of the saved name of a result file.
 #### reorder-result-file  [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
+#### result-file-raw-to-csv  [Test Result]
+This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
+#### result-file-stats  [Test Result]
+This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
 #### result-file-to-csv  [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 #### result-file-to-json  [Test Result]
@@ -193,7 +209,7 @@ This option will read a saved test results file and output the system hardware a
 This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.
 #### show-result  [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
-#### winners-and-losers  [Test Result]
+#### wins-and-losses  [Test Result]
 This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
 
 ## Other
@@ -236,10 +252,6 @@ This option can be used for resetting the Phoronix Test Suite user configuration
 #### user-config-set
 This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
 
-## Web / GUI Support
-#### gui
-Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.
-
 ## Phoromatic
 #### start-phoromatic-server
 Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.
@@ -297,6 +309,10 @@ This module alerts the user if the system configuration may not be the right one
 phoronix-test-suite perf_tips.show
 This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.
 
+### Benchmarking Compiler PGO Impact
+This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.
+phoronix-test-suite pgo.benchmark
+
 ### Phoromatic Client
 The Phoromatic client is used for connecting to a Phoromatic server (Phoromatic.com or a locally run server) to facilitate the automatic running of tests, generally across multiple test nodes in a routine manner. For more details visit http://www.phoromatic.com/. This module is intended to be used with Phoronix Test Suite 5.2+ clients and servers.
 phoronix-test-suite phoromatic.connect
@@ -337,6 +353,10 @@ This module utilizes the following environmental variables: HALT_SCREENSAVER.
 ### Update Checker
 This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.
 
+### Utilize Wine On Linux Benchmarking
+This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.
+This module utilizes the following environmental variables: USE_WINE.
+
 ### System Event Watchdog
 This module has support for stopping/interrupting tests if various system issues occur, like a temperature sensor exceeds a defined threshold.
 This module utilizes the following environmental variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD.
@@ -344,7 +364,7 @@ This module utilizes the following environmental variables: WATCHDOG_SENSOR, WAT
 # Installation Instructions
 
 ## Setup Overview
-The Phoronix Test Suite supports Linux, Apple Mac OS X, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or Mac OS X, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.
+The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.
 Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.
 Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.
 
@@ -393,11 +413,9 @@ Running *install-sh* from the root directory of the Phoronix Test Suite will ins
 
 ### Debian/Ubuntu Installation
 Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the *phoronix-test-suite* executable in */usr/bin/* , the Phoronix Test Suite files in */usr/share/phoronix-test-suite/* , and the documentation in */usr/share/doc/phoronix-test-suite/* .
-A phoronix-test-suite package can also be found in the official Ubuntu universe repository beginning with the Ubuntu 9.04 (Jaunty) release, but it may be out of date compared to the latest upstream version.
-On Ubuntu 13.04 and newer if you experience an issue of system sensor monitoring not working, other multi-process tasks, or related issues reported as PHP PCNTL missing, the pcntl functions need to be removed from the *disable_functions* line within the */etc/php5/cli/php.ini* file. This may also affect other Linux distribution support too if the vendor disables access to the PCNTL functions.
 
 ### Fedora / Red Hat Installation
-The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a *phoronix-test-suite* package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. At this time, however, the Fedora-supplied package is vastly out of date.
+The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a *phoronix-test-suite* package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.
 
 ### Gentoo Installation
 Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the [Gentoo BugZilla](http://bugs.gentoo.org/show_bug.cgi?id=216656) .
@@ -412,8 +430,8 @@ The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris
 The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.
 For DragonFlyBSD, it's a matter of just using *pkg_radd* with php5, php5-json, php5-dom, and php5-zip or *pkgsrc* . With DragonFlyBSD using dports, the command is *pkg install php5 php5-dom php5-zip php5-json php5-simplexml* . Under FreeBSD/PC-BSD, PHP can be obtained from the *php5* and *php5-extensions* Ports collection.
 
-### Mac OS X Installation
-The Phoronix Test Suite is fully supported on Apple's Mac OS X operating system as of Mac OS X 10.5 with improved support under Mac OS X 10.6. PHP ships with Mac OS X by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.
+### MacOS Installation
+The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.
 
 ### Windows Installation
 The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.
@@ -431,7 +449,7 @@ Below are a list of the operating systems that currently have external dependenc
 ---
 
 
-Alpine LinuxAmazonAngstromArch LinuxCentOSClearOSClearOS Core ServerDebianDragonFlyBSDFedoraFluxbuntuGNU KFreeBSDGentooGoobuntuHPJoli CloudLinaroLinux Embedded Development EnvironmentLinux MintMEPISMac OS XMacPortsMageiaMandrivaMeeGoMicrosoft WindowsMidnightBSDMoblinMythbuntuNetBSDNexenta CoreOLPCOpenIndianaOpenMandrivaOpenSolarisOpenSuSEOptwareOracle ServerPCLinuxOSPTS Desktop LivePTS Linux LivePalmPardus LinuxRed Hat EnterpriseRed Hat Enterprise ServerSUSESUSE LinuxScientificScientificSLSolusSolus LinuxTermuxUbuntuVoid LinuxZenwalkgNewSensemacOS Brew
+Alpine LinuxAmazonAngstromArch LinuxCentOSClearOSClearOS Core ServerDebianDragonFlyBSDFedoraFluxbuntuGNU KFreeBSDGentooGoobuntuHPJoli CloudLinaroLinux Embedded Development EnvironmentLinux MintMEPISMac OS XMacPortsMageiaMandrivaMeeGoMicrosoft WindowsMidnightBSDMoblinMythbuntuNetBSDNexenta CoreOLPCOpenIndianaOpenMandrivaOpenMandrivaLinuxOpenSolarisOpenSuSEOptwareOracle ServerPCLinuxOSPTS Desktop LivePTS Linux LivePalmPardus LinuxRed Hat EnterpriseRed Hat Enterprise ServerSUSESUSE LinuxScientificScientificSLSolusSolus LinuxTermuxUbuntuVoid LinuxZenwalkgNewSensemacOS Brew
 # Configuration
 
 ## User Files & Folders
@@ -487,8 +505,12 @@ If you want to ensure that the time for a given Phoronix Test Suite process does
 When running any test(s), if you would like the test(s) to continue running for a number of times, this variable can be used. The value should be the number of times to loop the testing process before ending.
 **FORCE_TIMES_TO_RUN**
 If you wish to override the number of times to run each test -- rather than the Phoronix Test Suite using the number of times specified in each test profile -- this variable can be used.
+**FORCE_TIMES_TO_RUN_MULTIPLE**
+This option allows specifying a multiple for increasing the number of times a test will run based upon the original TimesToRun value specified in the test definition. This allows for increasing the expected times to run based on a multiple of that default rather than a static value.
 **FORCE_MIN_TIMES_TO_RUN**
 This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.
+**FORCE_MIN_TIMES_TO_RUN_CUTOFF**
+When used in conjunction with FORCE_MIN_TIMES_TO_RUN, the override value will only be applied to test profiles where its average run-time length (in minutes) is less than the value specified by FORCE_MIN_TIMES_TO_RUN_CUTOFF.
 **NO_FILE_HASH_CHECKS**
 To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as *NO_MD5_CHECKS* , which is still honored but was changed to *NO_FILE_HASH_CHECKS* to reflect other kind of file hash sum checks.
 **NO_HTTPS**
@@ -609,14 +631,14 @@ This is a collection of test profiles found within the specified OpenBenchmarkin
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
 #### Cuda Tests  pts/cuda
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
+#### Openmp Tests  pts/openmp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.
 #### Python Tests  pts/python
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.
 #### Go Tests  pts/go
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.
 #### Mpi Tests  pts/mpi
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing mpi.
-#### Openmp Tests  pts/openmp
-This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.
 #### Vdpau Tests  pts/vdpau
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing vdpau.
 #### Video Tests  pts/video
@@ -671,6 +693,62 @@ This is a collection of test profiles found within the specified OpenBenchmarkin
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.
 #### Cuda Tests  system/cuda
 This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
+#### All PTS Tests  pts/all
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### Installed Tests  pts/installed
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
+#### Linux Operating System Tests  pts/linux
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+#### Solaris Operating System Tests  pts/solaris
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
+#### BSD Operating System Tests  pts/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
+#### MacOSX Operating System Tests  pts/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
+#### Windows Operating System Tests  pts/windows
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
+#### Hurd Operating System Tests  pts/hurd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
+#### System Subsystem Tests  pts/system
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
+#### Processor Subsystem Tests  pts/processor
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
+#### Graphics Subsystem Tests  pts/graphics
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
+#### Other Subsystem Tests  pts/other
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### Utility Tests  pts/utility
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
+#### Simulator Tests  pts/simulator
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
+#### Scientific Tests  pts/scientific
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
+#### Benchmark Tests  pts/benchmark
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
+#### Application Tests  pts/application
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
+#### Game Tests  pts/game
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+#### Smp Tests  pts/smp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
+#### Cuda Tests  pts/cuda
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
+#### Openmp Tests  pts/openmp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.
+#### Python Tests  pts/python
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.
+#### Go Tests  pts/go
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.
+#### Mpi Tests  pts/mpi
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing mpi.
+#### Vdpau Tests  pts/vdpau
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing vdpau.
+#### Video Tests  pts/video
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing video.
+#### Responsiveness Tests  pts/responsiveness
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.
+#### Openmpi Tests  pts/openmpi
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.
 
 # Component Testing
 

documentation/stubs/00_overview.html

@@ -4,7 +4,7 @@
 </head>
 <body>
 <h1>Overview</h1>
-<p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
+<p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
 <p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
 <p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
 <h1>OpenBenchmarking.org</h1>

documentation/stubs/00_user_options.html

@@ -5,10 +5,29 @@
 </head>
 <body>
 <p>The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running <em>phoronix-test-suite help.</em><hr></p>
+<h1>System</h1>
+<h3>diagnostics<em>  </em>
+</h3>
+<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
+<h3>interactive<em>  </em>
+</h3>
+<p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
+<h3>php-conf<em>  </em>
+</h3>
+<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
+<h3>shell<em>  </em>
+</h3>
+<p>A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite</p>
+<h3>system-info<em>  </em>
+</h3>
+<p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
+<h3>system-sensors<em>  </em>
+</h3>
+<p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
 <h1>Test Installation</h1>
 <h3>force-install<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
-<p>This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.</p>
+<p>This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.</p>
 <h3>install<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.</p>
@@ -46,6 +65,9 @@
 <h3>stress-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.</p>
+<h3>strict-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.</p>
 <h1>Batch Testing</h1>
 <h3>batch-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
@@ -65,6 +87,9 @@
 <h3>default-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.</p>
+<h3>dry-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.</p>
 <h3>internal-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.</p>
@@ -93,6 +118,9 @@
 <h3>openbenchmarking-uploads<em>  </em>
 </h3>
 <p>This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.</p>
+<h3>recently-added-tests<em>  </em>
+</h3>
+<p>This option will list the most recently added (newest) test profiles.</p>
 <h3>upload-result<em>  [Test Result]</em>
 </h3>
 <p>This option is used for uploading a test result to OpenBenchmarking.org.</p>
@@ -102,38 +130,25 @@
 <h3>upload-test-suite<em>  </em>
 </h3>
 <p>This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.</p>
-<h1>System</h1>
-<h3>diagnostics<em>  </em>
-</h3>
-<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>interactive<em>  </em>
-</h3>
-<p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
-<h3>php-conf<em>  </em>
-</h3>
-<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>shell<em>  </em>
-</h3>
-<p>A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite</p>
-<h3>system-info<em>  </em>
-</h3>
-<p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
-<h3>system-sensors<em>  </em>
-</h3>
-<p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
 <h1>Information</h1>
 <h3>info<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will show details about the supplied test, suite, virtual suite, or result file.</p>
+<h3>list-all-tests<em>  </em>
+</h3>
+<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
 <h3>list-available-suites<em>  </em>
 </h3>
 <p>This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.</p>
 <h3>list-available-tests<em>  </em>
 </h3>
-<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.</p>
+<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.</p>
 <h3>list-available-virtual-suites<em>  </em>
 </h3>
 <p>This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.</p>
+<h3>list-cached-tests<em>  </em>
+</h3>
+<p>This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.</p>
 <h3>list-installed-dependencies<em>  </em>
 </h3>
 <p>This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.</p>
@@ -208,6 +223,9 @@
 <h3>auto-sort-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to automatically attempt to sort the results by their result identifier string.</p>
+<h3>compare-results-to-baseline<em>  [Test Result] [Test Result]</em>
+</h3>
+<p>This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.</p>
 <h3>edit-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to edit the title and description of an existing result file.</p>
@@ -235,6 +253,12 @@
 <h3>reorder-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
+<h3>result-file-raw-to-csv<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
+<h3>result-file-stats<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
 <h3>result-file-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.</p>
@@ -250,7 +274,7 @@
 <h3>show-result<em>  [Test Result]</em>
 </h3>
 <p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
-<h3>winners-and-losers<em>  [Test Result]</em>
+<h3>wins-and-losses<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
 <h1>Other</h1>
@@ -307,10 +331,6 @@
 <h3>user-config-set<em>  </em>
 </h3>
 <p>This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.</p>
-<h1>Web / GUI Support</h1>
-<h3>gui<em>  </em>
-</h3>
-<p>Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.</p>
 <h1>Phoromatic</h1>
 <h3>start-phoromatic-server<em>  </em>
 </h3>

documentation/stubs/01_install.html

@@ -4,7 +4,7 @@
 </head>
 <body>
 <h1>Setup Overview</h1>
-<p>The Phoronix Test Suite supports Linux, Apple Mac OS X, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or Mac OS X, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
+<p>The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
 <p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
 <p>Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.</p>
 <h1>Dependencies</h1>
@@ -37,10 +37,8 @@
 <p>Running <em>install-sh</em> from the root directory of the Phoronix Test Suite will install the software for system-wide access. By default the <em>phoronix-test-suite</em> executable is in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in /<em>usr/share/doc/phoronix-test-suite/</em>. Root access is required. The default installation prefix is /usr/ but can be adjusted as the first argument (example: <em>install-sh /home/user/</em> to install the Phoronix Test Suite in your home directory).</p>
 <h2>Debian/Ubuntu Installation</h2>
 <p>Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the <em>phoronix-test-suite</em> executable in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in <em>/usr/share/doc/phoronix-test-suite/</em>.</p>
-<p>A phoronix-test-suite package can also be found in the official Ubuntu universe repository beginning with the Ubuntu 9.04 (Jaunty) release, but it may be out of date compared to the latest upstream version.</p>
-<p>On Ubuntu 13.04 and newer if you experience an issue of system sensor monitoring not working, other multi-process tasks, or related issues reported as PHP PCNTL missing, the pcntl functions need to be removed from the <em>disable_functions</em> line within the <em>/etc/php5/cli/php.ini</em> file. This may also affect other Linux distribution support too if the vendor disables access to the PCNTL functions.</p>
 <h2>Fedora / Red Hat Installation</h2>
-<p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. At this time, however, the Fedora-supplied package is vastly out of date.</p>
+<p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.</p>
 <h2>Gentoo Installation</h2>
 <p>Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the <a href="http://bugs.gentoo.org/show_bug.cgi?id=216656">Gentoo BugZilla</a>.</p>
 <h2>Mandriva Installation</h2>
@@ -50,8 +48,8 @@
 <h2>BSD Installation</h2>
 <p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
 <p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. With DragonFlyBSD using dports, the command is <em>pkg install php5 php5-dom php5-zip php5-json php5-simplexml</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
-<h2>Mac OS X Installation</h2>
-<p>The Phoronix Test Suite is fully supported on Apple's Mac OS X operating system as of Mac OS X 10.5 with improved support under Mac OS X 10.6. PHP ships with Mac OS X by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
+<h2>MacOS Installation</h2>
+<p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
 <h2>Windows Installation</h2>
 <p>The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
 <p>At this time the Phoronix Test Suite client is dependent upon the <em>php.exe</em> executable being installed to <em>C:\Program Files (x86)\PHP\ </em> or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run <em>phoronix-test-suite.bat</em></p>

documentation/stubs/02_external_dependencies.html

@@ -1 +1,10 @@
-<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>External Dependencies</title></head><body><p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p><p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p></body></html>
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>External Dependencies</title>
+</head>
+<body>
+<p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
+<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p>
+</body>
+</html>

documentation/stubs/40_configuration.html

@@ -55,8 +55,12 @@
 <p>When running any test(s), if you would like the test(s) to continue running for a number of times, this variable can be used. The value should be the number of times to loop the testing process before ending.</p>
 <p><strong>FORCE_TIMES_TO_RUN</strong></p>
 <p>If you wish to override the number of times to run each test -- rather than the Phoronix Test Suite using the number of times specified in each test profile -- this variable can be used.</p>
+<p><strong>FORCE_TIMES_TO_RUN_MULTIPLE</strong></p>
+<p>This option allows specifying a multiple for increasing the number of times a test will run based upon the original TimesToRun value specified in the test definition. This allows for increasing the expected times to run based on a multiple of that default rather than a static value.</p>
 <p><strong>FORCE_MIN_TIMES_TO_RUN</strong></p>
 <p>This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.</p>
+<p><strong>FORCE_MIN_TIMES_TO_RUN_CUTOFF</strong></p>
+<p>When used in conjunction with FORCE_MIN_TIMES_TO_RUN, the override value will only be applied to test profiles where its average run-time length (in minutes) is less than the value specified by FORCE_MIN_TIMES_TO_RUN_CUTOFF.</p>
 <p><strong>NO_FILE_HASH_CHECKS</strong></p>
 <p>To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as <em>NO_MD5_CHECKS</em>, which is still honored but was changed to <em>NO_FILE_HASH_CHECKS</em> to reflect other kind of file hash sum checks.</p>
 <p><strong>NO_HTTPS</strong></p>

documentation/stubs/readme-basics.txt

@@ -10,7 +10,7 @@ Full details on the Phoronix Test Suite setup and usage is available from the in
 
 ## Installation & Setup
 
-The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
+The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
 
 The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php5-cli`, or `php5`.
 

install-sh

@@ -2,7 +2,7 @@
 
 # Phoronix Test Suite
 # URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-# Copyright (C) 2008 - 2018, Phoronix Media
+# Copyright (C) 2008 - 2019, Phoronix Media
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by

phoronix-test-suite

@@ -3,8 +3,8 @@
 #
 #	Phoronix Test Suite
 #	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-#	Copyright (C) 2008 - 2018, Phoronix Media
-#	Copyright (C) 2008 - 2018, Michael Larabel
+#	Copyright (C) 2008 - 2019, Phoronix Media
+#	Copyright (C) 2008 - 2019, Michael Larabel
 #	phoronix-test-suite: The Phoronix Test Suite is an extensible open-source testing / benchmarking platform
 #
 #	This program is free software; you can redistribute it and/or modify
@@ -21,6 +21,12 @@
 #	along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
 
+if which sed >/dev/null 2>&1 ;
+then
+	# Remove the current dir from PATH if present, as it causes infinitely resursive shell scripts.
+	PATH="`echo $PATH | sed -e s/:[\.]:/:/g -e s/:[\.]// -e  s/[\.]://`"
+fi
+
 # Full path to root directory of the actual Phoronix Test Suite code
 # export PTS_DIR=$(readlink -f `dirname $0`)
 export PTS_DIR=`pwd`
@@ -53,18 +59,12 @@ then
 elif [ -x /usr/bin/php7 ] || [ -x /usr/local/bin/php7 ] || [ -x /usr/pkg/bin/php7 ]
 then
 	export PHP_BIN="php7"
-elif [ -x /usr/bin/php5 ] || [ -x /usr/local/bin/php5 ] || [ -x /usr/pkg/bin/php5 ]
-then
-	export PHP_BIN="php5"
 elif [ -x /usr/bin/php ] || [ -x /usr/pkg/bin/php ]
 then
 	export PHP_BIN="php"
 elif [ -x /usr/php7/bin/php ]
 then
 	export PHP_BIN="/usr/php7/bin/php"
-elif [ -x /usr/php5/bin/php ]
-then
-	export PHP_BIN="/usr/php5/bin/php"
 elif [ -x /usr/php/bin/php ]
 then
 	export PHP_BIN="/usr/php/bin/php"
@@ -92,6 +92,12 @@ then
 elif which php7 >/dev/null 2>&1 ;
 then
 	export PHP_BIN="php7"
+elif [ -x /usr/php5/bin/php ]
+then
+	export PHP_BIN="/usr/php5/bin/php"
+elif [ -x /usr/bin/php5 ] || [ -x /usr/local/bin/php5 ] || [ -x /usr/pkg/bin/php5 ]
+then
+	export PHP_BIN="php5"
 elif which php5 >/dev/null 2>&1 ;
 then
 	export PHP_BIN="php5"
@@ -113,9 +119,9 @@ if [ "X$PHP_BIN" = "X" ]
 then
 	cat <<'EOT'
 
-PHP 5.3 or newer or PHP 7 must be installed for the Phoronix Test Suite
-The PHP command-line package is commonly called php-cli, php5-cli, or php.
-For more information visit: http://www.phoronix-test-suite.com/
+PHP must be installed for the Phoronix Test Suite
+The PHP command-line package is commonly called php-cli, php7-cli, or php.
+For more information visit: https://www.phoronix-test-suite.com/
 
 EOT
 
@@ -123,7 +129,7 @@ if [ -x /usr/bin/zypper ]
 then
 	# SUSE / OpenSUSE
 	echo "The command to likely run for your operating system is: "
-	echo "# zypper install php7 php7-zip"
+	echo "# zypper install php7 php7-zip php7-openssl"
 	echo " "
 elif [ -x /usr/bin/apt-get ]
 then
@@ -135,7 +141,7 @@ elif [ -x /usr/bin/dnf ]
 then
 	# Modern Fedora
 	echo "The command to likely run for your operating system is: "
-	echo "# dnf install php-cli php-xml"
+	echo "# dnf install php-cli php-xml php-json"
 	echo " "
 elif [ -x /usr/bin/yum ]
 then
@@ -147,7 +153,7 @@ elif [ -x /usr/bin/pkg_radd ]
 then
 	# BSD
 	echo "The command to likely run for your operating system is: "
-	echo "# pkg_radd php5 php5-json php5-zip php5-dom"
+	echo "# pkg_radd php7 php7-json php7-zip php7-dom"
 	echo " "
 elif [ -x /usr/bin/pacman ]
 then
@@ -161,7 +167,7 @@ then
 	echo "The command to likely run for your operating system is: "
 	echo "# pkg install php72 php72-dom php72-zip php72-json php72-simplexml php72-openssl"
 	echo " "
-elif [ -x /usr/local/swupd ]
+elif [ -x /usr/bin/swupd ]
 then
 	# Clear Linux
 	echo "The command to likely run for your operating system is: "
@@ -195,7 +201,7 @@ elif [ -x /sbin/apk ]
 then
 	# Alpine Linux
 	echo "The command to likely run for your operating system is: "
-	echo "# apk add php5 php5-dom php5-zip php5-json"
+	echo "# apk add php7 php7-dom php7-zip php7-json php7-simplexml"
 	echo " "
 elif [ -x /usr/bin/urpmi ]
 then
@@ -213,7 +219,7 @@ elif [ -x /usr/sbin/mport ]
 then
 	# MidnightBSD
 	echo "The command to likely run for your operating system is: "
-	echo "# mport install php70 php70-dom php70-zip php70-json php70-simplexml"
+	echo "# mport install php71 php71-dom php71-zip php71-json php71-simplexml"
 	echo " "
 elif [ -x /bin/opkg ]
 then
@@ -249,7 +255,7 @@ then
 	cat <<'EOT'
 
 To run the Phoronix Test Suite locally you must first change directories to phoronix-test-suite/ 
-or install the program using the install-sh script. For support visit: http://www.phoronix-test-suite.com/
+or install the program using the install-sh script. For support visit: https://www.phoronix-test-suite.com/
 
 EOT
 exit

phoronix-test-suite.bat

@@ -1,7 +1,7 @@
 ::	Phoronix Test Suite
 ::	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-::	Copyright (C) 2008 - 2018, Phoronix Media
-::	Copyright (C) 2008 - 2018, Michael Larabel
+::	Copyright (C) 2008 - 2019, Phoronix Media
+::	Copyright (C) 2008 - 2019, Michael Larabel
 ::	phoronix-test-suite: The Phoronix Test Suite is an extensible open-source testing / benchmarking platform
 ::
 ::	This program is free software; you can redistribute it and/or modify
@@ -24,6 +24,11 @@ set PTS_DIR=%~dp0
 set PTS_MODE=CLIENT
 set PTS_LAUNCHER=%0
 
+:: See if php was setup via Cygwin64, the benefit there is allowing access to PCNTL extensions, etc
+:: If exist C:\cygwin64\bin\php.exe (
+:: set PHP_BIN=C:\cygwin64\bin\php.exe
+:: )
+
 :: TODO: Other work to bring this up to sync with the *NIX phoronix-test-suite launcher
 If defined PHP_BIN goto SkipBinSearch
   
@@ -45,4 +50,4 @@ set PHP_BIN=C:\PHP\php.exe
 
 :SkipBinSearch
 
-%PHP_BIN% %PTS_DIR%\pts-core\phoronix-test-suite.php %*
+%PHP_BIN% "%PTS_DIR%\pts-core\phoronix-test-suite.php" %*

pts-core/commands/build_suite.php

@@ -34,8 +34,8 @@ class build_suite implements pts_option_interface
 		$suite_maintainer = pts_user_io::prompt_user_input('Enter suite maintainer name');
 		$suite_description = pts_user_io::prompt_user_input('Enter suite description');
 
-		$possible_suites = pts_openbenchmarking::available_suites();
-		$possible_tests = pts_openbenchmarking::available_tests();
+		$possible_suites = pts_openbenchmarking::available_suites(false);
+		$possible_tests = pts_openbenchmarking::available_tests(false);
 
 		$new_suite = new pts_test_suite();
 		$new_suite->set_title($suite_name);

pts-core/commands/compare_results_to_baseline.php

@@ -0,0 +1,65 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class compare_results_to_baseline implements pts_option_interface
+{
+	const doc_section = 'Result Management';
+	const doc_description = 'This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'is_result_file'), null),
+		new pts_argument_check(1, array('pts_types', 'is_result_file'), null)
+		);
+	}
+	public static function run($r)
+	{
+		if(count($r) != 2)
+		{
+			echo PHP_EOL . 'Two results must be specified: phoronix-test-suite compare-results-to-baseline <baseline result file> <compare result file>' . PHP_EOL;
+			return false;
+		}
+		$baseline = new pts_result_file($r[0]);
+		echo 'Baseline: ' . $baseline->get_identifier() . PHP_EOL;
+		if(count($baseline->get_system_identifiers()) != 1)
+		{
+			echo PHP_EOL . 'This feature requires only one system/result to be present in the result file.' . PHP_EOL;
+			return false;
+		}
+		$baseline->rename_run(null, 'Baseline');
+
+		$result = new pts_result_file($r[1]);
+		echo 'Result: ' . $result->get_identifier() . PHP_EOL;
+		if(count($result->get_system_identifiers()) != 1)
+		{
+			echo PHP_EOL . 'This feature requires only one system/result to be present in the result file.' . PHP_EOL;
+			return false;
+		}
+		$result->rename_run(null, 'Result');
+
+		$baseline->add_to_result_file($result, true);
+		echo pts_result_file_analyzer::display_results_baseline_two_way_compare($baseline, false, true);
+	}
+}
+
+?>

pts-core/commands/debug_benchmark.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2015, Phoronix Media
-	Copyright (C) 2009 - 2015, Michael Larabel
+	Copyright (C) 2009 - 2019, Phoronix Media
+	Copyright (C) 2009 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -40,7 +40,10 @@ class debug_benchmark implements pts_option_interface
 		// Make sure you're debugging the latest test script...
 		pts_test_installer::standard_install($r);
 		// For debugging, usually running just once is sufficient, unless FORCE_TIMES_TO_RUN is preset
-		pts_client::pts_set_environment_variable('FORCE_TIMES_TO_RUN', 1);
+		if(getenv('FORCE_TIMES_TO_RUN') == null)
+		{
+			pts_client::pts_set_environment_variable('FORCE_TIMES_TO_RUN', 1);
+		}
 		// Run the test(s) in debug mode
 		pts_client::set_debug_mode(true);
 		$test_run_manager = new pts_test_run_manager();

pts-core/commands/diagnostics.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2014, Phoronix Media
-	Copyright (C) 2008 - 2014, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -34,18 +34,18 @@ class diagnostics implements pts_option_interface
 				continue;
 			}
 
-			echo $constant . ' = ' . $constant_value . PHP_EOL;
+			echo pts_client::cli_just_bold($constant) . ' = ' . $constant_value . PHP_EOL;
 		}
 
 		echo PHP_EOL . 'Variables That Can Be Used As Result Identifiers / File Names:' . PHP_EOL;
 		foreach(pts_client::user_run_save_variables() as $var => $var_value)
 		{
-			echo $var . ' = ' . $var_value . PHP_EOL;
+			echo pts_client::cli_just_bold($var) . ' = ' . $var_value . PHP_EOL;
 		}
 		echo PHP_EOL . 'Environmental Variables (accessible via test scripts):' . PHP_EOL;
 		foreach(pts_client::environmental_variables() as $var => $var_value)
 		{
-			echo $var . ' = ' . $var_value . PHP_EOL;
+			echo pts_client::cli_just_bold($var) . ' = ' . $var_value . PHP_EOL;
 		}
 		echo PHP_EOL;
 	}

pts-core/commands/dry_run.php

@@ -0,0 +1,51 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2013 - 2018, Phoronix Media
+	Copyright (C) 2013 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class dry_run implements pts_option_interface
+{
+	const doc_section = 'Batch Testing';
+	const doc_description = 'This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check('VARIABLE_LENGTH', array('pts_types', 'identifier_to_object'), null)
+		);
+	}
+	public static function run($r)
+	{
+		pts_test_installer::standard_install($r);
+		$test_run_manager = new pts_test_run_manager(array(
+			'UploadResults' => false,
+			'SaveResults' => false,
+			'PromptForTestDescription' => false,
+			'RunAllTestCombinations' => false,
+			'PromptSaveName' => false,
+			'PromptForTestIdentifier' => false,
+			'OpenBrowser' => false
+			));
+
+		$test_run_manager->standard_run($r);
+	}
+}
+
+?>

pts-core/commands/dump_documentation.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2018, Phoronix Media
-	Copyright (C) 2010 - 2018, Michael Larabel
+	Copyright (C) 2010 - 2019, Phoronix Media
+	Copyright (C) 2010 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -28,9 +28,9 @@ class dump_documentation implements pts_option_interface
 		$html_doc = new pts_html_template(pts_core::program_title(false), 'Test Client Documentation');
 
 		$pdf->AddPage();
-		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'http://www.phoronix-test-suite.com/');
+		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'https://www.phoronix-test-suite.com/');
 		$pdf->Ln(120);
-		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'http://www.phoronix-test-suite.com/');
+		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'https://www.phoronix-test-suite.com/');
 		$pdf->Ln(15);
 		$pdf->WriteBigHeaderCenter(pts_core::program_title(true));
 		$pdf->WriteHeaderCenter('User Manual');
@@ -85,6 +85,7 @@ class dump_documentation implements pts_option_interface
 
 		// Write the module options HTML
 		$dom = new DOMDocument();
+		$dom->formatOutput = true;
 		$html = $dom->createElement('html');
 		$dom->appendChild($html);
 		$head = $dom->createElement('head');
@@ -138,6 +139,7 @@ class dump_documentation implements pts_option_interface
 
 		// Write the external dependencies HTML
 		$dom = new DOMDocument();
+		$dom->formatOutput = true;
 		$html = $dom->createElement('html');
 		$dom->appendChild($html);
 		$head = $dom->createElement('head');
@@ -173,6 +175,7 @@ class dump_documentation implements pts_option_interface
 
 		// Write the virtual suites HTML
 		$dom = new DOMDocument();
+		$dom->formatOutput = true;
 		$html = $dom->createElement('html');
 		$dom->appendChild($html);
 		$head = $dom->createElement('head');
@@ -250,7 +253,7 @@ class dump_documentation implements pts_option_interface
 					$man_page .= '.B ' . trim($option[0] . ' ' . (!empty($option[1]) && is_array($option[1]) ? implode(' ', $option[1]) : null)) . PHP_EOL . $option[2] . PHP_EOL . '.TP' . PHP_EOL;
 				}
 			}
-			$man_page .= '.SH SEE ALSO' . PHP_EOL . '.B Websites:' . PHP_EOL . '.br' . PHP_EOL . 'http://www.phoronix-test-suite.com/' . PHP_EOL . '.br' . PHP_EOL . 'http://commercial.phoronix-test-suite.com/' . PHP_EOL . '.br' . PHP_EOL . 'http://www.openbenchmarking.org/' . PHP_EOL . '.br' . PHP_EOL . 'http://www.phoronix.com/' . PHP_EOL . '.br' . PHP_EOL . 'http://www.phoronix.com/forums/' . PHP_EOL;
+			$man_page .= '.SH SEE ALSO' . PHP_EOL . '.B Websites:' . PHP_EOL . '.br' . PHP_EOL . 'https://www.phoronix-test-suite.com/' . PHP_EOL . '.br' . PHP_EOL . 'https://commercial.phoronix-test-suite.com/' . PHP_EOL . '.br' . PHP_EOL . 'https://www.openbenchmarking.org/' . PHP_EOL . '.br' . PHP_EOL . 'https://www.phoronix.com/' . PHP_EOL . '.br' . PHP_EOL . 'https://www.phoronix.com/forums/' . PHP_EOL;
 			$man_page .= '.SH AUTHORS' . PHP_EOL . 'Copyright 2008 - ' . date('Y') . ' by Phoronix Media, Michael Larabel.' . PHP_EOL . '.TP' . PHP_EOL;
 
 			file_put_contents(PTS_PATH . 'documentation/man-pages/phoronix-test-suite.1', $man_page);
@@ -258,7 +261,7 @@ class dump_documentation implements pts_option_interface
 
 
 		// simple README
-		$readme = '# Phoronix Test Suite ' . PTS_VERSION . PHP_EOL . 'http://www.phoronix-test-suite.com/' . PHP_EOL . PHP_EOL;
+		$readme = '# Phoronix Test Suite ' . PTS_VERSION . PHP_EOL . 'https://www.phoronix-test-suite.com/' . PHP_EOL . PHP_EOL;
 		$readme .= pts_documentation::basic_description() . PHP_EOL . PHP_EOL;
 		$readme .= pts_file_io::file_get_contents(PTS_PATH . 'documentation/stubs/readme-basics.txt') . PHP_EOL . PHP_EOL;
 		$readme = wordwrap($readme, 80, PHP_EOL);
@@ -269,12 +272,12 @@ class dump_documentation implements pts_option_interface
 		$html_doc = new pts_html_template(pts_core::program_title(false), 'Phoromatic Documentation');
 
 		$pdf->AddPage();
-		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'http://www.phoronix-test-suite.com/');
+		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'https://www.phoronix-test-suite.com/');
 		$pdf->Ln(120);
-		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'http://www.phoronix-test-suite.com/');
+		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'https://www.phoronix-test-suite.com/');
 		$pdf->Ln(15);
-		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 55, 250, 0, 0, 'PNG', 'http://www.phoronix-test-suite.com/');
-		//$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 69, 85, 73, 38, 'PNG', 'http://www.phoromatic.com/');
+		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 55, 250, 0, 0, 'PNG', 'https://www.phoronix-test-suite.com/');
+		//$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 69, 85, 73, 38, 'PNG', 'https://www.phoromatic.com/');
 		$pdf->WriteBigHeaderCenter(pts_core::program_title(true));
 		$pdf->WriteHeaderCenter('Phoromatic User Manual');
 		$pdf->html_to_pdf(PTS_PATH . 'documentation/phoromatic.html');

pts-core/commands/dump_ob_to_ae_db.php

@@ -0,0 +1,139 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2019, Phoronix Media
+	Copyright (C) 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+ini_set('memory_limit', '8G');
+
+class dump_ob_to_ae_db implements pts_option_interface
+{
+	public static function run($r)
+	{
+		$dir_to_recursively_scan = $r[0];
+		if(!is_dir($dir_to_recursively_scan))
+		{
+			echo $dir_to_recursively_scan . ' is not a dir.';
+			return false;
+		}
+		$storage_dir = $r[1];
+		if(!is_dir($storage_dir))
+		{
+			echo $storage_dir . ' is not a dir.';
+			return false;
+		}
+
+		$ae = new pts_ae_data($storage_dir);
+		$xml_files = array();
+		pts_file_io::recursively_find_files_in_directory($dir_to_recursively_scan, $xml_files, '.xml');
+
+		foreach($xml_files as $file)
+		{
+			$result_reference = null; // TODO fill in OpenBenchmarking.org ID for that
+			$rf = new pts_result_file($file, false, true);
+			$systems = $rf->get_systems();
+			$system_data = array();
+			$timestamps = array();
+			$system_types = array();
+			foreach($systems as $system)
+			{
+				$system_data[$system->get_identifier()] = array_map(array('pts_strings', 'trim_search_query'), array_merge(pts_result_file_analyzer::system_component_string_to_array($system->get_hardware()), pts_result_file_analyzer::system_component_string_to_array($system->get_software())));
+				$timestamps[$system->get_identifier()] = strtotime($system->get_timestamp());
+				$system_types[$system->get_identifier()] = phodevi_base::determine_system_type($system->get_hardware(), $system->get_software());
+			}
+
+			foreach($rf->get_result_objects() as $ro)
+			{
+				if($ro->test_profile->get_identifier() == null)
+				{
+					continue;
+				}
+				$comparison_hash = $ro->get_comparison_hash(true, false);
+				$inserts = 0;
+				foreach($ro->test_result_buffer as &$buffers)
+				{
+					if(empty($buffers))
+						continue;
+
+					foreach($buffers as &$buffer_item)
+					{
+						$result = $buffer_item->get_result_value();
+						if(stripos($result, ',') !== false || !is_numeric($result))
+						{
+							continue;
+						}
+						$system_identifier = $buffer_item->get_result_identifier();
+						$system_layer = isset($system_data[$system_identifier]['System Layer']) ? $system_data[$system_identifier]['System Layer'] : null;
+
+						switch($ro->test_profile->get_test_hardware_type())
+						{
+							case 'Processor':
+								$component = 'Processor';
+								$related_component = 'OS';
+								break;
+							case 'System':
+								$component = 'Processor';
+								$related_component = 'Motherboard';
+								break;
+							case 'Graphics':
+								$component = 'Graphics';
+								$related_component = 'OpenGL';
+								break;
+							case 'Disk':
+								$component = 'Disk';
+								$related_component = 'File-System';
+								break;
+							case 'Network':
+								$component = 'Network';
+								$related_component = 'OS';
+								break;
+							case 'Memory':
+								$component = 'Memory';
+								$related_component = 'Processor';
+								break;
+							default:
+								$component = 'Processor';
+								$related_component = 'OS';
+								break;
+						}
+
+						if(!isset($system_data[$system_identifier][$component]) || empty($system_data[$system_identifier][$component]))
+						{
+							continue;
+						}
+						$component_value = $system_data[$system_identifier][$component];
+						$related_component_value = isset($system_data[$system_identifier][$related_component]) ? $system_data[$system_identifier][$related_component] : null;
+						$ae->insert_result_into_analytic_results($comparison_hash, $result_reference, $component_value, $component, $related_component_value, $related_component, $result, $timestamps[$system_identifier], $system_types[$system_identifier], $system_layer);
+						$inserts++;
+					}
+
+				}
+
+				if($inserts > 0)
+				{
+					$ae->insert_composite_hash_entry_by_result_object($comparison_hash, $ro);
+				}
+			}
+			
+		}
+
+		$ae->rebuild_composite_listing();
+	}
+}
+
+?>

pts-core/commands/dump_possible_options.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2016, Phoronix Media
-	Copyright (C) 2009 - 2016, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -24,20 +24,8 @@ class dump_possible_options implements pts_option_interface
 {
 	public static function run($r)
 	{
-		$options = array();
-
-		foreach(pts_file_io::glob(PTS_COMMAND_PATH . '*.php') as $option_php)
-		{
-			$name = str_replace('_', '-', basename($option_php, '.php'));
-
-			if(!in_array(pts_strings::first_in_string($name, '-'), array('dump', 'debug', 'task')))
-			{
-				$options[] = $name;
-			}
-		}
-
 		$is_true = isset($r[0]) && $r[0] == 'TRUE';
-		echo implode($is_true ? ' ' : PHP_EOL, $options) . ($is_true ? null : PHP_EOL);
+		echo implode($is_true ? ' ' : PHP_EOL, pts_client::possible_sub_commands()) . ($is_true ? null : PHP_EOL);
 	}
 }
 

pts-core/commands/estimate_run_time.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2017, Phoronix Media
-	Copyright (C) 2017, Michael Larabel
+	Copyright (C) 2017 - 2019, Phoronix Media
+	Copyright (C) 2017 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -45,7 +45,7 @@ class estimate_run_time implements pts_option_interface
 		$test_count = 0;
 		foreach($args as $arg)
 		{
-			foreach(pts_types::identifiers_to_test_profile_objects($arg) as $t)
+			foreach(pts_types::identifiers_to_test_profile_objects($arg, false, false) as $t)
 			{
 				$tests[] = array($t->get_identifier(), pts_strings::format_time($t->get_estimated_run_time()));
 				$total_time += $t->get_estimated_run_time();

pts-core/commands/force_install.php

@@ -23,8 +23,12 @@
 class force_install implements pts_option_interface
 {
 	const doc_section = 'Test Installation';
-	const doc_description = 'This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.';
+	const doc_description = 'This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.';
 
+	public static function command_aliases()
+	{
+		return array('reinstall', 're-install');
+	}
 	public static function argument_checks()
 	{
 		return array(

pts-core/commands/gui.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2012 - 2018, Phoronix Media
-	Copyright (C) 2012 - 2018, Michael Larabel
+	Copyright (C) 2012 - 2019, Phoronix Media
+	Copyright (C) 2012 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -23,7 +23,8 @@
 class gui implements pts_option_interface
 {
 	const doc_section = 'Web / GUI Support';
-	const doc_description = 'Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.';
+	const doc_description = 'Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER IN ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.';
+		const doc_skip = true;
 
 	public static function command_aliases()
 	{

pts-core/commands/help.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2017, Phoronix Media
-	Copyright (C) 2010 - 2017, Michael Larabel
+	Copyright (C) 2010 - 2019, Phoronix Media
+	Copyright (C) 2010 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -28,7 +28,7 @@ class help implements pts_option_interface
 	public static function run($r)
 	{
 		echo PHP_EOL . pts_client::cli_colored_text(pts_core::program_title(true), 'green', true) . PHP_EOL . PHP_EOL;
-		echo pts_documentation::basic_description() . PHP_EOL . PHP_EOL . 'View the included PDF / HTML documentation or visit http://www.phoronix-test-suite.com/ for full details.' . PHP_EOL;
+		echo pts_documentation::basic_description() . PHP_EOL . PHP_EOL . 'View the included documentation or visit https://www.phoronix-test-suite.com/ for full details.' . PHP_EOL;
 		$options = pts_documentation::client_commands_array();
 
 		foreach($options as $section => &$contents)

pts-core/commands/info.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2018, Phoronix Media
-	Copyright (C) 2008 - 2018, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -175,7 +175,7 @@ class info implements pts_option_interface
 
 
 				// OpenBenchmarking.org Change-Log
-				if(stripos($o->get_identifier(), 'local/') === false)
+				if(stripos($o->get_identifier(), 'local/') === false && !defined('PHOROMATIC_PROCESS'))
 				{
 
 					$change_log = pts_openbenchmarking_client::fetch_repository_test_profile_changelog($o->get_identifier(false));
@@ -193,24 +193,27 @@ class info implements pts_option_interface
 				}
 
 				// Recent Test Results With This Test
-				$o_identifier = $o->get_identifier(false);
-				$table = array();
-				foreach(pts_client::saved_test_results() as $saved_results_identifier)
+				if(!defined('PHOROMATIC_PROCESS'))
 				{
-					$result_file = new pts_result_file($saved_results_identifier);
-					foreach($result_file->get_result_objects() as $result_object)
+					$o_identifier = $o->get_identifier(false);
+					$table = array();
+					foreach(pts_client::saved_test_results() as $saved_results_identifier)
 					{
-						if($result_object->test_profile->get_identifier(false) == $o_identifier)
+						$result_file = new pts_result_file($saved_results_identifier);
+						foreach($result_file->get_result_objects() as $result_object)
 						{
-							$table[] = array(pts_client::cli_just_bold($result_file->get_identifier()), $result_file->get_title());
-							break;
+							if($result_object->test_profile->get_identifier(false) == $o_identifier)
+							{
+								$table[] = array(pts_client::cli_just_bold($result_file->get_identifier()), $result_file->get_title());
+								break;
+							}
 						}
 					}
-				}
-				if(!empty($table))
-				{
-					echo PHP_EOL . pts_client::cli_just_bold('Results Containing This Test') . PHP_EOL;
-					echo pts_user_io::display_text_table($table);
+					if(!empty($table))
+					{
+						echo PHP_EOL . pts_client::cli_just_bold('Results Containing This Test') . PHP_EOL;
+						echo pts_user_io::display_text_table($table);
+					}
 				}
 			}
 			else if($o instanceof pts_result_file)
@@ -218,6 +221,11 @@ class info implements pts_option_interface
 				echo pts_client::cli_just_bold('Title: ') . $o->get_title() . PHP_EOL . pts_client::cli_just_bold('Identifier: ') . $o->get_identifier() . PHP_EOL;
 				echo PHP_EOL . pts_client::cli_just_bold('Test Result Identifiers:') . PHP_EOL;
 				echo pts_user_io::display_text_list($o->get_system_identifiers());
+				$system_count = count($o->get_system_identifiers());
+				if($system_count > 8)
+				{
+					echo pts_client::cli_just_italic($system_count . ' Systems') . PHP_EOL;
+				}
 
 				$test_titles = array();
 				foreach($o->get_result_objects() as $result_object)
@@ -232,6 +240,7 @@ class info implements pts_option_interface
 				{
 					echo PHP_EOL . pts_client::cli_just_bold('Contained Tests:') . PHP_EOL;
 					echo pts_user_io::display_text_list(array_unique($test_titles));
+					echo '  ' . pts_client::cli_just_italic(pts_strings::plural_handler($o->get_test_count(), 'Test')) . PHP_EOL;
 				}
 				echo PHP_EOL;
 			}

pts-core/commands/interactive.php

@@ -35,15 +35,16 @@ class interactive implements pts_option_interface
 
 		do
 		{
+			$only_show_available_cached_tests = pts_network::internet_support_available() == false;
 			$options = array(
-				'RUN_TEST' => 'Run A Test',
-				'RUN_SUITE' => 'Run A Suite [A Collection Of Tests]',
-				'RUN_SYSTEM_TEST' => 'Run Complex System Test',
+				'RUN_TEST' => 'Run A Test / Benchmark',
+				'RUN_SUITE' => 'Run A Suite      [A Collection Of Tests]',
+				'RUN_STRESS_TEST' => 'Run A Stress Test      [Concurrent Benchmarks For Heavy System Load]',
 				'SHOW_INFO' => 'Show System Hardware / Software Information',
 				'SHOW_SENSORS' => 'Show Available System Sensors',
 				'LIST_TESTS' => 'List Available Tests',
 				'LIST_RECOMMENDED_TESTS' => 'List Recommended Tests',
-				'SET_RUN_COUNT' => 'Set Test Run Repetition',
+			//	'SET_RUN_COUNT' => 'Set Test Run Repetition',
 				'SEARCH' => 'Search Tests / Suites / Results'
 				);
 
@@ -58,13 +59,18 @@ class interactive implements pts_option_interface
 			switch($response)
 			{
 				case 'RUN_TEST':
-					$supported_tests = pts_openbenchmarking::available_tests();
+					$supported_tests = pts_openbenchmarking::available_tests(!$only_show_available_cached_tests, false, false, false, $only_show_available_cached_tests);
 					$supported_tests = pts_types::identifiers_to_test_profile_objects($supported_tests, false, true);
 					$longest_title_length = 0;
 
 					foreach($supported_tests as $i => &$test_profile)
 					{
-						if($test_profile->get_title() == null)
+						if($test_profile->get_title() == null || $test_profile->get_license() == 'Retail')
+						{
+							unset($supported_tests[$i]);
+							continue;
+						}
+						if(!pts_test_run_manager::test_profile_system_compatibility_check($test_profile))
 						{
 							unset($supported_tests[$i]);
 							continue;
@@ -84,7 +90,7 @@ class interactive implements pts_option_interface
 					$supported_tests = $t;
 					asort($supported_tests);
 
-					$tests_to_run = pts_user_io::prompt_text_menu('Select Test', $supported_tests, true, true);
+					$tests_to_run = pts_user_io::prompt_text_menu('Select Test(s)', $supported_tests, true, true);
 					$tests_to_run = explode(',', $tests_to_run);
 					pts_test_installer::standard_install($tests_to_run);
 					$run_manager = new pts_test_run_manager(false, 2);
@@ -94,6 +100,49 @@ class interactive implements pts_option_interface
 						pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $run_manager->get_file_name() . '/index.html', null, true, true);
 					}
 					break;
+				case 'RUN_STRESS_TEST':
+					$supported_tests = pts_openbenchmarking::available_tests(!$only_show_available_cached_tests, false, false, false, $only_show_available_cached_tests);
+					$supported_tests = pts_types::identifiers_to_test_profile_objects($supported_tests, false, true);
+					$longest_title_length = 0;
+
+					foreach($supported_tests as $i => &$test_profile)
+					{
+						if($test_profile->get_title() == null || $test_profile->get_license() == 'Retail')
+						{
+							unset($supported_tests[$i]);
+							continue;
+						}
+						if(!pts_test_run_manager::test_profile_system_compatibility_check($test_profile))
+						{
+							unset($supported_tests[$i]);
+							continue;
+						}
+
+						$longest_title_length = max($longest_title_length, strlen($test_profile->get_title()));
+					}
+
+					$t = array();
+					foreach($supported_tests as $i => &$test_profile)
+					{
+						if($test_profile instanceof pts_test_profile)
+						{
+							$t[$test_profile->get_identifier()] = sprintf('%-' . ($longest_title_length + 1) . 'ls - %-10ls', $test_profile->get_title(), $test_profile->get_test_hardware_type());
+						}
+					}
+					$supported_tests = $t;
+					asort($supported_tests);
+
+					$tests_to_run = pts_user_io::prompt_text_menu('Select Test(s)', $supported_tests, true, true);
+					$tests_to_run = explode(',', $tests_to_run);
+
+					$concurrent_runs = pts_user_io::prompt_user_input('Number of tests to run concurrently');
+					putenv('PTS_CONCURRENT_TEST_RUNS=' . trim($concurrent_runs));
+					$minutes_loop_time = pts_user_io::prompt_user_input('Number of minutes to stress run');
+					putenv('TOTAL_LOOP_TIME=' . trim($minutes_loop_time));
+
+					pts_test_installer::standard_install($tests_to_run);
+					pts_client::execute_command('stress_run', $tests_to_run);
+					break;
 				case 'RUN_SUITE':
 					$possible_suites = pts_openbenchmarking::available_suites();
 
@@ -116,17 +165,6 @@ class interactive implements pts_option_interface
 				case 'SEARCH':
 					pts_client::execute_command('search');
 					break;
-				case 'RUN_SYSTEM_TEST':
-					pts_client::$display->generic_heading('System Test');
-					$system_tests = array('apache', 'c-ray', 'ramspeed', 'postmark');
-					pts_test_installer::standard_install($system_tests);
-					$run_manager = new pts_test_run_manager(false, 2);
-					$run_manager->standard_run($system_tests);
-					if($run_manager != false)
-					{
-						pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $run_manager->get_file_name() . '/index.html', null, true, true);
-					}
-					break;
 				case 'SHOW_INFO':
 					pts_client::execute_command('system_info');
 					break;

pts-core/commands/list_all_tests.php

@@ -0,0 +1,80 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class list_all_tests implements pts_option_interface
+{
+	const doc_section = 'Information';
+	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('Available Tests');
+		$list_all_tests = pts_client::get_sent_command() == 'list_all_tests';
+		$only_show_available_cached_tests = !$list_all_tests && pts_network::internet_support_available() == false;
+
+		if($only_show_available_cached_tests)
+		{
+			echo 'Internet support is not available/enabled, so the Phoronix Test Suite is only listing test profiles where any necessary test assets are already downloaded to the system or available via a network download cache. To override this behavior, use the ' . pts_client::cli_just_bold('phoronix-test-suite list-all-tests') . ' option.' . PHP_EOL . PHP_EOL;
+			pts_client::execute_command('list_cached_tests');
+			return true;
+		}
+
+		$test_count = 0;
+		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
+		{
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($list_all_tests == false && !empty($repo_index['tests'][$id]['status']) && $repo_index['tests'][$id]['status'] != 'Verified')
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			$test_count++;
+		}
+
+		foreach(pts_tests::local_tests() as $identifier)
+		{
+			$test_profile = new pts_test_profile($identifier);
+
+			if($test_profile->get_title() != null && $test_profile->is_supported(false))
+			{
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				$test_count++;
+			}
+		}
+
+		if($test_count == 0)
+		{
+			echo PHP_EOL . 'No tests found. Please check that you have Internet connectivity to download test profile data from OpenBenchmarking.org. The Phoronix Test Suite has documentation on configuring the network setup, proxy settings, and PHP network options. Please contact Phoronix Media if you continuing to experience problems.' . PHP_EOL . PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/list_available_suites.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -31,7 +31,7 @@ class list_available_suites implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		$available_suites = array_merge(pts_openbenchmarking::available_suites(false), pts_tests::local_suites());
+		$available_suites = pts_tests::all_suites();
 		pts_client::$display->generic_heading('Available Suites');
 
 		if(count($available_suites) > 0)

pts-core/commands/list_available_tests.php

@@ -23,7 +23,7 @@
 class list_available_tests implements pts_option_interface
 {
 	const doc_section = 'Information';
-	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.';
+	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.';
 
 	public static function command_aliases()
 	{
@@ -32,14 +32,22 @@ class list_available_tests implements pts_option_interface
 	public static function run($r)
 	{
 		pts_client::$display->generic_heading('Available Tests');
+		$only_show_available_cached_tests = pts_network::internet_support_available() == false;
+
+		if($only_show_available_cached_tests)
+		{
+			echo 'Internet support is not available/enabled, so the Phoronix Test Suite is only listing test profiles where any necessary test assets are already downloaded to the system or available via a network download cache. To override this behavior, use the ' . pts_client::cli_just_bold('phoronix-test-suite list-all-tests') . ' option.' . PHP_EOL . PHP_EOL;
+			pts_client::execute_command('list_cached_tests');
+			return true;
+		}
+
 		$test_count = 0;
 		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
 		{
 			$repo = substr($identifier, 0, strpos($identifier, '/'));
 			$id = substr($identifier, strlen($repo) + 1);
 			$repo_index = pts_openbenchmarking::read_repository_index($repo);
-
-			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::operating_system(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
 			{
 				// Don't show unsupported tests
 				continue;
@@ -49,8 +57,13 @@ class list_available_tests implements pts_option_interface
 				// Don't show unsupported tests
 				continue;
 			}
+			if($repo_index['tests'][$id]['test_type'] == 'Graphics' && !phodevi::is_display_server_active())
+			{
+				// Don't display graphics tests that can't run
+				continue;
+			}
 
-			echo sprintf('%-30ls - %-35ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
 			$test_count++;
 		}
 
@@ -60,7 +73,7 @@ class list_available_tests implements pts_option_interface
 
 			if($test_profile->get_title() != null && $test_profile->is_supported(false))
 			{
-				echo sprintf('%-30ls - %-35ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
 				$test_count++;
 			}
 		}

pts-core/commands/list_cached_tests.php

@@ -0,0 +1,99 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class list_cached_tests implements pts_option_interface
+{
+	const doc_section = 'Information';
+	const doc_description = 'This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('Cached Tests');
+
+		$test_count = 0;
+		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
+		{
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($repo_index['tests'][$id]['test_type'] == 'Graphics' && !phodevi::is_display_server_active())
+			{
+				// Don't display graphics tests that can't run
+				continue;
+			}
+
+			$show = false;
+			foreach($repo_index['tests'][$id]['versions'] as $version)
+			{
+				if(!pts_openbenchmarking::is_test_profile_downloaded($identifier . '-' . $version))
+				{
+					// Without Internet, won't be able to download test, so don't show it
+					continue;
+				}
+				$test_profile = new pts_test_profile($identifier . '-' . $version);
+				if(pts_test_install_request::test_files_available_via_cache($test_profile) == false)
+				{
+					// only show tests where files are local or in an available cache
+					continue;
+				}
+				$show = true;
+				$identifier .= '-' . $version;
+				break;
+			}
+			if($show == false)
+			{
+				continue;
+			}
+
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			$test_count++;
+		}
+
+		foreach(pts_tests::local_tests() as $identifier)
+		{
+			$test_profile = new pts_test_profile($identifier);
+			if(pts_test_install_request::test_files_available_via_cache($test_profile) == false)
+			{
+				// only show tests where files are local or in an available cache
+				continue;
+			}
+
+			if($test_profile->get_title() != null && $test_profile->is_supported(false))
+			{
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				$test_count++;
+			}
+		}
+
+		if($test_count == 0)
+		{
+			echo PHP_EOL . 'No tests found. Please check that you have Internet connectivity to download test profile data from OpenBenchmarking.org. The Phoronix Test Suite has documentation on configuring the network setup, proxy settings, and PHP network options. Please contact Phoronix Media if you continuing to experience problems.' . PHP_EOL . PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/list_not_installed_tests.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2016, Phoronix Media
-	Copyright (C) 2016, Michael Larabel
+	Copyright (C) 2016 - 2018, Phoronix Media
+	Copyright (C) 2016 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -39,7 +39,7 @@ class list_not_installed_tests implements pts_option_interface
 			$id = substr($identifier, strlen($repo) + 1);
 			$repo_index = pts_openbenchmarking::read_repository_index($repo);
 
-			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::operating_system(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
 			{
 				// Don't show unsupported tests
 				continue;

pts-core/commands/list_recommended_tests.php

@@ -28,24 +28,47 @@ class list_recommended_tests implements pts_option_interface
 	public static function run($r)
 	{
 		pts_client::$display->generic_heading('Recommended OpenBenchmarking.org Test Profiles');
-		$test_count = 0;
-		$recommendation_index = pts_openbenchmarking::make_openbenchmarking_request('recommended_tests_index');
-		$recommendation_index = json_decode($recommendation_index, true);
 
-		foreach($recommendation_index['recommended_tests'] as $subsystem => $tests)
+		$tests = array();
+		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
 		{
-			pts_client::$display->generic_heading($subsystem . ' Tests');
-			foreach($tests as $test)
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
 			{
-				echo sprintf('%-32ls - %-35ls', $test['test_profile'], $test['title']) . PHP_EOL;
+				// Don't show unsupported tests
+				continue;
+			}
+			if(!empty($repo_index['tests'][$id]['status']) && $repo_index['tests'][$id]['status'] != 'Verified')
+			{
+				// Don't show unsupported tests
+				continue;
 			}
 
-			$test_count++;
+			if($repo_index['tests'][$id]['last_updated'] < (time() - (60 * 60 * 24 * 365)))
+			{
+				// Don't show tests not actively maintained
+				continue;
+			}
+
+			if(!isset($tests[$repo_index['tests'][$id]['test_type']]))
+			{
+				$tests[$repo_index['tests'][$id]['test_type']] = array();
+			}
+
+			$tests[$repo_index['tests'][$id]['test_type']][$identifier] = $repo_index['tests'][$id];
 		}
 
-		if($test_count == 0)
+		foreach($tests as $subsystem => $test_json)
 		{
-			echo PHP_EOL . 'No tests found. Please check that you have Internet connectivity to download test profile data from OpenBenchmarking.org. The Phoronix Test Suite has documentation on configuring the network setup, proxy settings, and PHP network options. Please contact Phoronix Media if you continuing to experience problems.' . PHP_EOL . PHP_EOL;
+			uasort($test_json, array('pts_openbenchmarking_client', 'compare_test_json_download_counts'));
+			$test_json = array_slice($test_json, 0, 10);
+			pts_client::$display->generic_heading($subsystem . ' Tests');
+			foreach($test_json as $identifier => $test_individual_json)
+			{
+				echo sprintf('%-30ls - %-39ls', $identifier, $test_individual_json['title']) . PHP_EOL;
+			}
 		}
 	}
 }

pts-core/commands/list_unsupported_tests.php

@@ -45,7 +45,7 @@ class list_unsupported_tests implements pts_option_interface
 
 			if($unsupported)
 			{
-				echo sprintf('%-28ls - %-30ls %-9ls', $identifier, $unsupported, $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+				echo sprintf('%-28ls - %-30ls', $identifier, $unsupported) . PHP_EOL;
 			}
 		}
 

pts-core/commands/make_download_cache.php

@@ -32,6 +32,10 @@ class make_download_cache implements pts_option_interface
 
 		// Determine cache location
 		$dc_write_directory = pts_client::download_cache_path();
+		if(($dc_override = getenv('PTS_DOWNLOAD_CACHE_OVERRIDE')) != false && is_dir($dc_override))
+		{
+			$dc_write_directory = $dc_override;
+		}
 		pts_file_io::mkdir($dc_write_directory);
 		echo PHP_EOL . 'Download Cache Directory: ' . $dc_write_directory . PHP_EOL;
 
@@ -48,7 +52,7 @@ class make_download_cache implements pts_option_interface
 			if(count($test_profiles) > 0)
 			{
 				echo PHP_EOL . 'Downloading Test Files For: ' . implode(' ', $test_profiles);
-				pts_test_installer::only_download_test_files($test_profiles, $dc_write_directory);
+				pts_test_installer::only_download_test_files($test_profiles, $dc_write_directory, getenv('PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT') !== false);
 			}
 		}
 
@@ -62,7 +66,19 @@ class make_download_cache implements pts_option_interface
 			$test_profile = new pts_test_profile($test);
 			echo PHP_EOL . pts_client::cli_just_bold('Checking Downloads For: ') . $test . PHP_EOL;
 
-			foreach($test_profile->get_downloads() as $file)
+			if(getenv('PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT') !== false)
+			{
+				// Don't get all download files but just those for the given platform
+				$tr = new pts_test_install_request($test_profile);
+				$tr->generate_download_object_list(true);
+				$downloads = $tr->get_download_objects();
+			}
+			else
+			{
+				$downloads = $test_profile->get_downloads();
+			}
+
+			foreach($downloads as $file)
 			{
 				$cached_valid = false;
 				if(is_file($dc_write_directory . $file->get_filename()) && $file->check_file_hash($dc_write_directory . $file->get_filename()))

pts-core/commands/make_openbenchmarking_cache.php

@@ -54,7 +54,14 @@ class make_openbenchmarking_cache implements pts_option_interface
 			$repo_index = pts_openbenchmarking::read_repository_index($repo);
 
 			echo $i . '/' . $total_count . ': ' . ($repo_index['tests'][$test]['title'] != null ? $repo_index['tests'][$test]['title'] . ' [' . $repo_index['tests'][$test]['test_type'] . ']' : null) . PHP_EOL;
-			foreach($repo_index['tests'][$test]['versions'] as $version)
+			$versions = $repo_index['tests'][$test]['versions'];
+
+			if(isset($r[0]) && $r[0] == 'lean')
+			{
+				$versions = array(array_shift($versions));
+			}
+
+			foreach($versions as $version)
 			{
 				$qualified_identifier = $repo . '/' . $test . '-' . $version;
 				echo $qualified_identifier;

pts-core/commands/openbenchmarking_changes.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2011 - 2018, Phoronix Media
-	Copyright (C) 2011 - 2018, Michael Larabel
+	Copyright (C) 2011 - 2019, Phoronix Media
+	Copyright (C) 2011 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -25,6 +25,10 @@ class openbenchmarking_changes implements pts_option_interface
 	const doc_section = 'OpenBenchmarking.org';
 	const doc_description = 'This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.';
 
+	public static function command_aliases()
+	{
+		return array('recently_updated_tests');
+	}
 	public static function run($r)
 	{
 		pts_client::$display->generic_heading('Recently Updated OpenBenchmarking.org Tests');

pts-core/commands/recently_added_tests.php

@@ -0,0 +1,45 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2011 - 2019, Phoronix Media
+	Copyright (C) 2011 - 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class recently_added_tests implements pts_option_interface
+{
+	const doc_section = 'OpenBenchmarking.org';
+	const doc_description = 'This option will list the most recently added (newest) test profiles.';
+
+	public static function command_aliases()
+	{
+		return array('newest_tests');
+	}
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('Recently Added OpenBenchmarking.org Tests');
+		foreach(pts_openbenchmarking_client::recently_added_tests(20) as $added => $test_identifier)
+		{
+			$test_profile = new pts_test_profile($test_identifier);
+			echo sprintf('%-36ls - %-35ls', pts_client::cli_just_bold($test_profile->get_identifier(false)), $test_profile->get_title()) . PHP_EOL;
+			echo sprintf('%-36ls    %-35ls', pts_client::cli_just_italic(date('d F Y', $added)), $test_profile->get_test_hardware_type() . ' Test') . PHP_EOL . PHP_EOL;
+		}
+		echo PHP_EOL;
+	}
+}
+
+?>

pts-core/commands/result_file_raw_to_csv.php

@@ -0,0 +1,47 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2019, Phoronix Media
+	Copyright (C) 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class result_file_raw_to_csv implements pts_option_interface
+{
+	const doc_section = 'Result Management';
+	const doc_description = 'This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
+		);
+	}
+	public static function run($r)
+	{
+		$result_file = new pts_result_file($r[0]);
+		$result_output = pts_result_file_output::result_file_raw_to_csv($result_file);
+
+		// To save the result:
+		$file = pts_core::user_home_directory() . $r[0] . '-raw.csv';
+		file_put_contents($file, $result_output);
+
+		echo PHP_EOL . pts_client::cli_just_bold('Saved To: ') . $file . PHP_EOL;
+	}
+}
+
+?>

pts-core/commands/result_file_stats.php

@@ -0,0 +1,54 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2019, Phoronix Media
+	Copyright (C) 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class result_file_stats implements pts_option_interface
+{
+	const doc_section = 'Result Management';
+	const doc_description = 'This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.';
+
+	public static function command_aliases()
+	{
+		return array('winners_and_losers', 'result_stats');
+	}
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
+		);
+	}
+	public static function run($args)
+	{
+		$result_file = new pts_result_file($args[0]);
+		echo '   ' . pts_client::cli_colored_text($result_file->get_title(), 'gray', true) . PHP_EOL . PHP_EOL;
+
+		if($result_file->get_system_count() < 2)
+		{
+			echo PHP_EOL . 'There are not multiple test runs in this result file.' . PHP_EOL;
+			return false;
+		}
+
+		echo pts_result_file_analyzer::display_results_wins_losses($result_file);
+		echo pts_result_file_analyzer::display_result_file_stats_pythagorean_means($result_file);
+	}
+}
+
+?>

pts-core/commands/result_file_to_csv.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2018, Phoronix Media
-	Copyright (C) 2009 - 2018, Michael Larabel
+	Copyright (C) 2009 - 2019, Phoronix Media
+	Copyright (C) 2009 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -40,7 +40,7 @@ class result_file_to_csv implements pts_option_interface
 		$file = pts_core::user_home_directory() . $r[0] . '.csv';
 		file_put_contents($file, $result_output);
 
-		echo $result_output;
+		echo PHP_EOL . pts_client::cli_just_bold('Saved To: ') . $file . PHP_EOL;
 	}
 }
 

pts-core/commands/result_file_to_pdf.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2018, Phoronix Media
-	Copyright (C) 2009 - 2018, Michael Larabel
+	Copyright (C) 2009 - 2019, Phoronix Media
+	Copyright (C) 2009 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,10 +33,10 @@ class result_file_to_pdf implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		if(!function_exists('getimagesizefromstring'))
+		if(!function_exists('getimagesizefromstring') || !function_exists('imagettftext') || !extension_loaded('gd'))
 		{
-			echo 'PHP GD support is required for this command.' . PHP_EOL . PHP_EOL;
-			return false;
+			echo pts_client::cli_just_bold('PHP GD support and TTF support are required for the PDF output if wanting to display the benchmark result graphs.') . PHP_EOL . PHP_EOL;
+			//return false;
 		}
 
 		$_REQUEST['force_format'] = 'PNG'; // Force to PNG renderer
@@ -47,7 +47,7 @@ class result_file_to_pdf implements pts_option_interface
 		$result_file = new pts_result_file($r[0]);
 		$pdf_file = pts_core::user_home_directory() . $r[0] . '.pdf';
 		$pdf_output = pts_result_file_output::result_file_to_pdf($result_file, $pdf_file, 'F');
-		echo PHP_EOL . 'Saved To: ' . $pdf_file . PHP_EOL;
+		echo PHP_EOL . pts_client::cli_just_bold('Saved To: ') . $pdf_file . PHP_EOL;
 	}
 }
 

pts-core/commands/result_file_to_text.php

@@ -34,7 +34,7 @@ class result_file_to_text implements pts_option_interface
 	public static function run($r)
 	{
 		$result_file = new pts_result_file($r[0]);
-		$result_output = pts_result_file_output::result_file_to_text($result_file, pts_client::terminal_width());
+		$result_output = pts_result_file_output::result_file_to_text($result_file, pts_client::terminal_width(), true);
 		echo $result_output;
 	}
 }

pts-core/commands/run_random_tests.php

@@ -145,8 +145,8 @@ class run_random_tests implements pts_option_interface
 					$test_run_manager->auto_save_results($title, null, 'Various open-source benchmarks by the ' . pts_core::program_title(true) . '.', true);
 					$test_run_manager->auto_generate_results_identifier();
 					echo PHP_EOL;
-					pts_client::$display->generic_sub_heading('Result File: ' . $test_run_manager->get_file_name());
-					pts_client::$display->generic_sub_heading('Result Identifier: ' . $test_run_manager->get_results_identifier());
+					pts_client::$display->generic_sub_heading(pts_client::cli_just_bold('Result File: ') . $test_run_manager->get_file_name());
+					pts_client::$display->generic_sub_heading(pts_client::cli_just_bold('Result Identifier: ') . $test_run_manager->get_results_identifier());
 
 					// BENCHMARK
 					$test_run_manager->pre_execution_process();

pts-core/commands/search.php

@@ -32,6 +32,7 @@ class search implements pts_option_interface
 
 		$table = array();
 		$tests = array();
+		$results_found = false;
 		foreach(pts_search::search_test_profiles($search_query) as $test_profile)
 		{
 			$table[] = array(pts_client::cli_just_bold($test_profile->get_identifier()), $test_profile->get_title(), $test_profile->get_test_hardware_type());
@@ -45,6 +46,7 @@ class search implements pts_option_interface
 		}
 		if(count($table) > 0)
 		{
+			$results_found = true;
 			echo pts_client::cli_colored_text('TEST PROFILES', 'green', true) . PHP_EOL . pts_user_io::display_text_table($table, null, 1) . PHP_EOL . pts_client::cli_colored_text(pts_strings::plural_handler(count($table), 'Test') . ' Matching', 'gray');
 
 			$table = array();
@@ -66,6 +68,7 @@ class search implements pts_option_interface
 		}
 		if(count($table) > 0)
 		{
+			$results_found = true;
 			echo PHP_EOL . PHP_EOL . pts_client::cli_colored_text('TEST SUITES', 'green', true) . PHP_EOL . pts_user_io::display_text_table($table, null, 1) . PHP_EOL . pts_client::cli_colored_text(pts_strings::plural_handler(count($table), 'Suite') . ' Matching', 'gray');
 		}
 
@@ -77,9 +80,15 @@ class search implements pts_option_interface
 		}
 		if(count($table) > 0)
 		{
+			$results_found = true;
 			echo PHP_EOL . PHP_EOL . pts_client::cli_colored_text('TEST RESULTS', 'green', true) . PHP_EOL . pts_user_io::display_text_table($table, null, 1) . PHP_EOL . pts_client::cli_colored_text(pts_strings::plural_handler(count($table), 'Test Result') . ' Matching', 'gray');
 		}
 
+		if(!$results_found)
+		{
+			echo PHP_EOL . pts_client::cli_just_bold('No Search Matches Found');
+		}
+
 		echo PHP_EOL . PHP_EOL;
 	}
 }

pts-core/commands/shell.php

@@ -25,18 +25,87 @@ class shell implements pts_option_interface
 	const doc_section = 'System';
 	const doc_description = 'A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite';
 
+	protected static $auto_completion_cache = null;
+
 	public static function run($r)
 	{
-		pts_openbenchmarking::refresh_repository_lists();
 		pts_client::$display->generic_heading('Interactive Shell');
+		echo 'Generating Shell Cache...' . PHP_EOL;
+		self::$auto_completion_cache = pts_documentation::client_commands_possible_values();
+		echo 'Refreshing OpenBenchmarking.org Repository Cache...' . PHP_EOL;
+		pts_openbenchmarking::refresh_repository_lists();
 		echo phodevi::system_centralized_view();
 		//echo PHP_EOL . (phodevi::read_property('motherboard', 'serial-number') != null ? PHP_EOL . 'System Serial Number: ' . phodevi::read_property('motherboard', 'serial-number') . PHP_EOL : null);
 		echo PHP_EOL;
 
+		$autocompletion = false;
+		if(function_exists('readline') && function_exists('readline_completion_function'))
+		{
+			$autocompletion = ' Tab auto-completion support available.';
+		}
+
+		$blacklisted_commands = array('shell', 'quit', 'exit');
+		do
+		{
+			self::sensor_overview();
+			echo PHP_EOL . 'Phoronix Test Suite command to run or ' . pts_client::cli_colored_text('help', 'green') . ' for all possible options, ' . pts_client::cli_colored_text('commands', 'green') . ' for a quick overview of options, ' . pts_client::cli_colored_text('interactive', 'green') . ' for a guided experience, ' . pts_client::cli_colored_text('system-info', 'green') . ' to view system hardware/software information, ' . pts_client::cli_colored_text('exit', 'green') . ' to exit. For new users, ' . pts_client::cli_colored_text('benchmark', 'green') . ' is the simplest and most important sub-command.' . ($autocompletion ? $autocompletion : '') . PHP_EOL;
+			echo PHP_EOL . pts_client::cli_colored_text((phodevi::is_root() ? '#' : '$'), 'white') . ' ' . pts_client::cli_colored_text('phoronix-test-suite', 'gray') . ' ';
+			if($autocompletion)
+			{
+				readline_completion_function(array('shell', 'shell_auto_completion_handler'));
+				$input = readline();
+			}
+			else
+			{
+				$input = pts_user_io::read_user_input();
+			}
+			$argv = explode(' ', $input);
+			if($argv[0] == 'phoronix-test-suite')
+			{
+				array_shift($argv);
+			}
+			$argc = count($argv);
+			$sent_command = strtolower(str_replace('-', '_', (isset($argv[0]) ? $argv[0] : null)));
+			if(!in_array($sent_command, $blacklisted_commands))
+			{
+				$passed = pts_client::handle_sent_command($sent_command, $argv, $argc);
+
+				if(!$passed)
+				{
+					if(empty($argv[0]))
+					{
+						echo PHP_EOL . pts_client::cli_colored_text('Enter command to run.', 'red', true) . PHP_EOL;
+					}
+					else
+					{
+						echo PHP_EOL . pts_client::cli_colored_text('Unsupported command: ' . $argv[0], 'red', true) . PHP_EOL;
+					}
+				}
+				else
+				{
+					if($autocompletion)
+					{
+						readline_add_history($input);
+					}
+
+					$pass_args = array();
+					for($i = 1; $i < $argc; $i++)
+					{
+						$pass_args[] = $argv[$i];
+					}
+
+					pts_client::execute_command($sent_command, $pass_args); // Run command
+				}
+			}
+		}
+		while($sent_command != 'exit' && $sent_command != 'quit');
+	}
+	protected static function sensor_overview()
+	{
 		// SENSORS
 		$terminal_width = pts_client::terminal_width();
 		$sensors = array();
-		foreach(phodevi::supported_sensors(array('cpu_usage', 'cpu_temp', 'sys_temp', 'gpu_usage', 'gpu_temp', 'memory_usage')) as $sensor)
+		foreach(phodevi::supported_sensors(array('cpu_usage', 'cpu_temp', 'sys_temp', 'sys_power', 'gpu_usage', 'gpu_temp', 'memory_usage')) as $sensor)
 		{
 			$supported_devices = call_user_func(array($sensor[2], 'get_supported_devices'));
 
@@ -54,16 +123,20 @@ class shell implements pts_option_interface
 			{
 				$sensor_object = new $sensor[2](0, $device);
 				$sensor_value = phodevi::read_sensor($sensor_object);
-				if($sensor_value < 0)
+				if($sensor_value < 0 || empty($sensor_value))
 				{
 					continue;
 				}
 
 				$sensor_name = phodevi::sensor_object_name($sensor_object) . ':';
-				$sensor_unit = phodevi::read_sensor_object_unit($sensor_object);
+				$sensor_unit = phodevi::read_sensor_object_unit_short($sensor_object);
 				$sensors[] = array($sensor_name, $sensor_value, $sensor_unit);
 			}
 		}
+		if(($uptime = phodevi::system_uptime()) > 0)
+		{
+			$sensors[] = array('System Uptime', round($uptime / 60), 'M');
+		}
 		$longest = array();
 		foreach($sensors as $ar)
 		{
@@ -78,6 +151,7 @@ class shell implements pts_option_interface
 		$sensor_length = array_sum($longest);
 		$sensors_per_line = floor($terminal_width / $sensor_length);
 
+		echo PHP_EOL;
 		$i = 0;
 		foreach($sensors as $sensor_data)
 		{
@@ -93,32 +167,39 @@ class shell implements pts_option_interface
 
 		echo PHP_EOL;
 		// END OF SENSORS
-		$blacklisted_commands = array('shell', 'quit', 'exit');
-		do
-		{
-			echo PHP_EOL . 'Phoronix Test Suite command to run or ' . pts_client::cli_colored_text('help', 'green') . ' for help, ' . pts_client::cli_colored_text('commands', 'green') . ' for possible options, ' . pts_client::cli_colored_text('interactive', 'green') . ' for a guided experience, ' . pts_client::cli_colored_text('exit', 'green') . ' to exit: ' . PHP_EOL;
-			echo PHP_EOL . pts_client::cli_colored_text((phodevi::is_root() ? '#' : '$'), 'white') . ' ' . pts_client::cli_colored_text('phoronix-test-suite', 'gray') . ' ';
-			$input = pts_user_io::read_user_input();
-			$argv = explode(' ', $input);
-			if($argv[0] == 'phoronix-test-suite')
-			{
-				array_shift($argv);
-			}
-			$argc = count($argv);
-			$sent_command = strtolower(str_replace('-', '_', (isset($argv[0]) ? $argv[0] : null)));
-			if(!in_array($sent_command, $blacklisted_commands))
-			{
-				pts_client::handle_sent_command($sent_command, $argv, $argc);
-				$pass_args = array();
-				for($i = 1; $i < $argc; $i++)
-				{
-					$pass_args[] = $argv[$i];
-				}
+	}
+	protected static function shell_auto_completion_handler($input)
+	{
+		$possibilities = array();
+		$readline_info = readline_info();
+		$input = isset($readline_info['end']) ? substr($readline_info['line_buffer'], 0, $readline_info['end']) : $readline_info['line_buffer'];
+		$input_length = strlen($input);
+		$possible_sub_commands = pts_client::possible_sub_commands();
 
-				pts_client::execute_command($sent_command, $pass_args); // Run command
+		$argv = explode(' ', trim($input));
+
+		if(count($argv) == 1 && !isset(self::$auto_completion_cache[$argv[0]]))
+		{
+			foreach($possible_sub_commands as $possibility)
+			{
+				if(substr($possibility, 0, $input_length) === $input)
+				{
+					$possibilities[] = $possibility;
+				}
 			}
 		}
-		while($sent_command != 'exit');
+		else
+		{
+			$targeted_sub_command = $argv[0];
+			if(isset(self::$auto_completion_cache[$argv[0]]))
+			{
+				$possibilities = self::$auto_completion_cache[$argv[0]];
+			}
+		}
+
+		//$possibilities[] = '';
+		sort($possibilities);
+		return $possibilities;
 	}
 }
 

pts-core/commands/strict_benchmark.php

@@ -0,0 +1,49 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2019, Phoronix Media
+	Copyright (C) 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class strict_benchmark implements pts_option_interface
+{
+	const doc_section = 'Testing';
+	const doc_description = 'This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.';
+
+	public static function command_aliases()
+	{
+		return array('test');
+	}
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check('VARIABLE_LENGTH', array('pts_types', 'identifier_to_object'), null)
+		);
+	}
+	public static function run($r)
+	{
+		putenv('FORCE_TIMES_TO_RUN_MULTIPLE=2');
+		putenv('FORCE_MIN_TIMES_TO_RUN=20');
+		putenv('FORCE_MIN_TIMES_TO_RUN_CUTOFF=8');
+		pts_test_installer::standard_install($r);
+		$run_manager = new pts_test_run_manager();
+		$run_manager->standard_run($r);
+	}
+}
+
+?>

pts-core/commands/system_sensors.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2017, Phoronix Media
-	Copyright (C) 2009 - 2017, Michael Larabel
+	Copyright (C) 2009 - 2019, Phoronix Media
+	Copyright (C) 2009 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -47,7 +47,7 @@ class system_sensors implements pts_option_interface
 				else
 				{
 					$sensor_object = new $sensor[2](0, $device);
-					$tabled[] = array(pts_client::cli_just_bold(phodevi::sensor_object_name($sensor_object) . ': '), phodevi::read_sensor($sensor_object), pts_client::cli_colored_text(phodevi::read_sensor_object_unit($sensor_object), 'gray'));
+					$tabled[] = array(pts_client::cli_just_italic(phodevi::sensor_object_identifier($sensor_object)) . ' ', pts_client::cli_just_bold(phodevi::sensor_object_name($sensor_object) . ': '), phodevi::read_sensor($sensor_object), pts_client::cli_colored_text(phodevi::read_sensor_object_unit($sensor_object), 'gray'));
 				}
 			}
 		}

pts-core/commands/winners_and_losers.php

@@ -1,102 +0,0 @@
-<?php
-
-/*
-	Phoronix Test Suite
-	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2017 - 2018, Phoronix Media
-	Copyright (C) 2017 - 2018, Michael Larabel
-
-	This program is free software; you can redistribute it and/or modify
-	it under the terms of the GNU General Public License as published by
-	the Free Software Foundation; either version 3 of the License, or
-	(at your option) any later version.
-
-	This program is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-	GNU General Public License for more details.
-
-	You should have received a copy of the GNU General Public License
-	along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-class winners_and_losers implements pts_option_interface
-{
-	const doc_section = 'Result Management';
-	const doc_description = 'This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.';
-
-	public static function argument_checks()
-	{
-		return array(
-		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
-		);
-	}
-	public static function run($args)
-	{
-		$result_file = new pts_result_file($args[0]);
-		$result_file_identifiers = $result_file->get_system_identifiers();
-
-		if(count($result_file_identifiers) < 2)
-		{
-			echo PHP_EOL . 'There are not multiple test runs in this result file.' . PHP_EOL;
-			return false;
-		}
-
-		echo  pts_client::cli_colored_text($result_file->get_title(), 'gray', true) . PHP_EOL;
-		echo  pts_client::cli_colored_text('RESULT COUNT: ', 'cyan', true) . $result_file->get_test_count() . PHP_EOL . PHP_EOL;
-		$winners = array();
-		$losers = array();
-		$tests_counted = 0;
-
-		foreach($result_file->get_result_objects() as $result)
-		{
-			if($result->test_result_buffer->get_count() < 2)
-			{
-				continue;
-			}
-
-			$tests_counted++;
-			$winner = $result->get_result_first();
-			$loser = $result->get_result_last();
-
-			if(!isset($winners[$winner]))
-			{
-				$winners[$winner] = 1;
-			}
-			else
-			{
-				$winners[$winner]++;
-			}
-
-			if(!isset($losers[$loser]))
-			{
-				$losers[$loser] = 1;
-			}
-			else
-			{
-				$losers[$loser]++;
-			}
-		}
-
-		arsort($winners);
-		arsort($losers);
-
-		echo  pts_client::cli_colored_text('WINS:', 'green', true) . PHP_EOL;
-		$table = array();
-		foreach($winners as $identifier => $count)
-		{
-			$table[] = array($identifier . ': ', $count, ' [' . pts_math::set_precision($count / $tests_counted * 100, 1) . '%]');
-		}
-		echo pts_user_io::display_text_table($table) . PHP_EOL;
-		echo PHP_EOL .  pts_client::cli_colored_text('LOSSES: ', 'red', true) . PHP_EOL;
-		$table = array();
-		foreach($losers as $identifier => $count)
-		{
-			$table[] = array($identifier . ': ', $count, ' [' . pts_math::set_precision($count / $tests_counted * 100, 1) . '%]');
-		}
-		echo pts_user_io::display_text_table($table) . PHP_EOL;
-		echo PHP_EOL;
-	}
-}
-
-?>

pts-core/commands/wins_and_losses.php

@@ -0,0 +1,53 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017 - 2019, Phoronix Media
+	Copyright (C) 2017 - 2019, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class wins_and_losses implements pts_option_interface
+{
+	const doc_section = 'Result Management';
+	const doc_description = 'This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.';
+
+	public static function command_aliases()
+	{
+		return array('winners_and_losers');
+	}
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
+		);
+	}
+	public static function run($args)
+	{
+		$result_file = new pts_result_file($args[0]);
+		echo '   ' . pts_client::cli_colored_text($result_file->get_title(), 'gray', true) . PHP_EOL . PHP_EOL;
+
+		if($result_file->get_system_count() < 2)
+		{
+			echo PHP_EOL . 'There are not multiple test runs in this result file.' . PHP_EOL;
+			return false;
+		}
+
+		echo pts_result_file_analyzer::display_results_wins_losses($result_file);
+	}
+}
+
+?>

pts-core/external-test-dependencies/dependency-handlers/microsoft_dependency_handler.php

@@ -36,7 +36,7 @@ class microsoft_dependency_handler implements pts_dependency_handler
 			$cwd = getcwd();
 			$cygwin_location = self::get_cygwin();
 			chdir(dirname($cygwin_location));
-			echo PHP_EOL . 'Configuring Cygwin...' . PHP_EOL;
+			echo PHP_EOL . 'Configuring Cygwin for handling of some portable test scripts, extracting .tar.gz/xz files, and other script helpers... More details on the Phoronix Test Suite for Windows @ https://github.com/phoronix-test-suite/phoronix-test-suite/blob/master/documentation/phoronix-test-suite-window.md ' . PHP_EOL;
 			shell_exec(basename($cygwin_location) . ' -q -P unzip -P wget -P bc -P which -W');
 			chdir($cwd);
 		}
@@ -123,17 +123,19 @@ class microsoft_dependency_handler implements pts_dependency_handler
 			foreach($files_to_download as $url)
 			{
 				$download_destination = $download_location . basename($url);
-				echo $url . PHP_EOL . ' - ' . $download_destination . PHP_EOL;
+			//	echo '        ' . $url . ' - ' . $download_destination . PHP_EOL;
 				if(is_file($download_destination))
 				{
 					echo 'File Already Present' . PHP_EOL;
 				}
 				else
 				{
-					echo 'Downloading...' . PHP_EOL;
+					//echo 'Downloading...' . PHP_EOL;
+					$download_package = new pts_test_file_download($url);
+					pts_client::$display->test_install_download_file('DOWNLOAD', $download_package);
 					pts_network::download_file($url, $download_destination);
 				}
-				echo 'Executing...' . PHP_EOL;
+				//echo 'Executing...' . PHP_EOL;
 				shell_exec(basename($url));
 			}
 		}

pts-core/external-test-dependencies/scripts/install-arch-packages.sh

@@ -3,5 +3,5 @@
 # Arch package installation
 
 echo "Please enter your root password below:" 1>&2
-su root -c "pacman -Sy --noconfirm --needed $*"
+su root -c "pacman -Sy --noconfirm --needed --asdeps $*"
 exit

pts-core/external-test-dependencies/scripts/install-debian-packages.sh

@@ -4,6 +4,9 @@ if [ -x /usr/bin/aptitude ]; then
 	# aptitude is nice since it doesn't fail if a non-existant package is hit
 	# See: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503215
 	su -c "aptitude -y install $*"
+elif [ `whoami` = "admin" ] && [ -x /usr/bin/sudo ]; then
+	# Amazon EC2 is admin user, sudo works
+	sudo apt-get -y --ignore-missing install $*
 else
 	su -c "apt-get -y --ignore-missing install $*"
 fi

pts-core/external-test-dependencies/scripts/install-fedora-packages.sh

@@ -8,8 +8,12 @@ elif [ -x /usr/bin/dnf ]; then
 elif [ `whoami` = "ec2-user" ]; then
 	sudo yum -y --skip-broken install $*
 else
-	echo "Please enter your root password below:" 1>&2
-	su root -c "yum -y --skip-broken install $*"
+	echo "Please enter your SUDO password below:" 1>&2 
+	read -s -p "Password: " passwd
+	if ! echo $passwd | sudo -S -p '' yum -y --skip-broken install $*; then
+        	echo "Please enter your ROOT password below:" 1>&2
+		su root -c "yum -y --skip-broken install $*"
+	fi
 fi
 
 exit

pts-core/external-test-dependencies/scripts/install-mandrivalinux-packages.sh

@@ -3,5 +3,11 @@
 # Mandriva package installation
 
 echo "Please enter your root password below:" 1>&2
-su - root -c "urpmi --auto $*"
+
+if which dnf >/dev/null 2>&1 ;
+then
+    su - root -c "dnf -y install $*"
+elif
+	su - root -c "urpmi --auto $*"
+fi
 exit

pts-core/external-test-dependencies/scripts/install-ubuntu-packages.sh

@@ -9,6 +9,7 @@ elif [ `whoami` != "root" ] && [ ! -z "$DISPLAY" ]; then
 	elif [ -x /usr/bin/sudo ]; then
 		ROOT="/usr/bin/sudo"
 	fi
+	$ROOT -- su -c "apt-get -y --ignore-missing install $*"
 elif [ -z "$DISPLAY" ]; then
 	sudo -- apt-get -y --ignore-missing install $*
 else
@@ -20,6 +21,4 @@ fi
 	# aptitude is nice since it doesn't fail if a non-existant package is hit
 	# See: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503215
 #	$ROOT -- "aptitude -y install $*"
-#else
-	$ROOT -- su -c "apt-get -y --ignore-missing install $*"
 #fi

pts-core/external-test-dependencies/xml/alpine-packages.xml

@@ -9,8 +9,8 @@
 		</Information>
 		<Package>
 			<GenericName>common-dependencies</GenericName>
-			<PackageName>unzip</PackageName>
-			<FileCheck>unzip</FileCheck>
+			<PackageName>unzip bash</PackageName>
+			<FileCheck>unzip bash</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>gtk-development</GenericName>
@@ -38,16 +38,12 @@
 		</Package>
 		<Package>
 			<GenericName>build-utilities</GenericName>
-			<PackageName>gcc make build-base automake autoconf</PackageName>
+			<PackageName>gcc make build-base automake autoconf linux-headers</PackageName>
 		</Package>
 		<Package>
 			<GenericName>cairo-development</GenericName>
 			<PackageName>cairo</PackageName>
 		</Package>
-		<Package>
-			<GenericName>xorg-development</GenericName>
-			<PackageName>xorg-server</PackageName>
-		</Package>
 		<Package>
 			<GenericName>tiff</GenericName>
 			<PackageName>tiff</PackageName>
@@ -174,7 +170,7 @@
 		</Package>
 		<Package>
 			<GenericName>python</GenericName>
-			<PackageName>python</PackageName>
+			<PackageName>python python3</PackageName>
 		</Package>
 		<Package>
 			<GenericName>yasm</GenericName>
@@ -208,5 +204,9 @@
 			<GenericName>httpd</GenericName>
 			<PackageName>apache2</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/arch-packages.xml

@@ -6,11 +6,6 @@
 			<Name>Arch Linux</Name>
 			<PackageManager>pacman</PackageManager>
 		</Information>
-		<Package>
-			<GenericName>common-dependencies</GenericName>
-			<PackageName>mesa-demos</PackageName>
-			<FileCheck>glxinfo</FileCheck>
-		</Package>
 		<Package>
 			<GenericName>gtk-development</GenericName>
 			<PackageName>gtk2</PackageName>
@@ -106,7 +101,7 @@
 		</Package>
 		<Package>
 			<GenericName>zlib-development</GenericName>
-			<PackageName>zlib1</PackageName>
+			<PackageName>zlib</PackageName>
 		</Package>
 		<Package>
 			<GenericName>jpeg-development</GenericName>
@@ -116,6 +111,10 @@
 			<GenericName>bc</GenericName>
 			<PackageName>bc</PackageName>
 		</Package>
+		<Package>
+			<GenericName>freeimage</GenericName>
+			<PackageName>freeimage</PackageName>
+		</Package>
 		<Package>
 			<GenericName>perl</GenericName>
 			<PackageName>perl</PackageName>
@@ -145,6 +144,10 @@
 			<GenericName>qt4-development</GenericName>
 			<PackageName>qt4</PackageName>
 		</Package>
+		<Package>
+			<GenericName>qt5-development</GenericName>
+			<PackageName>qt5-base qt5-examples</PackageName>
+		</Package>
 		<Package>
 			<GenericName>autoconf</GenericName>
 			<PackageName>autoconf</PackageName>
@@ -205,11 +208,12 @@
 		</Package>
 		<Package>
 			<GenericName>python</GenericName>
-			<PackageName>python</PackageName>
+			<PackageName>python python-pip</PackageName>
+			<FileCheck>pip3</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>yasm</GenericName>
-			<PackageName>yasm</PackageName>
+			<PackageName>yasm nasm</PackageName>
 		</Package>
 		<Package>
 			<GenericName>lib3ds</GenericName>
@@ -239,6 +243,10 @@
 			<GenericName>openmpi-development</GenericName>
 			<PackageName>openmpi</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>libevent</PackageName>
+		</Package>
 		<Package>
 			<GenericName>dune</GenericName>
 			<PackageName>dune-common dune-grid dune-istl</PackageName>
@@ -287,5 +295,17 @@
 			<GenericName>ruby</GenericName>
 			<PackageName>ruby</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/brew-packages.xml

@@ -230,6 +230,10 @@
 			<GenericName>redis-server</GenericName>
 			<PackageName>redis</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>libevent</PackageName>
+		</Package>
                 <Package>
 			<GenericName>opencv</GenericName>
 			<PackageName>opencv</PackageName>
@@ -246,5 +250,17 @@
 			<GenericName>ruby</GenericName>
 			<PackageName>ruby</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/dragonfly-packages.xml

@@ -50,8 +50,8 @@
                 </Package>
 		<Package>
 			<GenericName>build-utilities</GenericName>
-			<PackageName>autoconf automake gmake bash</PackageName>
-			<FileCheck>autoconf, gmake, bash</FileCheck>
+			<PackageName>autoconf automake gmake bash pkgconf</PackageName>
+			<FileCheck>autoconf, gmake, bash, pkgconf</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>nasm</GenericName>
@@ -139,11 +139,11 @@
 		</Package>
 		<Package>
 			<GenericName>yasm</GenericName>
-			<PackageName>yasm</PackageName>
+			<PackageName>nasm yasm</PackageName>
 		</Package>
 		<Package>
 			<GenericName>gmp-library</GenericName>
-			<PackageName>fgmp</PackageName>
+			<PackageName>gmp</PackageName>
 		</Package>
 		<Package>
 			<GenericName>subversion</GenericName>
@@ -216,5 +216,17 @@
 			<GenericName>Rscript</GenericName>
 			<PackageName>R</PackageName>
 		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
+		<Package>
+			<GenericName>rust</GenericName>
+			<PackageName>rust</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/fedora-packages.xml

@@ -9,8 +9,8 @@
 		</Information>
 		<Package>
 			<GenericName>common-dependencies</GenericName>
-			<PackageName>unzip</PackageName>
-			<FileCheck>unzip</FileCheck>
+			<PackageName>unzip bzip2</PackageName>
+			<FileCheck>unzip, bzip2</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>32bit-compatibility</GenericName>
@@ -64,7 +64,7 @@
 		</Package>
 		<Package>
 			<GenericName>build-utilities</GenericName>
-			<PackageName>gcc gcc-c++ make autoconf automake glibc-static patch</PackageName>
+			<PackageName>gcc gcc-c++ make autoconf automake glibc-static patch expat-devel</PackageName>
 			<FileCheck>gcc, c++</FileCheck>
 		</Package>
 		<Package>
@@ -163,6 +163,10 @@
 			<GenericName>qt4-development</GenericName>
 			<PackageName>qt4-devel</PackageName>
 		</Package>
+		<Package>
+			<GenericName>qt5-development</GenericName>
+			<PackageName>qt5-qtbase-devel qt5-devel</PackageName>
+		</Package>
 		<Package>
 			<GenericName>autoconf</GenericName>
 			<PackageName>autoconf</PackageName>
@@ -171,6 +175,10 @@
 			<GenericName>libtool</GenericName>
 			<PackageName>libtool</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>libevent-devel</PackageName>
+		</Package>
 		<Package>
 			<GenericName>ncurses-development</GenericName>
 			<PackageName>ncurses-devel</PackageName>
@@ -246,7 +254,7 @@
 		</Package>
 		<Package>
 			<GenericName>yasm</GenericName>
-			<PackageName>yasm</PackageName>
+			<PackageName>yasm nasm</PackageName>
 		</Package>
 		<Package>
 			<GenericName>lib3ds</GenericName>
@@ -320,5 +328,17 @@
 			<GenericName>ruby</GenericName>
 			<PackageName>ruby</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/generic-packages.xml

@@ -247,6 +247,12 @@
 			<PossibleNames>libqt4-dev, libqt4-devel, qt4-devel</PossibleNames>
 			<FileCheck>qmake OR qmake-qt4</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>qt5-development</GenericName>
+			<Title>Qt5 Development Files</Title>
+			<PossibleNames>libqt5-dev, qt5-devel</PossibleNames>
+			<FileCheck>qmake</FileCheck>
+		</Package>
 		<Package>
 			<GenericName>autoconf</GenericName>
 			<Title>Autoconf</Title>
@@ -257,7 +263,13 @@
 			<GenericName>libtool</GenericName>
 			<Title>Libtool</Title>
 			<PossibleNames>libtool</PossibleNames>
-			<FileCheck>libtool</FileCheck>
+			<FileCheck>libtool OR libtoolize</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<Title>Libevent</Title>
+			<PossibleNames>libevent, libevent-dev</PossibleNames>
+			<FileCheck>event.h</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>popt</GenericName>
@@ -342,7 +354,7 @@
 			<GenericName>python</GenericName>
 			<Title>Python</Title>
 			<PossibleNames>python</PossibleNames>
-			<FileCheck>python</FileCheck>
+			<FileCheck>python OR python3 OR python2</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>python-boost-development</GenericName>
@@ -495,10 +507,28 @@
 			<FileCheck>v8.h, libv8.so</FileCheck>
 		</Package>
 		<Package>
-			<GenericName>Ruby</GenericName>
+			<GenericName>ruby</GenericName>
 			<Title>Ruby</Title>
 			<PossibleNames>ruby</PossibleNames>
 			<FileCheck>ruby</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<Title>Wine</Title>
+			<PossibleNames>wine</PossibleNames>
+			<FileCheck>wine</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<Title>MongoDB</Title>
+			<PossibleNames>mongodb</PossibleNames>
+			<FileCheck>mongo</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<Title>Node.js + NPM</Title>
+			<PossibleNames>npm, nodejs</PossibleNames>
+			<FileCheck>npm</FileCheck>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/gentoo-packages.xml

@@ -304,9 +304,21 @@
 			<GenericName>V8</GenericName>
 			<PackageName>dev-lang/v8</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>dev-libs/libevent</PackageName>
+		</Package>
 		<Package>
 			<GenericName>ruby</GenericName>
 			<PackageName>dev-lang/ruby</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>virtual/wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>dev-db/mongodb</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/macports-packages.xml

@@ -188,5 +188,9 @@
 			<GenericName>git</GenericName>
 			<PackageName>git-core</PackageName>
 		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/mandrivalinux-packages.xml

@@ -4,7 +4,7 @@
 	<ExternalDependencies>
 		<Information>
 			<Name>Mandriva</Name>
-			<Aliases>Mageia, OpenMandriva</Aliases>
+			<Aliases>Mageia, OpenMandriva, OpenMandrivaLinux</Aliases>
 			<PackageManager>urpmi</PackageManager>
 		</Information>
 		<Package>
@@ -53,8 +53,8 @@
 		</Package>
 		<Package>
 			<GenericName>build-utilities</GenericName>
-			<PackageName>task-c-devel task-c++-devel</PackageName>
-			<FileCheck>/usr/bin/gcc, /usr/bin/bison, /usr/bin/flex</FileCheck>
+			<PackageName>clang automake autoconf bison flex</PackageName>
+			<FileCheck>/usr/bin/clang, /usr/bin/bison, /usr/bin/flex</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>xorg-development</GenericName>
@@ -263,5 +263,13 @@
                         <GenericName>perl-digest-md5</GenericName>
                         <PackageName>perl-Digest-MD5</PackageName>
                 </Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/microsoft-packages.xml

@@ -18,17 +18,17 @@
 		</Package>
 		<Package>
 			<GenericName>java</GenericName>
-			<PackageName>https://github.com/ojdkbuild/ojdkbuild/releases/download/9.0.4-1/java-9-openjdk-9.0.4-1.b11.ojdkbuild.windows.x86_64.msi</PackageName>
+			<PackageName>https://github.com/ojdkbuild/ojdkbuild/releases/download/11.0.1-1/java-11-openjdk-11.0.1.13-1.ojdkbuild.windows.x86_64.msi</PackageName>
 			<FileCheck>C:\Program Files\ojdkbuild OR C:\Program Files\Java</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>golang</GenericName>
-			<PackageName>https://dl.google.com/go/go1.10.windows-amd64.msi</PackageName>
+			<PackageName>https://dl.google.com/go/go1.11.4.windows-amd64.msi</PackageName>
 			<FileCheck>C:\Go</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>perl</GenericName>
-			<PackageName>http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-64bit.msi</PackageName>
+			<PackageName>http://strawberryperl.com/download/5.28.1.1/strawberry-perl-5.28.1.1-64bit.msi</PackageName>
 			<FileCheck>C:\Strawberry\perl\bin</FileCheck>
 		</Package>
 		<Package>
@@ -38,12 +38,12 @@
 		</Package>
 		<Package>
 			<GenericName>python</GenericName>
-			<PackageName>https://www.python.org/ftp/python/3.6.4/python-3.6.4-amd64.exe https://www.python.org/ftp/python/2.7.14/python-2.7.14.amd64.msi python2</PackageName>
+			<PackageName>https://www.python.org/ftp/python/3.7.1/python-3.7.1-amd64.exe https://www.python.org/ftp/python/2.7.15/python-2.7.15.amd64.msi python2 python37 python37-pip</PackageName>
 			<FileCheck>C:\Python27</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>git</GenericName>
-			<PackageName>https://github.com/git-for-windows/git/releases/download/v2.16.2.windows.1/Git-2.16.2-64-bit.exe</PackageName>
+			<PackageName>https://github.com/git-for-windows/git/releases/download/v2.20.1.windows.1/Git-2.20.1-64-bit.exe</PackageName>
 			<FileCheck>C:\Program Files\Git</FileCheck>
 		</Package>
 		<Package>
@@ -61,5 +61,34 @@
 			<PackageName>cmake</PackageName>
 			<FileCheck>C:\cygwin64\bin\cmake.exe</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>https://nodejs.org/dist/latest-v11.x/node-v11.5.0-x64.msi</PackageName>
+			<FileCheck>C:\Program Files\nodejs</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>tesseract</GenericName>
+			<PackageName>https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w64-setup-v4.0.0.20181030.exe</PackageName>
+			<FileCheck>C:\Program Files (x86)\Tesseract-OCR</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>firefox</GenericName>
+			<PackageName>https://ftp.mozilla.org/pub/firefox/releases/65.0.2/win64/en-US/Firefox%20Setup%2065.0.2.exe</PackageName>
+			<FileCheck>C:\Program Files\Mozilla Firefox</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>google-chrome</GenericName>
+			<PackageName></PackageName>
+			<FileCheck>C:\Program Files (x86)\Google\Chrome</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>rust</GenericName>
+			<PackageName>https://static.rust-lang.org/dist/rust-beta-x86_64-pc-windows-msvc.msi</PackageName>
+			<FileCheck>rustc.exe</FileCheck>
+		</Package>
+		<Package>
+			<GenericName>opencv</GenericName>
+			<PackageName>https://github.com/opencv/opencv/releases/download/3.4.3/opencv-3.4.3-vc14_vc15.exe</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/opensuse-packages.xml

@@ -9,7 +9,7 @@
 		</Information>
 		<Package>
 			<GenericName>common-dependencies</GenericName>
-			<PackageName>dmidecode Mesa-demo-x</PackageName>
+			<PackageName>Mesa-demo-x</PackageName>
 			<FileCheck>glxinfo</FileCheck>
 		</Package>
 		<Package>
@@ -306,5 +306,21 @@
 			<GenericName>ruby</GenericName>
 			<PackageName>ruby</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>libevent-devel</PackageName>
+		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/pclinuxos-packages.xml

@@ -238,5 +238,9 @@
 			<PackageName>libgmp3</PackageName>
 			<FileCheck>/usr/include/gmp.h</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/solus-packages.xml

@@ -277,5 +277,9 @@
 			<PackageName>scikit-learn</PackageName>
 			<FileCheck>/usr/lib/python2.7/site-packages/sklearn</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/ubuntu-packages.xml

@@ -73,7 +73,7 @@
 		</Package>
 		<Package>
 			<GenericName>tiff</GenericName>
-			<PackageName>libtiff-dev</PackageName>
+			<PackageName>libtiff5-dev</PackageName>
 		</Package>
 		<Package>
 			<GenericName>bison</GenericName>
@@ -89,7 +89,7 @@
 		</Package>
 		<Package>
 			<GenericName>java</GenericName>
-			<PackageName>openjdk-8-jdk</PackageName>
+			<PackageName>openjdk-11-jdk</PackageName>
 		</Package>
 		<Package>
 			<GenericName>maven</GenericName>
@@ -183,6 +183,11 @@
 			<PackageName>libqt4-dev libqt4-opengl-dev</PackageName>
 			<FileCheck>qmake-qt4, qt4/Qt/qgl.h</FileCheck>
 		</Package>
+		<Package>
+			<GenericName>qt5-development</GenericName>
+			<PackageName>qml-module-qtquick2 qtdeclarative5-dev qt5-default qtbase5-dev</PackageName>
+			<FileCheck>qmake</FileCheck>
+		</Package>
 		<Package>
 			<GenericName>autoconf</GenericName>
 			<PackageName>autoconf</PackageName>
@@ -191,6 +196,10 @@
 			<GenericName>libtool</GenericName>
 			<PackageName>libtool</PackageName>
 		</Package>
+		<Package>
+			<GenericName>libevent</GenericName>
+			<PackageName>libevent-dev</PackageName>
+		</Package>
 		<Package>
 			<GenericName>ncurses-development</GenericName>
 			<PackageName>libncurses5-dev</PackageName>
@@ -205,7 +214,7 @@
 		</Package>
 		<Package>
 			<GenericName>curl</GenericName>
-			<PackageName>libcurl3 curl libcurl4-openssl-dev</PackageName>
+			<PackageName>curl libcurl4-openssl-dev</PackageName>
 		</Package>
 		<Package>
 			<GenericName>fftw3-development</GenericName>
@@ -213,7 +222,7 @@
 		</Package>
 		<Package>
 			<GenericName>blas-development</GenericName>
-			<PackageName>libblas-dev</PackageName>
+			<PackageName>libblas-dev libopenblas-base libopenblas-dev</PackageName>
 		</Package>
 		<Package>
 			<GenericName>lapack-development</GenericName>
@@ -260,7 +269,8 @@
 		</Package>
 		<Package>
 			<GenericName>python</GenericName>
-			<PackageName>python python3-pip</PackageName>
+			<PackageName>python python-pip python3-pip</PackageName>
+			<FileCheck>pip3, python, python3</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>python-boost-development</GenericName>
@@ -273,11 +283,11 @@
 		<Package>
 			<GenericName>python-numpy</GenericName>
 			<PackageName>python-numpy</PackageName>
-			<FileCheck>/usr/share/pyshared/numpy/version.py</FileCheck>
+			<FileCheck>/usr/share/pyshared/numpy/version.py OR /usr/lib/python2.7/dist-packages/numpy/version.py</FileCheck>
 		</Package>
 		<Package>
 			<GenericName>yasm</GenericName>
-			<PackageName>yasm</PackageName>
+			<PackageName>yasm nasm</PackageName>
 		</Package>
 		<Package>
 			<GenericName>gmp-library</GenericName>
@@ -369,5 +379,17 @@
 			<GenericName>darktable</GenericName>
 			<PackageName>darktable</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
+		<Package>
+			<GenericName>mongodb</GenericName>
+			<PackageName>mongodb</PackageName>
+		</Package>
+		<Package>
+			<GenericName>node-npm</GenericName>
+			<PackageName>npm</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/external-test-dependencies/xml/void-packages.xml

@@ -260,5 +260,9 @@
 			<GenericName>golang</GenericName>
 			<PackageName>go</PackageName>
 		</Package>
+		<Package>
+			<GenericName>wine</GenericName>
+			<PackageName>wine</PackageName>
+		</Package>
 	</ExternalDependencies>
 </PhoronixTestSuite>

pts-core/modules/backup.php

@@ -35,7 +35,7 @@ class backup extends pts_module_interface
 	{
 		if(!isset($r[0]) || empty($r[0]))
 		{
-			echo PHP_EOL . pts_client::cli_just_bold('ERROR:') . 'You must pass the name of the backup file to create and/or any absolute path for the said file you would like to create.' . PHP_EOL . PHP_EOL;
+			echo PHP_EOL . pts_client::cli_just_bold('ERROR: ') . 'You must pass the name of the backup file to create and/or any absolute path for the said file you would like to create.' . PHP_EOL . PHP_EOL;
 			return false;
 		}
 		$backup_location = $r[0];

pts-core/modules/graphics_override.php

@@ -50,18 +50,9 @@ class graphics_override extends pts_module_interface
 			shell_exec("nvidia-settings --assign " . $attribute . "=" . $value . " 2>&1");
 		}
 	}
-	public static function set_amd_pcsdb($attribute, $value)
-	{
-		// Sets a value for AMD's PCSDB, Persistent Configuration Store Database
-		if(phodevi::is_ati_graphics() && phodevi::is_linux() && !empty($value))
-		{
-			$DISPLAY = substr(pts_client::read_env("DISPLAY"), 1, 1);
-			$info = shell_exec("DISPLAY=:" . $DISPLAY . " aticonfig --set-pcs-val=" . $attribute . "," . $value . "  2>&1");
-		}
-	}
 	public static function __pre_run_process()
 	{
-		if(!(phodevi::is_nvidia_graphics() || (phodevi::is_ati_graphics() && phodevi::is_linux())))
+		if(!phodevi::is_nvidia_graphics())
 		{
 			echo "\nNo supported driver found for graphics_override module!\n";
 			return pts_module::MODULE_UNLOAD; // Not using a supported driver, quit the module
@@ -100,34 +91,6 @@ class graphics_override extends pts_module_interface
 					self::set_nvidia_extension("FSAAAppControlled", 0);
 				}
 			}
-			else if(phodevi::is_ati_graphics())
-			{
-				self::$preset_aa = phodevi_linux_parser::read_amd_pcsdb("OpenGL,AntiAliasSamples");
-				self::$preset_aa_control = phodevi_linux_parser::read_amd_pcsdb("OpenGL,AAF");
-
-				switch($force_aa)
-				{
-					case 2:
-						$ati_aa = "0x00000002";
-						break;
-					case 4:
-						$ati_aa = "0x00000004";
-						break;
-					case 8:
-						$ati_aa = "0x00000008";
-						break;
-					case 16:
-						echo "\nThe ATI fglrx driver currently does not support 16x AA! Defaulting to 8x AA!\n";
-						$ati_aa = "0x00000008";
-						break;
-				}
-
-				if(isset($ati_aa))
-				{
-					self::set_amd_pcsdb("OpenGL,AntiAliasSamples", $ati_aa);
-					self::set_amd_pcsdb("OpenGL,AAF", "0x00000000");
-				}
-			}
 		}
 
 		if($force_af !== FALSE && in_array($force_af, self::$supported_af_levels))
@@ -160,31 +123,6 @@ class graphics_override extends pts_module_interface
 					self::set_nvidia_extension("LogAnisoAppControlled", 0);
 				}
 			}
-			else if(phodevi::is_ati_graphics())
-			{
-				self::$preset_af = phodevi_linux_parser::read_amd_pcsdb("OpenGL,AnisoDegree");
-
-				switch($force_af)
-				{
-					case 2:
-						$ati_af = "0x00000002";
-						break;
-					case 4:
-						$ati_af = "0x00000004";
-						break;
-					case 8:
-						$ati_af = "0x00000008";
-						break;
-					case 16:
-						$ati_af = "0x00000010";
-						break;
-				}
-
-				if(isset($ati_af))
-				{
-					self::set_amd_pcsdb("OpenGL,AnisoDegree", $ati_af);
-				}
-			}
 		}
 	}
 	public static function __post_option_process()
@@ -202,18 +140,6 @@ class graphics_override extends pts_module_interface
 				self::set_nvidia_extension("LogAnisoAppControlled", self::$preset_af_control);
 			}
 		}
-		else if(phodevi::is_ati_graphics())
-		{
-			if(self::$preset_aa !== FALSE)
-			{
-				self::set_amd_pcsdb("OpenGL,AntiAliasSamples", self::$preset_aa);
-				self::set_amd_pcsdb("OpenGL,AAF", self::$preset_aa_control);
-			}
-			if(self::$preset_af !== FALSE)
-			{
-				self::set_amd_pcsdb("OpenGL,AnisoDegree", self::$preset_af);
-			}
-		}
 	}
 }
 

pts-core/modules/ob_auto_compare.php

@@ -3,7 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2017, Michael Larabel
+	Copyright (C) 2017 - 2019, Phoronix Media
+	Copyright (C) 2017 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -22,11 +23,15 @@
 class ob_auto_compare extends pts_module_interface
 {
 	const module_name = 'OpenBenchmarking.org Auto Comparison';
-	const module_version = '1.0.0';
+	const module_version = '1.3.0';
 	const module_description = 'This module prints comparable OpenBenchmarking.org results in the command-line for reference purposes as tests are being run. OpenBenchmarking.org is automatically queried for results to show based on the test comparison hash and the system type (mobile, desktop, server, cloud, workstation, etc). No other system information or result data is transmitted..';
 	const module_author = 'Michael Larabel';
 
 	private static $response_time = 0;
+	protected static $current_result_file = null;
+
+	private static $longest_args_string_length = 0;
+	private static $archived_percentiles = array();
 
 	public static function user_commands()
 	{
@@ -40,12 +45,13 @@ class ob_auto_compare extends pts_module_interface
 			return;
 		}
 		$result_file = new pts_result_file($r[0]);
+		self::$current_result_file = $r[0];
 		foreach($result_file->get_result_objects() as $result_object)
 		{
 			echo trim($result_object->test_profile->get_title() . ' ' . $result_object->test_profile->get_app_version() . PHP_EOL . $result_object->get_arguments_description()) . PHP_EOL;
 			echo 'COMPARISON HASH:' .  $result_object->get_comparison_hash(true, false) . PHP_EOL;
 			echo 'SYSTEM TYPE: ' . phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()) . PHP_EOL;
-			$auto_comparison_result_file = self::request_compare_from_ob($result_object->get_comparison_hash(), phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()));
+			$auto_comparison_result_file = self::request_compare($result_object, phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()));
 
 			if($auto_comparison_result_file instanceof pts_result_file)
 			{
@@ -63,48 +69,322 @@ class ob_auto_compare extends pts_module_interface
 
 			echo PHP_EOL . PHP_EOL;
 		}
+		self::show_post_run_ob_percentile_summary();
 	}
-	protected static function request_compare_from_ob($comparison_hash, $system_type)
+	protected static function request_compare($result_object, $system_type)
 	{
-		if(!pts_network::internet_support_available() || self::$response_time > 12)
+		$result_file = null;
+
+		if(pts_network::internet_support_available())
 		{
-			// If no network or OB requests are being slow...
-			return false;
+			$comparison_hash = $result_object->get_comparison_hash();
+			$result_file = self::request_compare_from_ob($result_object, $comparison_hash, $system_type);
 		}
 
-		$ob_request_time = time();
-		$json_response = pts_openbenchmarking::make_openbenchmarking_request('auto_compare_via_hash', array('comparison_hash' => $comparison_hash, 'system_type' => $system_type));
-		self::$response_time = time() - $ob_request_time;
-
-
-		$json_response = json_decode($json_response, true);
-
-		if(is_array($json_response) && isset($json_response['openbenchmarking']['result']['composite_xml']))
+		if(empty($result_file))
 		{
-			$composite_xml = $json_response['openbenchmarking']['result']['composite_xml'];
-			if(!empty($composite_xml))
+			$comparison_hash = $result_object->get_comparison_hash(true, false);
+			$result_file = self::request_compare_from_local_results($comparison_hash);
+		}
+
+		return $result_file;
+	}
+	protected static function request_compare_from_local_results($comparison_hash)
+	{
+		$saved_results = pts_client::saved_test_results();
+		shuffle($saved_results);
+
+		foreach($saved_results as $tr)
+		{
+			$result_file = new pts_result_file($tr, true, true);
+
+			if(self::$current_result_file != null && self::$current_result_file == $result_file->get_identifier())
 			{
-				$result_file = new pts_result_file($composite_xml);
-				$result_file->set_reference_id($json_response['openbenchmarking']['result']['public_id']);
+				continue;
+			}
+			if($result_file->get_system_count() < 2)
+			{
+				continue;
+			}
+
+			if($result_file->get_result($comparison_hash) != false)
+			{
+				$result_file->set_reference_id($result_file->get_identifier());
 				return $result_file;
 			}
 		}
 
 		return null;
 	}
+	protected static function show_post_run_ob_percentile_summary()
+	{
+		// self::$archived_percentiles[$result_object->test_profile->get_test_hardware_type()][$result_object->test_profile->get_title()][$result_object->get_arguments_description()] = $this_result_percentile;
+		if(!empty(self::$archived_percentiles))
+		{
+			$tab = '    ';
+			echo PHP_EOL . $tab . pts_client::cli_colored_text('Percentile Classification Of Current Benchmark Run', 'blue', true) . PHP_EOL;
+			foreach(self::$archived_percentiles as $subsystem => $results)
+			{
+				echo $tab . pts_client::cli_just_bold(strtoupper($subsystem)) . PHP_EOL;
+				foreach($results as $test => $tests)
+				{
+					echo $tab . $tab . pts_client::cli_colored_text($test, 'cyan', true) . PHP_EOL;
+					foreach($tests as $args => $values)
+					{
+						echo $tab . $tab . $tab . $args . (!empty($args) ? ':' : ' ') . ' ' . str_repeat(' ', self::$longest_args_string_length + 1 - strlen($args)) . pts_client::cli_just_bold(pts_strings::number_suffix_handler($values)) . PHP_EOL;
+					}
+				}
+			}
+			echo $tab . $tab . $tab . str_repeat(' ', self::$longest_args_string_length + 3) . pts_client::cli_just_italic('OpenBenchmarking.org Percentile') . PHP_EOL;
+		}
+	}
+	protected static function request_compare_from_ob(&$result_object, $comparison_hash, $system_type)
+	{
+		$terminal_width = pts_client::terminal_width();
+		if(!pts_network::internet_support_available() || self::$response_time > 12 || $terminal_width < 60)
+		{
+			// If no network or OB requests are being slow...
+			return false;
+		}
+
+		$rf = null;
+		$ob_request_time = time();
+		$ch = $result_object->get_comparison_hash(true, false);
+		$test_profile = $result_object->test_profile->get_identifier(false);
+		$json_response = pts_openbenchmarking::make_openbenchmarking_request('auto_compare_via_hash', array('comparison_hash' => $comparison_hash, 'system_type' => $system_type, 'test_profile' => $test_profile, 'comparison_hash_string' => $ch));
+		self::$response_time = time() - $ob_request_time;
+
+		$json_response = json_decode($json_response, true);
+
+		if(is_array($json_response))
+		{
+			$active_result = is_object($result_object->active) ? $result_object->active->get_result() : null;
+			if(empty($active_result) && $result_object->test_result_buffer->get_count() == 1)
+			{
+				$v = $result_object->test_result_buffer->get_values();
+				$active_result = array_pop($v);
+			}
+
+			// XXX DEBUG XXX
+			//$v = $result_object->test_result_buffer->get_values();
+			//$active_result = array_pop($v);
+
+			if(is_numeric($active_result) && $active_result > 0 && isset($json_response['openbenchmarking']['result']['ae']['percentiles']) && !empty($json_response['openbenchmarking']['result']['ae']['percentiles']) && isset($json_response['openbenchmarking']['result']['ae']['samples']))
+			{
+				$percentiles = $json_response['openbenchmarking']['result']['ae']['percentiles'];
+				$sample_count = $json_response['openbenchmarking']['result']['ae']['samples'];
+
+				$box_plot = str_repeat(' ', $terminal_width - 4);
+				$box_plot_size = strlen($box_plot);
+				$box_plot = str_split($box_plot);
+				$max_value = array_pop($percentiles);
+				$results_at_pos = array(0, 1, ($box_plot_size - 1));
+
+				// BOX PLOT
+				$whisker_bottom = $percentiles[2];
+				$whisker_top = $percentiles[98];
+				$whisker_start_char = round($whisker_bottom / $max_value * $box_plot_size);
+				$whisker_end_char = round($whisker_top / $max_value * $box_plot_size);
+
+				for($i = $whisker_start_char; $i <= $whisker_end_char && $i < ($box_plot_size - 1); $i++)
+				{
+					$box_plot[$i] = '-';
+				}
+
+				$box_left = floor(($percentiles[25] / $max_value) * $box_plot_size);
+				$box_middle = round(($percentiles[50] / $max_value) * $box_plot_size);
+				$box_right = ceil(($percentiles[75] / $max_value) * $box_plot_size);
+				for($i = $box_left; $i <= $box_right; $i++)
+				{
+					$box_plot[$i] = '#';
+				}
+				$box_plot[$whisker_start_char] = '|';
+				$box_plot[min($whisker_end_char, ($box_plot_size - 1))] = '|';
+				$box_plot[$box_middle] = '!';
+
+				// END OF BOX PLOT
+				if($result_object->test_profile->get_result_proportion() == 'LIB')
+				{
+					$box_plot = array_reverse($box_plot);
+				}
+				$box_plot[0] = '[';
+				$box_plot[($box_plot_size - 1)] = ']';
+
+				$this_result_percentile = -1;
+				foreach($percentiles as $percentile => $v)
+				{
+					if($result_object->test_profile->get_result_proportion() == 'LIB')
+					{
+						if($v > $active_result)
+						{
+							$this_result_percentile = 100 - $percentile ;
+							break;
+						}
+					}
+					else if($v > $active_result)
+					{
+						$this_result_percentile = $percentile - 1;
+						break;
+					}
+				}
+
+				if($this_result_percentile > 0 && $this_result_percentile < 100)
+				{
+					self::$archived_percentiles[$result_object->test_profile->get_test_hardware_type()][$result_object->test_profile->get_title()][$result_object->get_arguments_description_shortened()] = $this_result_percentile;
+					self::$longest_args_string_length = max(self::$longest_args_string_length, strlen($result_object->get_arguments_description_shortened()), strlen($result_object->test_profile->get_title()) - 3);
+				}
+
+				if($active_result < $max_value)
+				{
+					$box_plot_complement = array();
+					for($i = 0; $i < 5; $i++)
+					{
+						$box_plot_complement[$i] = str_repeat(' ', $terminal_width - 4);
+						$box_plot_complement[$i] = str_split($box_plot_complement[$i]);
+
+					}
+
+					$reference_results_added = 0;
+					$this_percentile = pts_strings::number_suffix_handler($this_result_percentile);
+					foreach(array_merge(array('This Result' . ($this_percentile > 0 && $this_percentile < 100 ? ' (' . $this_percentile . ' Percentile)' : null) => ($active_result > 99 ? round($active_result) : $active_result)), (is_array($json_response['openbenchmarking']['result']['ae']['reference_results']) ? $json_response['openbenchmarking']['result']['ae']['reference_results'] : array())) as $component => $value)
+					{
+						if($value > $max_value)
+						{
+							continue;
+						}
+						$this_result_pos = round($value / $max_value * $box_plot_size);
+						if(in_array($this_result_pos, $results_at_pos) || (strpos($component, 'This Result') === false && !in_array($box_plot[$this_result_pos], array(' ', '-', '#'))))
+						{
+							continue;
+						}
+						$results_at_pos[] = $this_result_pos;
+						$results_at_pos[] = $this_result_pos - 1;
+						$results_at_pos[] = $this_result_pos - 2;
+						$results_at_pos[] = $this_result_pos - 3;
+						$results_at_pos[] = $this_result_pos + 1;
+						$results_at_pos[] = $this_result_pos + 2;
+
+						if($result_object->test_profile->get_result_proportion() == 'LIB')
+						{
+							$this_result_pos = $box_plot_size - $this_result_pos;
+						}
+
+						$string_to_show_length = strlen('^ ' . $component . ': ' . $value);
+						if($this_result_pos - $string_to_show_length - 3 > 4)
+						{
+							// print to left
+							$string_to_print = $component . ': ' . $value . ' ^';
+							$write_pos = ($this_result_pos - strlen($string_to_print) + 1);
+						}
+						else if($this_result_pos + $string_to_show_length < $terminal_width)
+						{
+							// print to right of line
+							$string_to_print = '^ ' . $component . ': ' . $value;
+							$write_pos = $this_result_pos;
+						}
+						else
+						{
+							continue;
+						}
+
+						// validate no overwrites
+						$complement_line = ($reference_results_added % 4);
+						if($complement_line == 0 && strpos($component, 'This Result') === false)
+						{
+							$complement_line = 1;
+						}
+						$no_overwrites = true;
+						for($i = $write_pos; $i < ($write_pos + $string_to_show_length) + 1 && isset($box_plot_complement[$complement_line][$i]); $i++)
+						{
+							if($box_plot_complement[$complement_line][$i] != ' ')
+							{
+								$no_overwrites = false;
+								break;
+							}
+						}
+						if($no_overwrites == false)
+						{
+							continue;
+						}
+						// end
+
+						if(strpos($component, 'This Result') !== false)
+						{
+							$brand_color = 'cyan';
+							$string_to_print = pts_client::cli_colored_text($string_to_print, 'cyan', true);
+							$box_plot[$this_result_pos] = pts_client::cli_colored_text('X', 'cyan', true);
+						}
+						else if(($brand_color = pts_render::identifier_to_brand_color($component, null)) != null)
+						{
+							$brand_color = pts_client::hex_color_to_string($brand_color);
+							$string_to_print = pts_client::cli_colored_text($string_to_print, $brand_color, false);
+						}
+						else
+						{
+							$brand_color = null;
+						}
+
+						for($i = $write_pos; $i < ($write_pos + $string_to_show_length) && $i < count($box_plot_complement[$complement_line]); $i++)
+						{
+							$box_plot_complement[$complement_line][$i] = '';
+						}
+
+						$box_plot_complement[$complement_line][$write_pos] = $string_to_print;
+						$box_plot[$this_result_pos] = pts_client::cli_colored_text('*', $brand_color, false);
+						$reference_results_added++;
+					}
+
+					echo PHP_EOL;
+					echo '    ' . pts_client::cli_just_italic('Result compared to ' . pts_client::cli_just_bold(number_format($sample_count)) . ' OpenBenchmarking.org samples; median: ' . pts_client::cli_just_bold(round($percentiles[50], 2)) . '. Box plot of sampling:') . PHP_EOL;
+					echo '    ' . implode('', $box_plot) . PHP_EOL;
+					foreach($box_plot_complement as $line_r)
+					{
+						$line = rtrim(implode('', $line_r));
+						if(!empty($line))
+						{
+							echo '    ' . $line . PHP_EOL;
+						}
+					}
+					$rf = 2; // No reason to show OpenBenchmarking.org Dynamic Comparison as this display is arguably much better
+				}
+			}
+
+			if($rf == null && isset($json_response['openbenchmarking']['result']['composite_xml']))
+			{
+				$composite_xml = $json_response['openbenchmarking']['result']['composite_xml'];
+				if(!empty($composite_xml))
+				{
+					$result_file = new pts_result_file($composite_xml);
+					$result_file->set_reference_id($json_response['openbenchmarking']['result']['public_id']);
+					$rf = $result_file;
+				}
+			}
+		}
+
+		return $rf;
+	}
 	public static function __pre_run_process($test_run_manager)
 	{
 		if(!$test_run_manager->is_interactive_mode())
 		{
 			return pts_module::MODULE_UNLOAD;
 		}
+		self::$current_result_file = $test_run_manager->get_file_name();
+		self::$archived_percentiles = array();
+	}
+	public static function __event_post_run_stats($test_run_manager)
+	{
+		if($test_run_manager->result_file->get_system_count() == 1 && $test_run_manager->result_file->get_test_count() > 3 && !empty(self::$archived_percentiles))
+		{
+			self::show_post_run_ob_percentile_summary();
+		}
 	}
 	public static function __test_run_success_inline_result($result_object)
 	{
 		// Passed is a copy of the successful pts_test_result after showing other inline metrics
 		if($result_object->test_result_buffer->get_count() < 3)
 		{
-			$auto_comparison_result_file = self::request_compare_from_ob($result_object->get_comparison_hash(), phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()));
+			$auto_comparison_result_file = self::request_compare($result_object, phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()));
 
 			if($auto_comparison_result_file instanceof pts_result_file)
 			{
@@ -112,9 +392,10 @@ class ob_auto_compare extends pts_module_interface
 				$ro =  $auto_comparison_result_file->get_result($merge_ch);
 				$ro->sort_results_by_performance();
 				$ro->test_result_buffer->buffer_values_reverse();
-				echo PHP_EOL.pts_client::cli_just_bold('    OpenBenchmarking.org Dynamic Comparison: ');
-				echo pts_result_file_output::test_result_to_text($ro, pts_client::terminal_width(), true, $result_object->test_result_buffer->get_identifiers());
-				echo PHP_EOL . pts_client::cli_just_bold('    Result Perspective:') . ' https://openbenchmarking.org/result/' . $auto_comparison_result_file->get_reference_id() . PHP_EOL;
+				$is_ob_comparison = pts_openbenchmarking::is_string_openbenchmarking_result_id_compliant($auto_comparison_result_file->get_reference_id());
+				echo PHP_EOL.pts_client::cli_just_bold('    ' . ($is_ob_comparison ? 'OpenBenchmarking.org ' : '') . 'Dynamic Comparison: ');
+				echo pts_result_file_output::test_result_to_text($ro, pts_client::terminal_width(), true, $result_object->test_result_buffer->get_identifiers(), false);
+				echo PHP_EOL . pts_client::cli_just_bold('    Result Perspective:') . ' ' . ($is_ob_comparison ? 'https://openbenchmarking.org/result/' : '') . $auto_comparison_result_file->get_reference_id() . PHP_EOL;
 			}
 		}
 	}

pts-core/modules/perf_tips.php

@@ -20,6 +20,11 @@
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 
+// Possibilities:
+// vm.dirty_ratio
+// noatime / nobarrier
+// transparent_hugepages
+
 class pts_perf_tip_msg
 {
 	public $message;
@@ -117,7 +122,7 @@ class perf_tips extends pts_module_interface
 
 			if(stripos($cpu_scaling_governor, 'powersave') !== false)
 			{
-				$perf_tips[] = new pts_perf_tip_msg('The powersave CPU scaling governor is currently in use. It\'s possible to obtain greater performance if using the performance governor.', 'echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor', 'http://openbenchmarking.org/result/1706268-TR-CPUGOVERN32');
+				$perf_tips[] = new pts_perf_tip_msg('The powersave CPU scaling governor is currently in use. It\'s possible to obtain greater performance if using the performance governor.', 'echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor', 'https://openbenchmarking.org/result/1706268-TR-CPUGOVERN32');
 			}
 		}
 

pts-core/modules/pgo.php

@@ -0,0 +1,116 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class pgo extends pts_module_interface
+{
+	const module_name = 'Benchmarking Compiler PGO Impact';
+	const module_version = '1.0.0';
+	const module_description = 'This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.';
+	const module_author = 'Michael Larabel';
+
+	protected static $phase = '';
+	protected static $pgo_storage_dir = '';
+	protected static $stock_cflags = '';
+	protected static $stock_cxxflags = '';
+
+	public static function user_commands()
+	{
+		return array('benchmark' => 'pgo_benchmark');
+	}
+	public static function pgo_benchmark($to_run)
+	{
+		self::$pgo_storage_dir = pts_client::create_temporary_directory('pgo', true);
+		echo 'PGO directory is: ' . self::$pgo_storage_dir . PHP_EOL;
+		self::$stock_cflags = getenv('CFLAGS');
+		self::$stock_cxxflags = getenv('CXXFLAGS');
+
+		// make the initial run manager, collect the result file data we'll need, and run the tests pre-PGO...
+		$run_manager = new pts_test_run_manager();
+		$save_name = $run_manager->prompt_save_name();
+		$result_identifier = $run_manager->prompt_results_identifier();
+		$run_manager->do_skip_post_execution_options();
+
+		// Also force a fresh install before doing any of the PGO-related args...
+		self::$phase = 'PRE_PGO';
+		pts_test_installer::standard_install($to_run, true);
+
+		// run the tests saving PRE-PGO results
+		$run_manager->standard_run($to_run);
+
+		// force install of tests with PGO generation bits...
+		self::$phase = 'GENERATE_PGO';
+
+		// at least some say serial make ends up being better for PGO generation to not confuse the PGO process, the below override ensures -j 1
+		pts_client::override_pts_env_var('NUM_CPU_CORES', 1);
+		pts_client::override_pts_env_var('NUM_CPU_JOBS', 1);
+
+		pts_test_installer::standard_install(array($save_name), true);
+
+		// restore env vars about CPU core/jobs count
+		pts_client::unset_pts_env_var_override('NUM_CPU_CORES');
+		pts_client::unset_pts_env_var_override('NUM_CPU_JOBS');
+
+		// run the tests one time each, not saving the results, in order to generate the PGO profiles...
+		putenv('FORCE_TIMES_TO_RUN=1');
+		$run_manager = new pts_test_run_manager(array('SaveResults' => false, 'RunAllTestCombinations' => false), true);
+		$run_manager->standard_run(array($save_name));
+		putenv('FORCE_TIMES_TO_RUN'); // unset
+
+		// force re-install of tests, in process set PGO using bits -fprofile-dir=/data/pgo -fprofile-use=/data/pgo -fprofile-correction
+		self::$phase = 'USE_PGO';
+		pts_client::override_pts_env_var('NUM_CPU_CORES', 1);
+		pts_client::override_pts_env_var('NUM_CPU_JOBS', 1);
+		pts_test_installer::standard_install(array($save_name), true);
+		pts_client::unset_pts_env_var_override('NUM_CPU_CORES');
+		pts_client::unset_pts_env_var_override('NUM_CPU_JOBS');
+
+		// run the tests saving results with " - PGO" postfix
+		$run_manager = new pts_test_run_manager(array('UploadResults' => false, 'SaveResults' => true, 'PromptForTestDescription' => false, 'RunAllTestCombinations' => false, 'PromptSaveName' => true, 'PromptForTestIdentifier' => true, 'OpenBrowser' => true), true);
+		$run_manager->set_save_name($save_name, false);
+		$run_manager->set_results_identifier($result_identifier . ' - PGO');
+		$run_manager->standard_run(array($save_name));
+
+		// remove PGO files
+		pts_file_io::delete(self::$pgo_storage_dir);
+
+	}
+	public static function __pre_test_install($test_install_request)
+	{
+		$pgo_dir = self::$pgo_storage_dir . $test_install_request->test_profile->get_identifier() . '/';
+		pts_file_io::mkdir($pgo_dir);
+
+		switch(self::$phase)
+		{
+			case 'PRE_PGO':
+				break;
+			case 'GENERATE_PGO':
+				putenv('CFLAGS=' . self::$stock_cflags . ' -fprofile-dir=' . $pgo_dir . ' -fprofile-generate=' . $pgo_dir);
+				putenv('CXXFLAGS=' . self::$stock_cxxflags . ' -fprofile-dir=' . $pgo_dir . ' -fprofile-generate=' . $pgo_dir);
+				break;
+			case 'USE_PGO':
+				putenv('CFLAGS=' . self::$stock_cflags . ' -fprofile-dir=' . $pgo_dir . ' -fprofile-use=' . $pgo_dir . ' -fprofile-correction');
+				putenv('CXXFLAGS=' . self::$stock_cxxflags . ' -fprofile-dir=' . $pgo_dir . ' -fprofile-use=' . $pgo_dir . ' -fprofile-correction');
+				break;
+		}
+	}
+}
+?>

pts-core/modules/phoromatic.php

@@ -465,6 +465,7 @@ class phoromatic extends pts_module_interface
 		$server_setup = self::setup_server_addressing($args);
 		//$http_comm = new phoromatic_client_comm_http();
 
+		/*
 		if(!$server_setup)
 		{
 			if(getenv('PTS_NO_REBOOT_ON_NETWORK_FAILURE') == false && PTS_IS_DAEMONIZED_SERVER_PROCESS)
@@ -474,6 +475,7 @@ class phoromatic extends pts_module_interface
 
 			return false;
 		}
+		*/
 
 		$times_failed = 0;
 		$has_success = false;
@@ -905,7 +907,7 @@ class phoromatic extends pts_module_interface
 			$res = phoromatic::upload_to_remote_server(array(
 				'r' => 'stress_log_upload',
 				'bid' => self::$benchmark_ticket_id,
-				'l' => $stress_log
+				'l' => pts_user_io::strip_ansi_escape_sequences($stress_log)
 				));
 
 			$times_tried++;
@@ -1047,7 +1049,7 @@ class phoromatic extends pts_module_interface
 				));
 		}
 	}
-	public static function __pre_test_install($test_identifier)
+	public static function __pre_test_install(&$test_install_request)
 	{
 	/*	if(self::$has_run_server_setup_func == false)
 		{
@@ -1065,7 +1067,7 @@ class phoromatic extends pts_module_interface
 
 		if(time() > ($last_update_time + 30))
 		{
-			phoromatic::update_system_status('Installing: ' . $test_identifier);
+			phoromatic::update_system_status('Installing: ' . $test_install_request->test_profile->get_identifier());
 			$last_update_time = time();
 		}
 	}

pts-core/modules/system_monitor.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2017, Phoronix Media
-	Copyright (C) 2008 - 2017, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 	system_monitor.php: System sensor monitoring module for PTS
 
 	This program is free software; you can redistribute it and/or modify
@@ -46,7 +46,7 @@ class system_monitor extends pts_module_interface
 	private static $cgroup_enabled_controllers = array();
 
 	private static $test_run_try_number = null;
-	private static $sensor_monitoring_frequency = 2;
+	private static $sensor_monitoring_frequency = 1;
 	private static $test_run_timer = 0;
 	private static $perf_per_watt_collection;
 
@@ -75,7 +75,7 @@ class system_monitor extends pts_module_interface
 	public static function __run_manager_setup(&$test_run_manager)
 	{
 		$test_run_manager->force_results_save();
-		$test_run_manager->disable_dynamic_run_count();
+		//$test_run_manager->disable_dynamic_run_count();
 	}
 
 	public static function __pre_run_process(&$test_run_manager)
@@ -167,7 +167,7 @@ class system_monitor extends pts_module_interface
 			self::process_perf_per_watt($result_file);
 		}
 
-		if(self::$individual_monitoring != false)
+		if(self::$individual_monitoring != false && self::$successful_test_run_request)
 		{
 			foreach(self::$to_monitor as $sensor)
 			{
@@ -193,7 +193,7 @@ class system_monitor extends pts_module_interface
 		{
 			self::process_summary_results($sensor, $test_run_manager);
 		}
-
+		echo PHP_EOL;
 		foreach(self::$monitor_pids as $pid)
 		{
 			if(function_exists('posix_kill'))
@@ -580,7 +580,7 @@ class system_monitor extends pts_module_interface
 	private static function process_perf_per_watt(&$result_file)
 	{
 		$sensor = array('sys', 'power');
-		$sensor_results = self::parse_monitor_log('logs/' . phodevi::sensor_identifier($sensor) . '.0', self::$individual_test_run_offsets[phodevi::sensor_identifier($sensor) . '.0']);
+		$sensor_results = self::parse_monitor_log('logs/' . phodevi::sensor_identifier($sensor), self::$individual_test_run_offsets[phodevi::sensor_identifier($sensor)]);
 
 		if(count($sensor_results) > 2 && self::$successful_test_run_request)
 		{
@@ -610,7 +610,7 @@ class system_monitor extends pts_module_interface
 				{
 					$test_result->test_profile->set_result_scale($test_result->test_profile->get_result_scale() . ' Per Watt');
 					$test_result->test_result_buffer->add_test_result(self::$result_identifier, pts_math::set_precision($test_result->active->get_result() / $watt_average));
-					$result_file->add_result($test_result);
+					$ro = $result_file->add_result_return_object($test_result);
 				}
 				else if($test_result->test_profile->get_result_proportion() == 'LIB')
 				{
@@ -618,7 +618,11 @@ class system_monitor extends pts_module_interface
 					$test_result->test_profile->set_result_proportion('HIB');
 					$test_result->test_profile->set_result_scale('Performance Per Watt');
 					$test_result->test_result_buffer->add_test_result(self::$result_identifier, pts_math::set_precision((1 / $test_result->active->get_result()) / $watt_average));
-					$result_file->add_result($test_result);
+					$ro = $result_file->add_result_return_object($test_result);
+				}
+				if($ro)
+				{
+					pts_client::$display->test_run_success_inline($ro);
 				}
 				self::$perf_per_watt_collection[] = $test_result->active->get_result();
 			}
@@ -631,7 +635,7 @@ class system_monitor extends pts_module_interface
 		if(is_array(self::$perf_per_watt_collection) && count(self::$perf_per_watt_collection) > 2)
 		{
 			// Performance per watt overall
-			$avg = array_sum(self::$perf_per_watt_collection) / count(self::$perf_per_watt_collection);
+			$avg = pts_math::geometric_mean(self::$perf_per_watt_collection);
 			$test_profile = new pts_test_profile();
 			$test_result = new pts_test_result($test_profile);
 			$test_result->test_profile->set_test_title('Meta Performance Per Watt');
@@ -644,7 +648,7 @@ class system_monitor extends pts_module_interface
 			$test_result->set_used_arguments_description('Performance Per Watt');
 			$test_result->set_used_arguments('Per-Per-Watt');
 			$test_result->test_result_buffer = new pts_test_result_buffer();
-			$test_result->test_result_buffer->add_test_result(self::$result_identifier, pts_math::set_precision($avg));
+			$test_result->test_result_buffer->add_test_result(self::$result_identifier, pts_math::set_precision($avg, 4));
 			$test_run_manager->result_file->add_result($test_result);
 		}
 	}
@@ -661,12 +665,17 @@ class system_monitor extends pts_module_interface
 			$sensor_results = self::parse_monitor_log('logs/' . phodevi::sensor_object_identifier($sensor),
 								self::$individual_test_run_offsets[phodevi::sensor_object_identifier($sensor)]);
 
-			if(count($sensor_results) > 0)
+			if(count($sensor_results) > 4)
 			{
 				$result_buffer->add_test_result(self::$result_identifier, implode(',', $sensor_results), implode(',', $sensor_results));
 			}
 		}
 
+		if($result_buffer->get_count() == 0)
+		{
+			return;
+		}
+
 		self::write_test_run_results($result_buffer, $result_file, $sensor);
 		self::$individual_test_run_offsets[phodevi::sensor_object_identifier($sensor)] = array();
 		self::$test_run_tries_offsets[phodevi::sensor_object_identifier($sensor)] = array();
@@ -706,7 +715,11 @@ class system_monitor extends pts_module_interface
 			$test_result->set_used_arguments_description($arguments_try_description);
 		}
 
-		$result_file->add_result($test_result);
+		$ro = $result_file->add_result_return_object($test_result);
+		if($ro)
+		{
+			pts_client::$display->test_run_success_inline($ro);
+		}
 	}
 	private static function prepare_per_try_results(&$sensor, &$result_buffer)
 	{
@@ -749,7 +762,11 @@ class system_monitor extends pts_module_interface
 			$test_result->set_used_arguments(phodevi::sensor_object_identifier($sensor));
 			$test_result->test_result_buffer = new pts_test_result_buffer();
 			$test_result->test_result_buffer->add_test_result(self::$result_identifier, implode(',', $sensor_results), implode(',', $sensor_results), implode(',', $sensor_results), implode(',', $sensor_results));
-			$test_run_manager->result_file->add_result($test_result);
+			$ro = $test_run_manager->result_file->add_result_return_object($test_result);
+			if($ro)
+			{
+				pts_client::$display->test_run_success_inline($ro);
+			}
 		}
 	}
 

pts-core/modules/update_checker.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2018, Phoronix Media
-	Copyright (C) 2008 - 2018, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 	update_checker.php: This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.
 
 	This program is free software; you can redistribute it and/or modify
@@ -41,7 +41,7 @@ class update_checker extends pts_module_interface
 			if(is_numeric($latest_reported_version) && $latest_reported_version > PTS_CORE_VERSION)
 			{
 				// New version of PTS is available
-				pts_client::$display->generic_heading(strtoupper('An outdated version of the Phoronix Test Suite is installed.' . PHP_EOL . 'The version in use is ' . PTS_VERSION . ' (' . PTS_CORE_VERSION . '), but the latest is pts-core ' . $latest_reported_version . '.' . PHP_EOL . 'Visit https://www.phoronix-test-suite.com/ to update this software.'));
+				pts_client::$display->generic_heading(pts_client::cli_just_bold('An outdated version of the Phoronix Test Suite is installed.' . PHP_EOL . 'The version in use is ' . PTS_VERSION . ' (' . PTS_CORE_VERSION . '), but the latest is pts-core ' . $latest_reported_version . '.' . PHP_EOL . 'Visit https://www.phoronix-test-suite.com/ to update this software.'));
 			}
 		}
 

pts-core/modules/use_wine.php

@@ -0,0 +1,227 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class use_wine extends pts_module_interface
+{
+	const module_name = 'Utilize Wine On Linux Benchmarking';
+	const module_version = '1.0.0';
+	const module_description = 'This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.';
+	const module_author = 'Michael Larabel';
+
+	protected static $wine_bin = false;
+	protected static $original_os_under_test = null;
+	public static function module_environmental_variables()
+	{
+		return array('USE_WINE');
+	}
+	public static function __startup()
+	{
+		$use_wine = getenv('USE_WINE');
+		if(is_executable($use_wine) || ($use_wine = pts_client::executable_in_path($use_wine)) !== false)
+		{
+			echo pts_client::cli_just_bold('Using Wine For Benchmarks: ') . $use_wine . PHP_EOL;
+		}
+		else
+		{
+			return pts_module::MODULE_UNLOAD;
+		}
+
+		echo pts_client::cli_just_bold('Wine Version: ') . phodevi::read_property('system', 'wine-version') . PHP_EOL;
+
+		$overrode = pts_tests::override_script_test_execution_handler(array('use_wine', 'test_script_handler'));
+
+		if(!$overrode)
+		{
+			echo 'Failed to override the test script handler.' . PHP_EOL;
+			return pts_module::MODULE_UNLOAD;
+		}
+
+		// Override the operating system string that is queried by test download/installation code for determining the OS-specific code paths in test profiles...
+		self::$original_os_under_test = phodevi::os_under_test();
+		phodevi::os_under_test(true, 'Windows');
+
+		// Set $wine_bin to the Wine binary specified via USE_WINE
+		self::$wine_bin = $use_wine;
+	}
+	public static function __pre_test_install(&$test_install_request)
+	{
+		// Restore the os_under_test back to the original OS type so it will use its native test script if it's explicitly using Wine...
+		if(in_array('wine', $test_install_request->test_profile->get_external_dependencies()))
+		{
+			phodevi::os_under_test(true, self::$original_os_under_test);
+		}
+	}
+	public static function __post_test_install(&$test_install_request)
+	{
+		// Reset the Wine override
+		if(in_array('wine', $test_install_request->test_profile->get_external_dependencies()))
+		{
+			phodevi::os_under_test(true, 'Windows');
+		}
+	}
+	public static function __pre_test_run(&$test_run_request)
+	{
+		// Restore the os_under_test back to the original OS type so it will use its native test script if it's explicitly using Wine...
+		if(in_array('wine', $test_run_request->test_profile->get_external_dependencies()))
+		{
+			phodevi::os_under_test(true, self::$original_os_under_test);
+		}
+	}
+	public static function __post_test_run(&$test_run_request)
+	{
+		// Reset the Wine override
+		if(in_array('wine', $test_run_request->test_profile->get_external_dependencies()))
+		{
+			phodevi::os_under_test(true, 'Windows');
+		}
+	}
+	public static function test_script_handler($test_directory, $shell, $run_file, $pass_argument, $extra_vars, $test_profile)
+	{
+		// Rather than conventional PTS code paths, whenever a pre/install/post/interim test profile script is to be executed, this function will now be called by PTS for its execution
+
+		if(in_array('wine', $test_profile->get_external_dependencies()))
+		{
+			// The test is a wine-focused test already, so don't try to further re-customize it...
+			return -1;
+		}
+
+		// Let's make a temporary file in test_directory so we can modify the script before execution....
+		$new_run_file = dirname($run_file) . '/' . 'wine_' . basename($run_file);
+		touch($new_run_file);
+
+		if(!is_file($run_file))
+		{
+			echo 'Problem finding: ' . $run_file . PHP_EOL;
+			return false;
+		}
+
+		// Let's go through the intended run file line by line and make intended modifications to what's needed for making Wine happy... As we go, write it to the new temporary file
+		$new_script = '';
+		foreach(explode(PHP_EOL, pts_file_io::file_get_contents($run_file)) as $line)
+		{
+			if($line == null)
+			{
+				continue;
+			}
+
+			$words_in_line = pts_strings::trim_explode(' ', $line);
+			if($words_in_line[0] == 'wine')
+			{
+				// if 'wine' is already found in the test profile, assume test is already customized for Wine usage
+
+				// reset $new_script to original script
+				$new_script = pts_file_io::file_get_contents($run_file);
+				break;
+			}
+
+			// Replace /cygdrive/c/ with $WINEPREFIX/drive_c/
+			if (stripos($line, '/cygdrive/c/') !== false)
+			{
+				if (getenv("WINEPREFIX") !== false)
+				{
+					$line = str_replace('/cygdrive/c/', '$WINEPREFIX/drive_c/', $line);
+				}
+				else
+				{
+					$line = str_replace('/cygdrive/c/', '$HOME/.wine/drive_c/', $line);
+				}
+			}
+
+			if($words_in_line[0] == 'msiexec' || $words_in_line[0] == 'msiexec.exe')
+			{
+				// At least in my tests with unigine-valley, calling its msiexec line didn't work and just silently failed... so let's turn that into running the Wine command.
+				$line = str_replace(array('msiexec.exe', 'msiexec'), self::$wine_bin, $line);
+
+				// Remove potential garbage from line...
+				$line = str_replace(array('/package', '/passive'), '', $line);
+			}
+			else if($words_in_line[0] == 'cmd')
+			{
+				$line = self::$wine_bin . ' ' . $line;
+				$line = str_replace('cmd /c', '', $line);
+			}
+			else if($words_in_line[0] == 'cd' && stripos($line, 'C:\\') !== false)
+			{
+				// Try to map a Windows C:\ path into correct directory for Wine...
+
+				// Trim off the "cd " portion of line
+				$cd_dir = substr($line, 3);
+
+				// Get rid of quotes for now... maybe have to change this...
+				$cd_dir = str_replace('"', '', $cd_dir);
+
+				// Map the drive. Using "dosdevices/c:" would be nicer, but the colon makes
+				// this an escaping pain.
+				if (getenv("WINEPREFIX") !== false)
+				{
+					$cd_dir = str_replace('C:\\', '$WINEPREFIX/drive_c/', $cd_dir);
+				}
+				else
+				{
+					$cd_dir = str_replace('C:\\', '$HOME/.wine/drive_c/', $cd_dir);
+				}
+
+				$cd_dir = str_replace('\\ ', ' ', $cd_dir);
+				$cd_dir = str_replace('\\(', '(', $cd_dir);
+				$cd_dir = str_replace('\\)', ')', $cd_dir);
+
+				// Switch over remaining \ to /
+				$cd_dir = str_replace('\\', '/', $cd_dir);
+
+				// escape necessary characters for path
+				$cd_dir = str_replace(' ', '\\ ', $cd_dir);
+				$cd_dir = str_replace(')', '\\)', $cd_dir);
+				$cd_dir = str_replace('(', '\\(', $cd_dir);
+
+				$line = 'cd ' . $cd_dir;
+			}
+			else if(stripos($words_in_line[0], '.exe') !== false || stripos($words_in_line[0], '.msi') !== false)
+			{
+				// Append wine to start of string that appears to be calling an EXE/MSI executable...
+				$line = self::$wine_bin . ' ' . $line;
+				$line = str_replace('cmd /c', '', $line);
+			}
+			else if($words_in_line[0] == './\\$@')
+			{
+				$line = self::$wine_bin . ' ' . $line;
+			}
+
+			$new_script .= $line . PHP_EOL;
+		}
+
+		// Write out the new temporary test script file...
+		file_put_contents($new_run_file, $new_script);
+		shell_exec('chmod +x ' . $new_run_file);
+
+		// Do the actual execution of the script... Albeit the script we modified, not the original test profile script
+		//echo 'cd ' .  $test_directory . ' && ' . $shell . ' ' . $new_run_file . ' ' . $pass_argument . ' 2>&1';
+		$this_result = pts_client::shell_exec('cd ' .  $test_directory . ' && ' . $shell . ' ' . $new_run_file . ' ' . $pass_argument . ' 2>&1', $extra_vars);
+
+		// Remove the temporary script...
+		unlink($new_run_file);
+
+		// Return the result back to pts-core
+		return $this_result;
+	}
+}
+
+?>

pts-core/objects/FPDF.php

@@ -1837,7 +1837,6 @@ function _putresources()
 	$this->_out('>>');
 	$this->_out('endobj');
 }
-
 function _putinfo()
 {
 	$this->_out('/Producer '.$this->_textstring('FPDF '.FPDF_VERSION));
@@ -1920,6 +1919,92 @@ function _enddoc()
 	$this->_out('%%EOF');
 	$this->state = 3;
 }
+// Multi-Cell Table: http://www.fpdf.org/en/script/script3.php
+function Row($data, $widths)
+{
+	//Calculate the height of the row
+	$nb = 0;
+	for($i=0;$i<count($data);$i++)
+	{
+		$nb=max($nb,$this->NbLines($widths[$i],$data[$i]));
+	}
+	$h=5*$nb;
+	//Issue a page break first if needed
+		$this->CheckPageBreak($h);
+//Draw the cells of the row
+	for($i=0;$i<count($data);$i++)
+	{
+		$w=$widths[$i];
+		$a= $i == 0 ? 'R' : 'L';
+		//Save the current position
+		$x=$this->GetX();
+		$y=$this->GetY();
+		//Draw the border
+		$this->Rect($x,$y,$w,$h);
+		//Print the text
+		$this->MultiCell($w,5,$data[$i],0,$a);
+		//Put the position to the right of the cell
+		$this->SetXY($x+$w,$y);
+	}
+	//Go to the next line
+	$this->Ln($h);
+}
+function CheckPageBreak($h)
+{
+	//If the height h would cause an overflow, add a new page immediately
+	if($this->GetY()+$h>$this->PageBreakTrigger)
+		$this->AddPage($this->CurOrientation);
+}
+function NbLines($w,$txt)
+{
+    //Computes the number of lines a MultiCell of width w will take
+    $cw=&$this->CurrentFont['cw'];
+    if($w==0)
+        $w=$this->w-$this->rMargin-$this->x;
+    $wmax=($w-2*$this->cMargin)*1000/$this->FontSize;
+    $s=str_replace("\r",'',$txt);
+    $nb=strlen($s);
+    if($nb>0 and $s[$nb-1]=="\n")
+        $nb--;
+    $sep=-1;
+    $i=0;
+    $j=0;
+    $l=0;
+    $nl=1;
+    while($i<$nb)
+    {
+        $c=$s[$i];
+        if($c=="\n")
+        {
+            $i++;
+            $sep=-1;
+            $j=$i;
+            $l=0;
+            $nl++;
+            continue;
+        }
+        if($c==' ')
+            $sep=$i;
+        $l+=$cw[$c];
+        if($l>$wmax)
+        {
+            if($sep==-1)
+            {
+                if($i==$j)
+                    $i++;
+            }
+            else
+                $i=$sep+1;
+            $sep=-1;
+            $j=$i;
+            $l=0;
+            $nl++;
+        }
+        else
+            $i++;
+    }
+    return $nl;
+}
 // End of class
 }
 

pts-core/objects/client/display_modes/pts_concise_display_mode.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2018, Phoronix Media
-	Copyright (C) 2009 - 2018, Michael Larabel
+	Copyright (C) 2009 - 2019, Phoronix Media
+	Copyright (C) 2009 - 2019, Michael Larabel
 	pts_concise_display_mode.php: The batch / concise display mode
 
 	This program is free software; you can redistribute it and/or modify
@@ -41,6 +41,7 @@ class pts_concise_display_mode implements pts_display_mode_interface
 	private $expected_trial_run_count = 0;
 	private $trial_run_count_current = 0;
 	private $current_saved_test_identifier = null;
+	private $current_test = null;
 
 	public function __construct()
 	{
@@ -158,7 +159,7 @@ class pts_concise_display_mode implements pts_display_mode_interface
 		}
 
 		echo PHP_EOL;
-		if($test_install_request->test_profile->get_status() != 'Verified')
+		if($test_install_request->test_profile->get_status() != 'Verified' && $test_install_request->test_profile->get_status() != '')
 		{
 			echo $this->tab . $this->tab . 'Test Profile Status: ' . pts_client::cli_just_bold($test_install_request->test_profile->get_status()) . PHP_EOL;
 		}
@@ -202,10 +203,10 @@ class pts_concise_display_mode implements pts_display_mode_interface
 		$expected_time = is_numeric($expected_time) && $expected_time > 0 ? pts_strings::format_time($expected_time, 'SECONDS', false, 60) : null;
 
 		// TODO: handle if file-name is too long for terminal width
-		$download_string = $this->tab . $this->tab . $process_string . ': ' . $pts_test_file_download->get_filename();
+		$download_string = $this->tab . $this->tab . pts_client::cli_just_bold($process_string) . ': ' . $pts_test_file_download->get_filename();
 		$download_size_string = $pts_test_file_download->get_filesize() > 0 ? ' [' . self::bytes_to_download_size($pts_test_file_download->get_filesize()) . 'MB]' : null;
 		$offset_length = pts_client::terminal_width() > 1 ? pts_client::terminal_width() : pts_test_file_download::$longest_file_name_length;
-		$offset_length = $offset_length - strlen($download_string) - strlen($download_size_string) - 2;
+		$offset_length = $offset_length - strlen(pts_user_io::strip_ansi_escape_sequences($download_string)) - strlen($download_size_string) - 2;
 
 		if($offset_length < 2)
 		{
@@ -219,7 +220,7 @@ class pts_concise_display_mode implements pts_display_mode_interface
 		$this->progress_line_prefix = $expected_time != null ? 'Estimated Download Time: ' . $expected_time : $progress_prefix;
 		$this->progress_last_float = -1;
 		$this->progress_tab_count = 2;
-		$this->progress_string_length = strlen($download_string);
+		$this->progress_string_length = strlen(pts_user_io::strip_ansi_escape_sequences($download_string));
 	}
 	public function display_interrupt_message($message, $prefix_tag = 'NOTICE', $text_color = 'gray')
 	{
@@ -309,6 +310,7 @@ class pts_concise_display_mode implements pts_display_mode_interface
 	public function test_run_start(&$test_run_manager, &$test_result)
 	{
 		$this->current_saved_test_identifier = $test_run_manager->get_results_identifier();
+		$this->current_test = $test_result->test_profile->get_identifier();
 		$test_title_string = $test_result->test_profile->get_title() . ($test_result->test_profile->get_app_version() != null ? ' ' . $test_result->test_profile->get_app_version() : null) . ':' . PHP_EOL . $this->tab . $test_result->test_profile->get_identifier();
 
 		if(($test_description = $test_result->get_arguments_description()) != false)
@@ -325,6 +327,12 @@ class pts_concise_display_mode implements pts_display_mode_interface
 		$estimated_length = $test_result->test_profile->get_estimated_run_time();
 		$display_table = array();
 
+
+		if($test_result->test_profile->get_status() != 'Verified' && $test_result->test_profile->get_status() != '')
+		{
+			array_push($display_table, array($this->tab . 'Test Profile Status:', pts_client::cli_just_italic($test_result->test_profile->get_status())));
+		}
+
 		array_push($display_table, array($this->tab . 'Estimated Trial Run Count:', $this->expected_trial_run_count));
 
 		if($estimated_length > 1 && $estimated_length != $remaining_length)
@@ -491,10 +499,15 @@ class pts_concise_display_mode implements pts_display_mode_interface
 	}
 	public function test_run_success_inline($test_result_orig)
 	{
+		if($test_result_orig->test_result_buffer->get_count() < 2)
+		{
+			return;
+		}
+
 		$test_result = clone $test_result_orig;
 		$test_result->sort_results_by_performance();
 		$test_result->test_result_buffer->buffer_values_reverse();
-		echo pts_result_file_output::test_result_to_text($test_result, pts_client::terminal_width(), true, $this->current_saved_test_identifier);
+		echo pts_result_file_output::test_result_to_text($test_result, pts_client::terminal_width(), true, $this->current_saved_test_identifier, ($test_result->test_profile->get_identifier() == null || $this->current_test != $test_result->test_profile->get_identifier()));
 		echo PHP_EOL;
 	}
 	public function test_run_error($error_string)
@@ -545,7 +558,7 @@ class pts_concise_display_mode implements pts_display_mode_interface
 	}
 	public function triggered_system_error($level, $message, $file, $line)
 	{
-		$error_msg = PHP_EOL . '[' . $level . '] ';
+		$error_msg = PHP_EOL . $this->tab . '[' . $level . '] ';
 		if(strpos($message, PHP_EOL) === false)
 		{
 			$error_msg .= $message . ' ';

pts-core/objects/client/pts_argument_check.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2017, Phoronix Media
-	Copyright (C) 2009 - 2017, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -44,7 +44,7 @@ class pts_argument_check
 	{
 		return $this->function_return_key;
 	}
-	public function __toString()
+	public function get_function_check_type()
 	{
 		if($this->get_function_check() == array('pts_types', 'is_result_file'))
 		{
@@ -86,11 +86,67 @@ class pts_argument_check
 		{
 			$type = 'Unknown Object';
 		}
+		return $type;
+	}
+	public function __toString()
+	{
+		$type = $this->get_function_check_type();
 
 		$type = '[' . $type . ']' . (($this->get_argument_index() === 'VARIABLE_LENGTH') ? '  ...' : null);
 
 		return $type;
 	}
+	protected static function available_tests()
+	{
+		$ob_tests = pts_openbenchmarking::available_tests(false);
+		$base_tests = array();
+
+		foreach($ob_tests as $t)
+		{
+			$base_tests[] = substr($t, strpos($t, '/') + 1);
+		}
+
+		return array_merge($base_tests, $ob_tests);
+	}
+	public function possible_values()
+	{
+		$possible_values = array();
+
+		if($this->get_function_check() == array('pts_types', 'is_result_file'))
+		{
+			$possible_values = pts_client::saved_test_results();
+		}
+		else if($this->get_function_check() == array('pts_types', 'identifier_to_object'))
+		{
+			$possible_values = array_merge(self::available_tests(), pts_client::saved_test_results(), pts_tests::all_suites(), array_keys(pts_openbenchmarking::result_uploads_from_this_ip()));
+		}
+		else if($this->get_function_check() == array('pts_types', 'is_test_or_suite'))
+		{
+			$possible_values = array_merge(self::available_tests(), pts_tests::all_suites());
+		}
+		else if($this->get_function_check() == array('pts_test_profile', 'is_test_profile'))
+		{
+			$possible_values = self::available_tests();
+		}
+		else if($this->get_function_check() == array('pts_test_suite', 'is_suite'))
+		{
+			$possible_values = pts_tests::all_suites();
+		}
+		else if($this->get_function_check() == array('pts_openbenchmarking', 'is_openbenchmarking_result_id'))
+		{
+			$possible_values = array_keys(pts_openbenchmarking::result_uploads_from_this_ip());
+		}
+		else if($this->get_function_check() == array('pts_result_file', 'is_test_result_file'))
+		{
+			$possible_values = pts_client::saved_test_results();
+		}
+		else if($this->get_function_check() == array('pts_module', 'is_module'))
+		{
+			$possible_values = pts_module_manager::available_modules();
+		}
+
+		return $possible_values;
+	}
 }
 
 ?>

pts-core/objects/client/pts_client.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2018, Phoronix Media
-	Copyright (C) 2008 - 2018, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -30,6 +30,8 @@ class pts_client
 	protected static $phoromatic_servers = array();
 	protected static $debug_mode = false;
 	protected static $full_output = false;
+	protected static $override_pts_env_vars = array();
+	protected static $sent_command = null;
 
 	public static function create_lock($lock_file)
 	{
@@ -47,6 +49,25 @@ class pts_client
 		chmod($lock_file, 0644);
 		return self::$lock_pointers[$lock_file] != false && flock(self::$lock_pointers[$lock_file], LOCK_EX | LOCK_NB);
 	}
+	public static function possible_sub_commands()
+	{
+		static $options = null;
+
+		if(empty($options))
+		{
+			$options = array();
+			foreach(pts_file_io::glob(PTS_COMMAND_PATH . '*.php') as $option_php)
+			{
+				$name = str_replace('_', '-', basename($option_php, '.php'));
+				if(!in_array(pts_strings::first_in_string($name, '-'), array('dump', 'debug', 'task')))
+				{
+					$options[] = $name;
+				}
+			}
+		}
+
+		return $options;
+	}
 	public static function is_locked($lock_file)
 	{
 		$fp = fopen($lock_file, 'w');
@@ -247,6 +268,7 @@ class pts_client
 			'PTS_LAUNCHER' => getenv('PTS_LAUNCHER'),
 			'PHP_BIN' => PHP_BIN,
 			'NUM_CPU_CORES' => phodevi::read_property('cpu', 'core-count'),
+			'NUM_CPU_PHYSICAL_CORES' => phodevi::read_property('cpu', 'physical-core-count'),
 			'NUM_CPU_NODES' => phodevi::read_property('cpu', 'node-count'),
 			'NUM_CPU_JOBS' => (phodevi::read_property('cpu', 'core-count') * 2),
 			'SYS_MEMORY' => phodevi::read_property('memory', 'capacity'),
@@ -259,21 +281,42 @@ class pts_client
 			'OPERATING_SYSTEM' => phodevi::read_property('system', 'vendor-identifier'),
 			'OS_VERSION' => phodevi::read_property('system', 'os-version'),
 			'OS_ARCH' => phodevi::read_property('system', 'kernel-architecture'),
-			'OS_TYPE' => phodevi::operating_system(),
+			'OS_TYPE' => phodevi::os_under_test(),
 			'THIS_RUN_TIME' => PTS_INIT_TIME,
 			'DEBUG_REAL_HOME' => pts_core::user_home_directory(),
+			'DEBUG_PATH' => pts_client::get_path(),
 			'SYSTEM_TYPE_ID' => phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()),
-			'SYSTEM_TYPE' => phodevi_base::system_type_to_string(phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software()))
+			'SYSTEM_TYPE' => phodevi_base::system_type_to_string(phodevi_base::determine_system_type(phodevi::system_hardware(), phodevi::system_software())),
+			'TERMINAL_WIDTH' => pts_client::terminal_width(),
+			'C_CXX_FLAGS_DEFAULT' => '-O3 -march=native' // mostly for future use
 			);
 
-			if(!pts_client::executable_in_path('cc') && pts_client::executable_in_path('gcc') && getenv('CC') == false)
+			if(!pts_client::executable_in_path('cc') && getenv('CC') == false)
 			{
 				// This helps some test profiles build correctly if they don't do a cc check internally
-				$env_variables['CC'] = 'gcc';
+				if(pts_client::executable_in_path('gcc'))
+				{
+					$env_variables['CC'] = 'gcc';
+				}
+				else if(pts_client::executable_in_path('clang'))
+				{
+					$env_variables['CC'] = 'clang';
+				}
 			}
 		}
 
-		return $env_variables;
+		return array_merge($env_variables, self::$override_pts_env_vars);
+	}
+	public static function override_pts_env_var($name, $value)
+	{
+		self::$override_pts_env_vars[$name] = $value;
+	}
+	public static function unset_pts_env_var_override($name)
+	{
+		if(isset(self::$override_pts_env_vars[$name]))
+		{
+			unset(self::$override_pts_env_vars[$name]);
+		}
 	}
 	public static function test_install_root_path()
 	{
@@ -326,23 +369,35 @@ class pts_client
 	}
 	public static function supports_colored_text_output()
 	{
-		$config_color_option = pts_config::read_user_config('PhoronixTestSuite/Options/General/ColoredConsole', 'AUTO');
+		static $supports = -1;
 
-		switch(strtoupper($config_color_option))
+		if($supports == -1)
 		{
-			case 'TRUE':
-				$supported = true;
-				break;
-			case 'FALSE':
-				$supported = false;
-				break;
-			case 'AUTO':
-			default:
-				$supported = (function_exists('posix_isatty') && posix_isatty(STDOUT)) || (PTS_IS_CLIENT && (getenv('LS_COLORS') || getenv('CLICOLOR'))) || phodevi::is_windows();
-				break;
+			$config_color_option = pts_config::read_user_config('PhoronixTestSuite/Options/General/ColoredConsole', 'AUTO');
+
+			switch(strtoupper($config_color_option))
+			{
+				case 'TRUE':
+					$supported = true;
+					break;
+				case 'FALSE':
+					$supported = false;
+					break;
+				case 'AUTO':
+				default:
+					$supported = (function_exists('posix_isatty') && posix_isatty(STDOUT)) || (PTS_IS_CLIENT && (getenv('LS_COLORS') || getenv('CLICOLOR'))) || (phodevi::is_windows() && strstr(phodevi::read_property('system', 'operating-system'), 'Windows 8') === false && strstr(phodevi::read_property('system', 'operating-system'), 'Windows 7') === false);
+					break;
+			}
+			$supports = $supported;
 		}
 
-		return $supported;
+		return $supports;
+	}
+	public static function hex_color_to_string($hex)
+	{
+		$colors = array();
+		list($colors['red'], $colors['green'], $colors['blue']) = sscanf($hex, "#%02x%02x%02x");
+		return array_search(max($colors), $colors);
 	}
 	public static function cli_colored_text($str, $color, $bold = false)
 	{
@@ -356,10 +411,12 @@ class pts_client
 			'black' => $attribute . ';30',
 			'gray' => '1;30', // gray not bold doesn't look good in all consoles
 			'blue' => $attribute . ';34',
+			'magenta' => $attribute . ';35',
 			'green' => $attribute . ';32',
 			'yellow' => $attribute . ';33',
 			'red' => $attribute . ';31',
 			'cyan' => $attribute . ';36',
+			'white' => $attribute . ';37',
 			);
 
 		if(!isset($colors[$color]))
@@ -378,6 +435,15 @@ class pts_client
 
 		return "\033[1m$str\033[0m";
 	}
+	public static function cli_just_italic($str)
+	{
+		if(!self::supports_colored_text_output())
+		{
+			return $str;
+		}
+
+		return "\e[3m$str\e[23m";
+	}
 	public static function save_test_result($save_to = null, $save_results = null, $render_graphs = true, $result_identifier = null)
 	{
 		// Saves PTS result file
@@ -483,7 +549,7 @@ class pts_client
 
 					if(($command_bin = pts_client::executable_in_path($command[0])))
 					{
-						$cmd_output = shell_exec('cd ' . dirname($command_bin) . ' && ./' . $command_string . ' 2>&1');
+						$cmd_output = shell_exec('cd "' . dirname($command_bin) . '" && ./' . basename($command_bin) . ' 2>&1');
 
 						if(strlen($cmd_output) > 900000)
 						{
@@ -827,8 +893,10 @@ class pts_client
 				foreach($ob_relay as $s)
 				{
 					$local_ip = pts_network::get_local_ip();
+					$local_ip_segments = explode('.', $local_ip);
+					$s_segments = explode('.', $s[0]);
 
-					if(substr($local_ip, 0, strrpos($local_ip, '.')) == substr($s[0], 0, strrpos($s[0], '.')))
+					if($s_segments[0] == $local_ip_segments[0] && $s_segments[1] == $local_ip_segments[1])
 					{
 						$possible_servers[] = array($s[0], $s[1]);
 					}
@@ -900,7 +968,7 @@ class pts_client
 			if($prompt_in_method == false || $usage_reporting == -1)
 			{
 				pts_client::$display->generic_heading('User Agreement');
-				echo wordwrap($user_agreement, 65);
+				echo wordwrap($user_agreement, (pts_client::terminal_width() - 2));
 				$agree = pts_user_io::prompt_bool_input('Do you agree to these terms and wish to proceed', -1);
 
 				$usage_reporting = $agree ? pts_user_io::prompt_bool_input('Enable anonymous usage / statistics reporting', -1) : -1;
@@ -1326,12 +1394,11 @@ class pts_client
 					}
 					else
 					{
-						trigger_error('Phoronix Test Suite Argument Missing.', E_USER_ERROR);
+						trigger_error('Phoronix Test Suite Argument Missing', E_USER_ERROR);
 					}
 
-					echo PHP_EOL . pts_client::cli_just_bold('CORRECT SYNTAX:') . PHP_EOL . 'phoronix-test-suite ' . str_replace('_', '-', $command_alias) . ' ' . implode(' ', $argument_checks) . PHP_EOL . PHP_EOL;
-//sort($pass_args);
-					pts_tests::invalid_command_helper($pass_args);
+					echo PHP_EOL . pts_client::cli_just_bold('CORRECT SYNTAX:') . PHP_EOL . 'phoronix-test-suite ' . str_replace('_', '-', $command_alias) . ' ' . pts_client::cli_just_italic(implode(' ', $argument_checks)) . PHP_EOL . PHP_EOL;
+					pts_tests::invalid_command_helper($pass_args, $argument_checks);
 
 					return false;
 				}
@@ -1371,8 +1438,13 @@ class pts_client
 
 		pts_module_manager::module_process('__post_option_process', $command);
 	}
+	public static function get_sent_command()
+	{
+		return self::$sent_command;
+	}
 	public static function handle_sent_command(&$sent_command, &$argv, &$argc)
 	{
+		self::$sent_command = $sent_command;
 		if(is_file(PTS_PATH . 'pts-core/commands/' . $sent_command . '.php') == false)
 		{
 			$replaced = false;
@@ -1409,6 +1481,12 @@ class pts_client
 				$sent_command = 'help';
 			}
 		}
+		else
+		{
+			$replaced = true;
+		}
+
+		return $replaced;
 	}
 	public static function current_command()
 	{
@@ -1418,7 +1496,9 @@ class pts_client
 	{
 		static $terminal_width = null;
 
-		if($terminal_width == null)
+		// XXX As of PTS 8.6.1, no longer caching this as not sure it makes sense to do... So be more responsive about terminal resizing
+		// Or at least cache on Windows where powershell calls can take longer
+		if(!phodevi::is_windows() || $terminal_width == null)
 		{
 			$terminal_width = 80;
 
@@ -1427,7 +1507,7 @@ class pts_client
 				// Powershell defaults to 120
 				$terminal_width = trim(shell_exec('powershell "(get-host).UI.RawUI.MaxWindowSize.width"'));
 			}
-			else if(pts_client::read_env('TERMINAL_WIDTH') != false && is_numeric(pts_client::read_env('TERMINAL_WIDTH')) >= 80)
+			else if(pts_client::read_env('TERMINAL_WIDTH') != false && is_numeric(pts_client::read_env('TERMINAL_WIDTH')) >= 20)
 			{
 				$terminal_width = pts_client::read_env('TERMINAL_WIDTH');
 			}
@@ -1435,7 +1515,7 @@ class pts_client
 			{
 				$terminal_width = explode(' ', trim(shell_exec('stty size 2>&1')));
 
-				if(count($terminal_width) == 2 && is_numeric($terminal_width[1]) && $terminal_width[1] >= 80)
+				if(count($terminal_width) == 2 && is_numeric($terminal_width[1]) && $terminal_width[1] >= 40)
 				{
 					$terminal_width = $terminal_width[1];
 				}
@@ -1461,13 +1541,51 @@ class pts_client
 
 		return $terminal_width;
 	}
+	public static function terminal_height()
+	{
+		static $terminal_height = null;
+
+		if($terminal_height == null)
+		{
+			$terminal_height = 12;
+
+			if(phodevi::is_windows())
+			{
+				$terminal_height = trim(shell_exec('powershell "(get-host).UI.RawUI.MaxWindowSize.height"'));
+			}
+			else if(pts_client::executable_in_path('stty'))
+			{
+				$th = explode(' ', trim(shell_exec('stty size 2>&1')));
+
+				if(count($th) == 2 && is_numeric($th[0]) && $th[0] >= 1)
+				{
+					$terminal_height = $th[1];
+				}
+			}
+			else if(pts_client::executable_in_path('tput'))
+			{
+				$th = trim(shell_exec('tput lines 2>&1'));
+
+				if(is_numeric($th) && $th > 1)
+				{
+					$terminal_height = $th;
+				}
+			}
+		}
+
+		return $terminal_height;
+	}
 	public static function is_process_running($process)
 	{
+		$running = null;
 		if(phodevi::is_linux() && pts_client::executable_in_path('ps'))
 		{
 			// Checks if process is running on the system
-			$running = shell_exec('ps -C ' . strtolower($process) . ' 2>&1');
-			$running = trim(str_replace(array('PID', 'TTY', 'TIME', 'CMD'), '', $running));
+			$ps = shell_exec('ps -C ' . strtolower($process) . ' 2>&1');
+			if(strpos($ps, 'unrecognized option') === false)
+			{
+				$running = trim(str_replace(array('PID', 'TTY', 'TIME', 'CMD'), '', $ps));
+			}
 		}
 		else if(phodevi::is_solaris())
 		{
@@ -1479,11 +1597,10 @@ class pts_client
 		{
 			// Checks if process is running on the system
 			$ps = shell_exec('ps -ax 2>&1');
-			$running = strpos($ps, strtolower($process)) != false ? 'TRUE' : null;
-		}
-		else
-		{
-			$running = null;
+			if(strpos($ps, 'unrecognized option') === false)
+			{
+				$running = strpos($ps, strtolower($process)) != false ? 'TRUE' : null;
+			}
 		}
 
 		return !empty($running);
@@ -1600,6 +1717,7 @@ class pts_client
 		if(phodevi::is_windows())
 		{
 			$possible_paths_to_add = array('C:\Users\\' . getenv('USERNAME') . '\AppData\Local\Programs\Python\Python36-32',
+				'C:\Users\\' . getenv('USERNAME') . '\AppData\Local\Programs\Python\Python37',
 				'C:\Python27',
 				'C:\Go\bin',
 				'C:\Strawberry\perl\bin',
@@ -1632,6 +1750,10 @@ class pts_client
 		}
 		return $path;
 	}
+	public static function get_path_separator()
+	{
+		return phodevi::is_windows() ? ';' : ':';
+	}
 	public static function executable_in_path($executable, $ignore_paths_with = false)
 	{
 		static $cache = null;
@@ -1639,7 +1761,7 @@ class pts_client
 		if(!isset($cache[$executable]) || empty($cache[$executable]) || $ignore_paths_with)
 		{
 			$path = pts_client::get_path();
-			$paths = pts_strings::trim_explode((phodevi::is_windows() ? ';' : ':'), $path);
+			$paths = pts_strings::trim_explode(pts_client::get_path_separator(), $path);
 			$executable_path = false;
 
 			foreach($paths as $path)
@@ -1676,7 +1798,7 @@ class pts_client
 	}
 	public static function display_web_page($URL, $alt_text = null, $default_open = true, $auto_open = false)
 	{
-		if((pts_client::read_env('DISPLAY') == false && pts_client::read_env('WAYLAND_DISPLAY') == false && phodevi::is_windows() == false && phodevi::is_macosx() == false) || defined('PHOROMATIC_PROCESS'))
+		if(!phodevi::is_display_server_active() || defined('PHOROMATIC_PROCESS'))
 		{
 			return;
 		}
@@ -1707,7 +1829,7 @@ class pts_client
 					$windows_browsers = array(
 						'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe',
 						'C:\Program Files (x86)\Mozilla Firefox\firefox.exe',
-						'C:\Program Files\internet explorer\iexplore.exe'
+						//'C:\Program Files\internet explorer\iexplore.exe'
 						);
 
 					foreach($windows_browsers as $browser_test)
@@ -1718,13 +1840,31 @@ class pts_client
 							break;
 						}
 					}
-					$browser = escapeshellarg($browser);
+
+					if(!empty($browser))
+					{
+						$browser = escapeshellarg($browser);
+					}
+
 					if(substr($URL, 0, 1) == '\\')
 					{
 						$URL = 'file:///C:' . str_replace('/', '\\', $URL);
 					}
+					else if(substr($URL, 0, 2) == 'C:')
+					{
+						$URL = 'file:///' . str_replace('//', '/', str_replace('\\', '/', $URL));
+					}
 
-					shell_exec($browser . ' "' . $URL . '"');
+
+					if(empty($browser))
+					{
+						// should allow the browser to be opened in Edge
+						shell_exec('start ' . $URL . '');
+					}
+					else
+					{
+						shell_exec($browser . ' "' . $URL . '"');
+					}
 					return;
 				}
 				else

pts-core/objects/client/pts_config.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2018, Phoronix Media
-	Copyright (C) 2008 - 2018, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -118,12 +118,14 @@ class pts_config
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Testing/ResultsDirectory', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Testing/AlwaysUploadResultsToOpenBenchmarking', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Testing/AutoSortRunQueue', $read_config);
+		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Testing/ShowPostRunStatistics', $read_config);
 
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/DynamicRunCount', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/LimitDynamicToTestLength', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/StandardDeviationThreshold', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/ExportResultsTo', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/MinimalTestTime', $read_config);
+		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/TestResultValidation/DropNoisyResults', $read_config);
 
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/BatchMode/SaveResults', $read_config);
 		$config->addXmlNodeFromReader('PhoronixTestSuite/Options/BatchMode/OpenBrowser', $read_config);

pts-core/objects/client/pts_documentation.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2016, Phoronix Media
-	Copyright (C) 2009 - 2016, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -48,7 +48,7 @@ class pts_documentation
 	}
 	public static function client_commands_array()
 	{
-		$options = array('Test Installation' => array(), 'Testing' => array(), 'Batch Testing' => array(), 'OpenBenchmarking.org' => array(), 'System' => array(), 'Information' => array(), 'Asset Creation' => array(), 'Result Management' => array(), 'Result Analytics' => array(), 'Other' => array());
+		$options = array('System' => array(), 'Test Installation' => array(), 'Testing' => array(), 'Batch Testing' => array(), 'OpenBenchmarking.org' => array(), 'Information' => array(), 'Asset Creation' => array(), 'Result Management' => array(), 'Result Analytics' => array(), 'Other' => array());
 
 		foreach(pts_file_io::glob(PTS_COMMAND_PATH . '*.php') as $option_php_file)
 		{
@@ -89,9 +89,52 @@ class pts_documentation
 
 		return $options;
 	}
+	public static function client_commands_possible_values()
+	{
+		static $commands_possible_values = null;
+
+		if(empty($commands_possible_values))
+		{
+			foreach(pts_file_io::glob(PTS_COMMAND_PATH . '*.php') as $option_php_file)
+			{
+				$option_php = basename($option_php_file, '.php');
+				$name = str_replace('_', '-', $option_php);
+
+				if(!in_array(pts_strings::first_in_string($name, '-'), array('task')))
+				{
+					include_once($option_php_file);
+
+					$reflect = new ReflectionClass($option_php);
+					$constants = $reflect->getConstants();
+
+					$args = null;
+					if(method_exists($option_php, 'argument_checks'))
+					{
+						$args = call_user_func(array($option_php, 'argument_checks'));
+					}
+					$command_aliases = array();
+					if(method_exists($option_php, 'command_aliases'))
+					{
+						$command_aliases = call_user_func(array($option_php, 'command_aliases'));
+					}
+					$command_aliases[] = $name;
+					if(isset($args[0]) && $args[0] instanceof pts_argument_check)
+					{
+						$arg_possible_values = $args[0]->possible_values();
+						foreach($command_aliases as $alias)
+						{
+							$commands_possible_values[$alias] = $arg_possible_values;
+						}
+					}
+				}
+			}
+		}
+
+		return $commands_possible_values;
+	}
 	public static function basic_description()
 	{
-		return 'The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.';
+		return 'The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.';
 	}
 }
 

pts-core/objects/client/pts_external_dependencies.php

@@ -392,16 +392,9 @@ class pts_external_dependencies
 			{
 				$present = true;
 			}
-			else
+			else if(pts_client::executable_in_path($file))
 			{
-				foreach(array('/usr/bin/', '/bin/', '/usr/sbin') as $possible_path)
-				{
-					if(is_file($possible_path . $file))
-					{
-						$present = true;
-						break;
-					}
-				}
+				$present = true;
 			}
 
 			if(!$present)
@@ -450,7 +443,7 @@ class pts_external_dependencies
 					if(substr($file[$i], -2) == '.h' || substr($file[$i], -4) == '.hpp')
 					{
 						// May just be a relative header file to look for...
-						$possible_paths = array('/usr/local/include/', '/usr/target/include/', '/usr/include/', '/usr/include/x86_64-linux-gnu/');
+						$possible_paths = array('/usr/local/include/', '/usr/target/include/', '/usr/include/', '/usr/include/x86_64-linux-gnu/', '/usr/include/powerpc64le-linux-gnu/');
 						foreach($possible_paths as $path)
 						{
 							if(is_file($path . $file[$i]) || is_link($path . $file[$i]))
@@ -462,7 +455,7 @@ class pts_external_dependencies
 					else if(strpos($file[$i], '.so') !== false || substr($file[$i], -2) == '.a')
 					{
 						// May just be a relative shared library to look for...
-						$possible_paths = array('/usr/local/lib/', '/usr/lib/', '/usr/lib64/', '/usr/lib/x86_64-linux-gnu/', '/usr/lib/i386-linux-gnu/', '/usr/lib/arm-linux-gnueabihf/');
+						$possible_paths = array('/usr/local/lib/', '/usr/lib/', '/usr/lib64/', '/usr/lib/x86_64-linux-gnu/', '/usr/lib/i386-linux-gnu/', '/usr/lib/arm-linux-gnueabihf/', '/usr/lib/powerpc64le-linux-gnu/');
 
 						if(getenv('LD_LIBRARY_PATH'))
 						{
@@ -501,7 +494,7 @@ class pts_external_dependencies
 		$pkg_vendor = self::vendor_identifier('package-list');
 
 		// Rebuild the array index since some OS package XML tags provide multiple package names in a single string
-		$os_packages_to_install = explode(' ', implode(' ', $os_packages_to_install));
+		$os_packages_to_install = array_unique(explode(' ', implode(' ', $os_packages_to_install)));
 
 		if(is_file($vendor_install_file))
 		{

pts-core/objects/client/pts_logger.php

@@ -55,7 +55,7 @@ class pts_logger
 		if(is_writable($log_file))
 			$this->log_file = $log_file;
 	}
-	public function default_log_file_path()
+	public static function default_log_file_path()
 	{
 		if(is_writable('/var/log') && (PTS_MODE == 'WEB_CLIENT' || defined('PHOROMATIC_SERVER') || defined('PTS_IS_DAEMONIZED_SERVER_PROCESS') || getenv('PTS_SERVER_PROCESS')))
 			$log_file = '/var/log/';
@@ -75,6 +75,7 @@ class pts_logger
 		if($this->log_file == null)
 			return;
 
+		$message = pts_user_io::strip_ansi_escape_sequences($message);
 		file_put_contents($this->log_file, ($date_prefix ? '[' . date('M ' . str_pad(date('j'), 2, ' ', STR_PAD_LEFT) . ' H:i:s Y') . '] ' : null) . $message . PHP_EOL, FILE_APPEND);
 	}
 	public function get_log_file_size()

pts-core/objects/client/pts_module_interface.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2016, Phoronix Media
-	Copyright (C) 2008 - 2016, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 	pts_module_interface.php: The generic Phoronix Test Suite module object that is extended by the specific modules/plug-ins
 
 	This program is free software; you can redistribute it and/or modify
@@ -113,12 +113,12 @@ class pts_module_interface
 	}
 	public static function __pre_test_install($obj = null)
 	{
-		// Passed as the first argument to this function is the test identifier for the name of the test about to be installed
+		// Passed as the first argument to this function is the test install request for the test about to be installed
 		return;
 	}
 	public static function __post_test_install($obj = null)
 	{
-		// Passed as the first argument to this function is the test identifier for the name of the test just installed
+		// Passed as the first argument to this function is the test install request for the test just installed
 		return;
 	}
 	public static function __post_install_process($obj = null)
@@ -211,6 +211,11 @@ class pts_module_interface
 		// Passed is a copy of the pts_test_run_manager
 		return;
 	}
+	public static function __event_post_run_stats($obj = null)
+	{
+		// Passed is a copy of the pts_test_run_manager for showing any post-run statistics
+		return;
+	}
 	public static function __event_user_error($obj = null)
 	{
 		// TODO use __event_user_error

pts-core/objects/client/pts_openbenchmarking_client.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2017, Phoronix Media
-	Copyright (C) 2010 - 2017, Michael Larabel
+	Copyright (C) 2010 - 2019, Phoronix Media
+	Copyright (C) 2010 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -194,7 +194,7 @@ class pts_openbenchmarking_client
 		}
 		if(isset($json_response['openbenchmarking']['upload']['url']))
 		{
-			echo PHP_EOL . pts_client::cli_just_bold('Results Uploaded To: ') . $json_response['openbenchmarking']['upload']['url'] . PHP_EOL;
+			echo PHP_EOL . '    ' . pts_client::cli_just_bold('Results Uploaded To: ') . $json_response['openbenchmarking']['upload']['url'] . PHP_EOL;
 			pts_module_manager::module_process('__event_openbenchmarking_upload', $json_response);
 		}
 		//$json['openbenchmarking']['upload']['id']
@@ -244,6 +244,39 @@ class pts_openbenchmarking_client
 
 		return $available_tests;
 	}
+	public static function recently_added_tests($limit = -1)
+	{
+		$available_tests = array();
+
+		foreach(pts_openbenchmarking::linked_repositories() as $repo)
+		{
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+
+			if(isset($repo_index['tests']) && is_array($repo_index['tests']))
+			{
+				foreach(array_keys($repo_index['tests']) as $identifier)
+				{
+					if($repo_index['tests'][$identifier]['title'] == null)
+					{
+						continue;
+					}
+
+					$version = array_shift($repo_index['tests'][$identifier]['versions']);
+					$add_time = $repo_index['tests'][$identifier]['first_added'];
+					$available_tests[$add_time] = $repo . '/' . $identifier . '-' . $version;
+				}
+			}
+		}
+
+		krsort($available_tests);
+
+		if($limit > 0)
+		{
+			$available_tests = array_slice($available_tests, 0, $limit, true);
+		}
+
+		return $available_tests;
+	}
 	public static function popular_tests($limit = -1, $test_type = null)
 	{
 		$available_tests = array();
@@ -518,6 +551,56 @@ class pts_openbenchmarking_client
 
 		return isset($json['openbenchmarking']['gsid']) ? $json['openbenchmarking']['gsid'] : false;
 	}
+	public static function compare_test_json_download_counts($a, $b)
+	{
+		$a = $a['downloads'];
+		$b = $b['downloads'];
+
+		if($a == $b)
+		{
+			return 0;
+		}
+
+		return ($a > $b) ? -1 : 1;
+	}
+	public static function most_popular_tests($limit = 10)
+	{
+		$only_show_available_cached_tests = pts_network::internet_support_available() == false;
+		$tests = array();
+
+		foreach(pts_openbenchmarking::available_tests(false, false, false, false, $only_show_available_cached_tests) as $identifier)
+		{
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if(!empty($repo_index['tests'][$id]['status']) && $repo_index['tests'][$id]['status'] != 'Verified')
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($repo_index['tests'][$id]['last_updated'] < (time() - (60 * 60 * 24 * 365)))
+			{
+				// Don't show tests not actively maintained
+				continue;
+			}
+			if($repo_index['tests'][$id]['test_type'] == 'Graphics' && !phodevi::is_display_server_active())
+			{
+				// Don't show graphics tests if no display active
+				continue;
+
+			}
+
+			$tests[$id] = $repo_index['tests'][$id];
+		}
+
+		uasort($tests, array('pts_openbenchmarking_client', 'compare_test_json_download_counts'));
+		return array_slice($tests, 0, $limit);
+	}
 }
 
 ?>

pts-core/objects/client/pts_test_execution.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2018, Phoronix Media
-	Copyright (C) 2008 - 2018, Michael Larabel
+	Copyright (C) 2008 - 2019, Phoronix Media
+	Copyright (C) 2008 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -121,16 +121,16 @@ class pts_test_execution
 		{
 			$execute_binary_prepend = $test_run_request->exec_binary_prepend;
 		}
-        else if(getenv('EXECUTE_BINARY_PREPEND') != false)
-        {
-			$execute_binary_prepend = getenv('EXECUTE_BINARY_PREPEND') . ' ';
-        }
+		else if(getenv('EXECUTE_BINARY_PREPEND') != false)
+		{
+				$execute_binary_prepend = getenv('EXECUTE_BINARY_PREPEND') . ' ';
+		}
 
 		if(!$cache_share_present && !$test_run_manager->DEBUG_no_test_execution_just_result_parse && $test_run_request->test_profile->is_root_required())
 		{
 			if(phodevi::is_root() == false)
 			{
-				pts_client::$display->test_run_error('This test must be run as the root / administrator account.');
+				pts_client::$display->test_run_error('This test must be run as root / administrator.');
 			}
 
 			$execute_binary_prepend .= ' ' . PTS_CORE_STATIC_PATH . 'root-access.sh ';
@@ -141,7 +141,7 @@ class pts_test_execution
 			$cache_share = new pts_storage_object(false, false);
 		}
 
-		if($test_run_manager->get_results_identifier() != null && $test_run_manager->get_file_name() != null && pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveTestLogs', 'FALSE'))
+		if($test_run_manager->do_save_results() && $test_run_manager->get_file_name() != null && pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveTestLogs', 'FALSE'))
 		{
 			$backup_test_log_dir = PTS_SAVE_RESULTS_PATH . $test_run_manager->get_file_name() . '/test-logs/active/' . $test_run_manager->get_results_identifier() . '/';
 			pts_file_io::delete($backup_test_log_dir);
@@ -173,6 +173,11 @@ class pts_test_execution
 					pts_client::test_profile_debug_message('Log File: ' . $test_log_file);
 				}
 			}
+			else if(phodevi::is_windows() && strpos($test_directory, ' ') !== false)
+			{
+				// On Windows systems with a space in the directory, to workaround some scripts easiest just punting the log file into temp dir
+				$test_log_file = sys_get_temp_dir() . '\\' . basename($test_identifier) . '-' . $runtime_identifier . '-' . ($i + 1) . '.log';
+			}
 			else
 			{
 				$test_log_file = $test_directory . basename($test_identifier) . '-' . $runtime_identifier . '-' . ($i + 1) . '.log';
@@ -212,9 +217,14 @@ class pts_test_execution
 
 			if(!$test_run_manager->DEBUG_no_test_execution_just_result_parse && $restored_from_cache == false)
 			{
+				if(!phodevi::is_windows() && is_file($to_execute . '/' . $execute_binary) && !is_executable($to_execute . '/' . $execute_binary) && pts_client::executable_in_path('chmod'))
+				{
+					shell_exec('chmod +x ' . $to_execute . '/' . $execute_binary);
+				}
+
 				$test_prepend = getenv('TEST_EXEC_PREPEND') != null ? getenv('TEST_EXEC_PREPEND') . ' ': null;
 				pts_client::$display->test_run_instance_header($test_run_request);
-				sleep(1);
+				sleep(2);
 				$test_run_command = 'cd ' . $to_execute . ' && ' . $test_prepend . $execute_binary_prepend . './' . $execute_binary . ' ' . $pts_test_arguments . ' 2>&1';
 
 				pts_client::test_profile_debug_message('Test Run Command: ' . $test_run_command);
@@ -240,6 +250,10 @@ class pts_test_execution
 						$use_phoroscript = false;
 						$test_extra_runtime_variables['PATH'] = (isset($test_extra_runtime_variables['PATH']) ? $test_extra_runtime_variables['PATH'] : null) . ';C:\cygwin64\bin';
 					}
+					else
+					{
+						$execute_binary = '"' . $execute_binary . '"';
+					}
 				}
 
 				$is_monitoring = pts_test_result_parser::system_monitor_task_check($test_run_request);
@@ -261,7 +275,20 @@ class pts_test_execution
 					{
 						$to_exec = '';
 					}
-					$test_process = proc_open($test_prepend . $to_exec . ' ' . $execute_binary_prepend . './' . $execute_binary . ' ' . $pts_test_arguments . $post_test_args, $descriptorspec, $pipes, $to_execute, array_merge($host_env, pts_client::environmental_variables(), $test_extra_runtime_variables));
+
+					$terv = $test_extra_runtime_variables;
+					if(phodevi::is_windows())
+					{
+						foreach($terv as $terv_i => &$value)
+						{
+							if((is_dir($value) || is_file($value) || $terv_i == 'LOG_FILE') && strpos($value, ' ') !== false)
+							{
+								$value = '"' . $value . '"';
+							}
+						}
+					}
+
+					$test_process = proc_open($test_prepend . $to_exec . ' ' . $execute_binary_prepend . './' . $execute_binary . ' ' . $pts_test_arguments . $post_test_args, $descriptorspec, $pipes, $to_execute, array_merge($host_env, pts_client::environmental_variables(), $terv));
 
 					if(is_resource($test_process))
 					{
@@ -334,7 +361,7 @@ class pts_test_execution
 			}
 			if(in_array(($i + 1), $ignore_runs))
 			{
-				pts_client::$display->test_run_instance_error('Ignoring run result per test profile definition.');
+				pts_client::$display->test_run_instance_error('Ignoring this run result per test profile definition.');
 			}
 			else if($exit_status_pass)
 			{
@@ -395,20 +422,21 @@ class pts_test_execution
 				$increase_run_count = false;
 				$runs_ignored_count = count($ignore_runs);
 
-				if($defined_times_to_run == ($i + 1) && $times_result_produced > 0 && ($times_result_produced + $runs_ignored_count) < $defined_times_to_run && $i < 64)
+				if($defined_times_to_run == ($i + 1) && $times_result_produced > 0 && $times_result_produced < $defined_times_to_run && $i < 64)
 				{
 					// At least one run passed, but at least one run failed to produce a result. Increase count to try to get more successful runs
-					$increase_run_count = $defined_times_to_run - $times_result_produced + $runs_ignored_count;
+					$increase_run_count = $defined_times_to_run - $times_result_produced;
 				}
 				else if($times_result_produced >= 2)
 				{
 					// Dynamically increase run count if needed for statistical significance or other reasons
 					$first_tr = array_slice($test_run_request->generated_result_buffers, 0, 1);
 					$first_tr = array_shift($first_tr);
-					$increase_run_count = $test_run_manager->increase_run_count_check($first_tr->active, $defined_times_to_run, $test_run_time); // XXX maybe check all generated buffers to see if to extend?
+					$increase_run_count = $test_run_manager->increase_run_count_check($test_run_request, $first_tr->active, $defined_times_to_run); // XXX maybe check all generated buffers to see if to extend?
 
 					if($increase_run_count === -1)
 					{
+						self::test_run_error($test_run_manager, $test_run_request, 'This run will not be saved due to noisy result.');
 						$abort_testing = true;
 					}
 					else if($increase_run_count == true)
@@ -436,7 +464,7 @@ class pts_test_execution
 					{
 						pts_client::$display->test_run_instance_output($interim_output);
 					}
-					//sleep(2); // Rest for a moment between tests
+					sleep(2); // Rest for a moment between tests
 				}
 
 				pts_module_manager::module_process('__interim_test_run', $test_run_request);
@@ -532,7 +560,7 @@ class pts_test_execution
 			unset($cache_share);
 		}
 
-		if($test_run_manager->get_results_identifier() != null && (pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveInstallationLogs', 'FALSE')))
+		if($test_run_manager->do_save_results() && (pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveInstallationLogs', 'FALSE')))
 		{
 			if(is_file($test_run_request->test_profile->get_install_dir() . 'install.log'))
 			{
@@ -777,43 +805,36 @@ class pts_test_execution
 					pts_module_manager::module_process('__post_test_run_success', $test_result);
 					$test_successful = true;
 
-					if($test_run_manager->get_results_identifier() != null)
+					if($generated_result_count >= 1)
 					{
-						if($generated_result_count >= 1)
+						// Prior to PTS 8.6, secondary result graphs wouldn't have their test profile identifier set but would be null
+						// With PTS 8.6+, the identifier is now preserved... Except with below logic for preserving compatibility with older result files, only clear the identifier if comparing against an old result file having a match for no identifier set
+						$ti_backup = $test_result->test_profile->get_identifier();
+						$test_result->test_profile->set_identifier('');
+						if(!$test_run_manager->result_file->result_hash_exists($test_result))
 						{
-							// No reason to have more than one identifier
-							// TODO XXX may want to rethink this behavior, we'll see...
-							$test_result->test_profile->set_identifier('');
+							$test_result->test_profile->set_identifier($ti_backup);
 						}
-
-						$test_result->test_result_buffer = new pts_test_result_buffer();
-						$test_result->test_result_buffer->add_test_result($test_run_manager->get_results_identifier(), $test_result->active->get_result(), $test_result->active->get_values_as_string(), pts_test_run_manager::process_json_report_attributes($test_result), $test_result->active->get_min_result(), $test_result->active->get_max_result());
-						$added_comparison_hash = $test_run_manager->result_file->add_result($test_result);
-						$generated_result_count++;
-
-						// The merged data, get back the merged test_result object
-						$results_comparison = clone $test_run_manager->result_file->get_result($added_comparison_hash);
-						if($results_comparison && $results_comparison->test_result_buffer->get_count() > 1)
-						{
-							pts_client::$display->test_run_success_inline($results_comparison);
-						}
-						pts_module_manager::module_process('__test_run_success_inline_result', $results_comparison);
 					}
-					else
+
+					$test_result->test_result_buffer = new pts_test_result_buffer();
+					$rid = $test_run_manager->get_results_identifier() != null ? $test_run_manager->get_results_identifier() : 'Result';
+					$test_result->test_result_buffer->add_test_result($rid, $test_result->active->get_result(), $test_result->active->get_values_as_string(), pts_test_run_manager::process_json_report_attributes($test_result), $test_result->active->get_min_result(), $test_result->active->get_max_result());
+					$added_comparison_hash = $test_run_manager->result_file->add_result($test_result);
+					$generated_result_count++;
+
+					// The merged data, get back the merged test_result object
+					$results_comparison = clone $test_run_manager->result_file->get_result($added_comparison_hash);
+					if($results_comparison && $results_comparison->test_result_buffer->get_count() > 1)
 					{
-						// Not a saved result
-						$test_result->test_result_buffer = new pts_test_result_buffer();
-						$test_result->test_result_buffer->add_test_result('Result', $test_result->active->get_result(), $test_result->active->get_values_as_string(), pts_test_run_manager::process_json_report_attributes($test_result), $test_result->active->get_min_result(), $test_result->active->get_max_result());
-						$temp_result_file = new pts_result_file(null);
-						$added_comparison_hash = $temp_result_file->add_result($test_result);
-						$results_comparison = clone $temp_result_file->get_result($added_comparison_hash);
-						pts_module_manager::module_process('__test_run_success_inline_result', $results_comparison);
+						pts_client::$display->test_run_success_inline($results_comparison);
 					}
+					pts_module_manager::module_process('__test_run_success_inline_result', $results_comparison);
 				}
 			}
 		}
 
-		if($test_run_manager->get_results_identifier() != null && $test_run_manager->get_file_name() != null && pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveTestLogs', 'FALSE'))
+		if($test_run_manager->do_save_results() && $test_run_manager->get_file_name() != null && pts_config::read_bool_config('PhoronixTestSuite/Options/Testing/SaveTestLogs', 'FALSE'))
 		{
 			static $xml_write_pos = 1;
 			pts_file_io::mkdir(PTS_SAVE_RESULTS_PATH . $test_run_manager->get_file_name() . '/test-logs/' . $xml_write_pos . '/');

pts-core/objects/client/pts_test_install_manager.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2016, Phoronix Media
-	Copyright (C) 2010 - 2016, Michael Larabel
+	Copyright (C) 2010 - 2018, Phoronix Media
+	Copyright (C) 2010 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -52,14 +52,14 @@ class pts_test_install_manager
 
 		return $added;
 	}
-	public function generate_download_file_lists()
+	public function generate_download_file_lists($do_file_checks = true)
 	{
 		pts_client::$display->test_install_progress_start('Determining File Requirements');
 		$test_count = count($this->tests_to_install);
 
 		foreach($this->tests_to_install as $i => &$test_install_request)
 		{
-			$test_install_request->generate_download_object_list();
+			$test_install_request->generate_download_object_list($do_file_checks);
 			pts_client::$display->test_install_progress_update(($i / $test_count));
 		}
 		pts_client::$display->test_install_progress_completed();

pts-core/objects/client/pts_test_install_request.php

@@ -71,7 +71,7 @@ class pts_test_install_request
 			$platforms = $download->get_platform_array();
 			if(!empty($platforms) && $do_file_checks)
 			{
-				if(!in_array(phodevi::operating_system(), $platforms) && !(phodevi::is_bsd() && in_array('Linux', $platforms) && (pts_client::executable_in_path('kldstat') && strpos(shell_exec('kldstat -n linux 2>&1'), 'linux.ko') != false)))
+				if(!in_array(phodevi::os_under_test(), $platforms) && !(phodevi::is_bsd() && in_array('Linux', $platforms) && (pts_client::executable_in_path('kldstat') && strpos(shell_exec('kldstat -n linux 2>&1'), 'linux.ko') != false)))
 				{
 					// This download does not match the operating system
 					continue;
@@ -91,29 +91,40 @@ class pts_test_install_request
 			$this->test_files[] = $download;
 		}
 	}
-	public static function test_files_available_locally(&$test_profile, $include_extended_test_profiles = true)
+	public static function test_files_available_via_cache(&$test_profile, $only_check_local_system = false)
 	{
+		static $remote_files, $local_download_caches, $remote_download_caches, $phoromatic_server_caches, $cached = false;
+
 		$install_request = new pts_test_install_request($test_profile);
 
-		$remote_files = pts_test_install_manager::remote_files_available_in_download_caches();
-		$local_download_caches = pts_test_install_manager::local_download_caches();
-		$remote_download_caches = pts_test_install_manager::remote_download_caches();
-		$phoromatic_server_caches = pts_test_install_manager::phoromatic_download_server_caches();
+		if($only_check_local_system)
+		{
+			$remote_files = false;
+			$local_download_caches = pts_test_install_manager::local_download_caches();
+			$remote_download_caches = false;
+			$phoromatic_server_caches = false;
+			$cached = false;
+		}
+		else if($cached == false)
+		{
+			$remote_files = pts_test_install_manager::remote_files_available_in_download_caches();
+			$local_download_caches = pts_test_install_manager::local_download_caches();
+			$remote_download_caches = pts_test_install_manager::remote_download_caches();
+			$phoromatic_server_caches = pts_test_install_manager::phoromatic_download_server_caches();
+			$cached = true;
+		}
 
 		$install_request->generate_download_object_list();
-		$install_request->scan_download_caches($local_download_caches, $remote_download_caches, $remote_files, $phoromatic_server_caches);
+		$all_files_accessible = $install_request->scan_download_caches($local_download_caches, $remote_download_caches, $remote_files, $phoromatic_server_caches, true, true);
 
-		foreach($install_request->get_download_objects() as $download_object)
+		if($all_files_accessible == false)
 		{
-			if($download_object->get_download_location_type() == null)
-			{
-				return false;
-			}
+			return false;
 		}
 
 		foreach($install_request->test_profile->extended_test_profiles() as $extended_test_profile)
 		{
-			if(self::test_files_available_locally($extended_test_profile) == false)
+			if(self::test_files_available_via_cache($extended_test_profile, $only_check_local_system) == false)
 			{
 				return false;
 			}
@@ -121,7 +132,7 @@ class pts_test_install_request
 
 		return true;
 	}
-	public static function test_files_in_cache(&$test_profile, $include_extended_test_profiles = true, $skip_hash_checks = false)
+	public static function test_files_available_on_local_system(&$test_profile)
 	{
 		// TODO XXX: rework this check into something more versatile for tests that could have files in cache
 		if(!is_file(PTS_TEST_PROFILE_PATH . $test_profile . '/test-definition.xml'))
@@ -129,33 +140,7 @@ class pts_test_install_request
 			return false;
 		}
 
-		$install_request = new pts_test_install_request($test_profile);
-
-		$remote_files = false;
-		$local_download_caches = pts_test_install_manager::local_download_caches();
-		$remote_download_caches = false;
-		$phoromatic_server_caches = false;
-
-		$install_request->generate_download_object_list();
-		$install_request->scan_download_caches($local_download_caches, $remote_download_caches, $remote_files, $phoromatic_server_caches, $skip_hash_checks);
-
-		foreach($install_request->get_download_objects() as $download_object)
-		{
-			if($download_object->get_download_location_type() == null)
-			{
-				return false;
-			}
-		}
-
-		foreach($install_request->test_profile->extended_test_profiles() as $extended_test_profile)
-		{
-			if(self::test_files_available_locally($extended_test_profile) == false)
-			{
-				return false;
-			}
-		}
-
-		return true;
+		return self::test_files_available_via_cache($test_profile, true);
 	}
 	public static function test_files_in_install_dir(&$test_profile)
 	{
@@ -173,7 +158,7 @@ class pts_test_install_request
 
 		return true;
 	}
-	public function scan_download_caches($local_download_caches, $remote_download_caches, $remote_files, $phoromatic_server_caches, $skip_hash_checks = false)
+	public function scan_download_caches(&$local_download_caches, &$remote_download_caches, &$remote_files, &$phoromatic_server_caches, $skip_extra_checks = false, $only_checking_for_cached_tests = false)
 	{
 		$download_location = $this->test_profile->get_install_dir();
 		$main_download_cache = pts_client::download_cache_path();
@@ -187,7 +172,7 @@ class pts_test_install_request
 				// File is already there in the test/destination directory, must have been previously downloaded
 				// Could add an MD5 check here to ensure validity, but if it made it here it was already valid unless user modified it
 
-				if($download_package->get_filesize() == 0)
+				if(!$skip_extra_checks && $download_package->get_filesize() == 0)
 				{
 					$download_package->set_filesize(filesize($download_location . $package_filename));
 				}
@@ -197,7 +182,7 @@ class pts_test_install_request
 			else if(is_file($main_download_cache . $package_filename))
 			{
 				// In main download cache
-				if($download_package->get_filesize() == 0)
+				if(!$skip_extra_checks && $download_package->get_filesize() == 0)
 				{
 					$download_package->set_filesize(filesize($main_download_cache . $package_filename));
 				}
@@ -207,7 +192,7 @@ class pts_test_install_request
 			else if(is_file(PTS_SHARE_PATH . 'download-cache/' . $package_filename))
 			{
 				// In system's /usr/share download cache
-				if($download_package->get_filesize() == 0)
+				if(!$skip_extra_checks && $download_package->get_filesize() == 0)
 				{
 					$download_package->set_filesize(filesize(PTS_SHARE_PATH . 'download-cache/' . $package_filename));
 				}
@@ -219,9 +204,9 @@ class pts_test_install_request
 				// Scan the local download caches
 				foreach($local_download_caches as &$cache_directory)
 				{
-					if(is_file($cache_directory . $package_filename) && ($skip_hash_checks || $download_package->check_file_hash($cache_directory . $package_filename)))
+					if(is_file($cache_directory . $package_filename) && ($skip_extra_checks || $download_package->check_file_hash($cache_directory . $package_filename)))
 					{
-						if($download_package->get_filesize() == 0)
+						if(!$skip_extra_checks && $download_package->get_filesize() == 0)
 						{
 							$download_package->set_filesize(filesize($cache_directory . $package_filename));
 						}
@@ -236,7 +221,7 @@ class pts_test_install_request
 				$lookaside_copy = pts_test_install_manager::file_lookaside_test_installations($download_package);
 				if($lookaside_copy)
 				{
-					if($download_package->get_filesize() == 0)
+					if(!$skip_extra_checks && $download_package->get_filesize() == 0)
 					{
 						$download_package->set_filesize(filesize($lookaside_copy));
 					}
@@ -249,7 +234,7 @@ class pts_test_install_request
 				{
 					foreach($phoromatic_server_caches as $server_url => $repo)
 					{
-						if(isset($repo[$package_filename]) && ($skip_hash_checks || $repo[$package_filename]['md5'] == $download_package->get_md5() || $repo[$package_filename]['sha256'] == $download_package->get_sha256() || ($download_package->get_sha256() == null && $download_package->get_md5() == null)))
+						if(isset($repo[$package_filename]) && ($skip_extra_checks || $repo[$package_filename]['md5'] == $download_package->get_md5() || $repo[$package_filename]['sha256'] == $download_package->get_sha256() || ($download_package->get_sha256() == null && $download_package->get_md5() == null)))
 						{
 							$download_package->set_download_location('REMOTE_DOWNLOAD_CACHE', array($server_url . '/download-cache.php?download=' . $package_filename));
 							break;
@@ -282,6 +267,16 @@ class pts_test_install_request
 					}
 				}
 			}
+
+			if($only_checking_for_cached_tests && $download_package->get_download_location_type() == null)
+			{
+				return false;
+			}
+		}
+
+		if($only_checking_for_cached_tests)
+		{
+			return true;
 		}
 	}
 	public function get_arguments_description()

pts-core/objects/client/pts_test_installer.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2017, Phoronix Media
-	Copyright (C) 2010 - 2017, Michael Larabel
+	Copyright (C) 2010 - 2019, Phoronix Media
+	Copyright (C) 2010 - 2019, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -44,6 +44,19 @@ class pts_test_installer
 		$unknown_tests = array();
 		$test_profiles = pts_types::identifiers_to_test_profile_objects($items_to_install, true, true, $unknown_tests);
 
+		if($force_install == false)
+		{
+			foreach($test_profiles as $i => $tp)
+			{
+				$valid = pts_test_run_manager::test_profile_system_compatibility_check($tp, true);
+				if($valid == false)
+				{
+					unset($test_profiles[$i]);
+				}
+			}
+
+		}
+
 		// Any external dependencies?
 		pts_external_dependencies::install_dependencies($test_profiles, $no_prompts, $skip_tests_with_missing_dependencies);
 
@@ -53,6 +66,12 @@ class pts_test_installer
 			trigger_error('The test installation directory is not writable.' . PHP_EOL . 'Location: ' . pts_client::test_install_root_path(), E_USER_ERROR);
 			return false;
 		}
+		$mount_options = phodevi::read_property('disk', 'mount-options');
+		if(isset($mount_options['mount-options']) && strpos($mount_options['mount-options'], 'noexec') !== false)
+		{
+			trigger_error('The test installation directory is on a file-system mounted with the \'noexec\' mount option. Re-mount the file-system appropriately or change the Phoronix Test Suite user configuration file to point to an alternative mount point.' . PHP_EOL . 'Location: ' . pts_client::test_install_root_path(), E_USER_ERROR);
+			return false;
+		}
 
 		pts_test_installer::start_install($test_profiles, $unknown_tests, $force_install, $no_prompts);
 		pts_client::release_lock($lock_path);
@@ -61,12 +80,6 @@ class pts_test_installer
 	}
 	public static function start_install(&$test_profiles, &$unknown_tests = null, $force_install = false, $no_prompts = false)
 	{
-		if(count($test_profiles) == 0)
-		{
-			trigger_error('No Tests Found For Installation.', E_USER_ERROR);
-			return false;
-		}
-
 		// Setup the install manager and add the tests
 		$test_install_manager = new pts_test_install_manager();
 		$install_table = array();
@@ -99,7 +112,10 @@ class pts_test_installer
 		{
 			foreach($unknown_tests as $unknown)
 			{
-				$install_table[] = array('Unknown:', $unknown);
+				if(!empty($unknown))
+				{
+					$install_table[] = array('Unknown:', $unknown);
+				}
 			}
 		}
 
@@ -172,7 +188,7 @@ class pts_test_installer
 			}
 		}
 	}
-	public static function only_download_test_files(&$test_profiles, $to_dir = null)
+	public static function only_download_test_files(&$test_profiles, $to_dir = null, $do_file_checks = true)
 	{
 		// Setup the install manager and add the tests
 		$test_install_manager = new pts_test_install_manager();
@@ -191,14 +207,14 @@ class pts_test_installer
 		}
 
 		// Let the pts_test_install_manager make some estimations, etc...
-		$test_install_manager->generate_download_file_lists();
+		$test_install_manager->generate_download_file_lists($do_file_checks);
 		$test_install_manager->check_download_caches_for_files();
 
 		// Begin the download process
 		while(($test_install_request = $test_install_manager->next_in_install_queue()) != false)
 		{
 			//pts_client::$display->test_install_start($test_install_request->test_profile->get_identifier());
-			pts_test_installer::download_test_files($test_install_request, $to_dir);
+			pts_test_installer::download_test_files($test_install_request, $to_dir, true);
 		}
 	}
 	protected static function download_test_files(&$test_install_request, $download_location = false, $no_prompts = false)
@@ -220,6 +236,8 @@ class pts_test_installer
 		pts_file_io::mkdir($download_location);
 		$module_pass = array($identifier, $test_install_request->get_download_objects());
 		pts_module_manager::module_process('__pre_test_download', $module_pass);
+		$objects_completed = 0;
+		$fail_if_no_downloads = false;
 
 		foreach($test_install_request->get_download_objects() as $download_package)
 		{
@@ -246,7 +264,8 @@ class pts_test_installer
 			{
 				case 'IN_DESTINATION_DIR':
 					pts_client::$display->test_install_download_file('FILE_FOUND', $download_package);
-					continue;
+					$objects_completed++;
+					continue 2;
 				case 'REMOTE_DOWNLOAD_CACHE':
 					$download_tries = 0;
 					do
@@ -279,7 +298,8 @@ class pts_test_installer
 
 					if(is_file($download_destination))
 					{
-						continue;
+						$objects_completed++;
+						continue 2;
 					}
 				case 'MAIN_DOWNLOAD_CACHE':
 				case 'LOCAL_DOWNLOAD_CACHE':
@@ -294,6 +314,7 @@ class pts_test_installer
 							// SymLinkFilesFromCache is disabled by default
 							pts_client::$display->test_install_download_file('LINK_FROM_CACHE', $download_package);
 							symlink($download_cache_file, $download_destination);
+							$objects_completed++;
 						}
 						else
 						{
@@ -329,12 +350,17 @@ class pts_test_installer
 
 						if(is_file($download_destination))
 						{
-							continue;
+							$objects_completed++;
+							continue 2;
 						}
 					}
 				default:
 					$package_urls = $download_package->get_download_url_array();
-
+					if(!is_file($download_destination) && empty($package_urls))
+					{
+						self::test_install_error(null, $test_install_request, $package_filename . ' must be manually placed in the Phoronix Test Suite download-cache.');
+						$fail_if_no_downloads = true;
+					}
 					// Download the file
 					if(!is_file($download_destination) && count($package_urls) > 0 && $package_urls[0] != null)
 					{
@@ -382,6 +408,7 @@ class pts_test_installer
 								if(is_file($download_destination_temp))
 								{
 									rename($download_destination_temp, $download_destination);
+									$objects_completed++;
 								}
 
 								if($download_package->get_filesize() > 0 && $download_end != $download_start)
@@ -459,7 +486,7 @@ class pts_test_installer
 
 		pts_module_manager::module_process('__post_test_download', $identifier);
 
-		return true;
+		return !$fail_if_no_downloads || $objects_completed > 0;
 	}
 	public static function create_compiler_mask(&$test_install_request)
 	{
@@ -561,6 +588,12 @@ class pts_test_installer
 				}
 				*/
 
+				// Since GCC POWER doesn't support -march=, in the compiler mask we can change it to -mcpu= before passed to the actual compiler
+				if(strpos(phodevi::read_property('system', 'kernel-architecture'), 'ppc') !== false && pts_client::executable_in_path('sed'))
+				{
+					$env_var_check .= 'COMPILER_OPTIONS=`echo "$COMPILER_OPTIONS" | sed -e "s/\-march=/-mcpu=/g"`' . PHP_EOL;
+				}
+
 				if(is_executable('/bin/bash'))
 				{
 					$shebang = '/bin/bash';
@@ -580,7 +613,12 @@ class pts_test_installer
 
 				// Write the main mask for the compiler
 				file_put_contents($main_compiler,
-					'#!' . $shebang . PHP_EOL . 'COMPILER_OPTIONS="$@"' . PHP_EOL . $env_var_check . PHP_EOL . 'echo $COMPILER_OPTIONS >> ' . $mask_dir . $compiler_type . '-options-' . $compiler_name . PHP_EOL . $compiler_path . ' "$@"' . PHP_EOL);
+					'#!' . $shebang . PHP_EOL .
+					'COMPILER_OPTIONS="$@"' . PHP_EOL .
+					$env_var_check . PHP_EOL .
+					'echo $COMPILER_OPTIONS >> ' . $mask_dir . $compiler_type . '-options-' . $compiler_name . PHP_EOL .
+					$compiler_path . ' "$@"' . PHP_EOL .
+					PHP_EOL);
 
 				// Make executable
 				chmod($main_compiler, 0755);
@@ -704,7 +742,6 @@ class pts_test_installer
 	protected static function install_test_process(&$test_install_request, $no_prompts)
 	{
 		// Install a test
-		$identifier = $test_install_request->test_profile->get_identifier();
 		$test_install_directory = $test_install_request->test_profile->get_install_dir();
 		pts_file_io::mkdir(dirname($test_install_directory));
 		pts_file_io::mkdir($test_install_directory);
@@ -733,8 +770,8 @@ class pts_test_installer
 
 			if($test_install_request->test_profile->get_file_installer() != false)
 			{
+				pts_module_manager::module_process('__pre_test_install', $test_install_request);
 				self::create_compiler_mask($test_install_request);
-				pts_module_manager::module_process('__pre_test_install', $identifier);
 				pts_client::$display->test_install_begin($test_install_request);
 
 				$pre_install_message = $test_install_request->test_profile->get_pre_install_message();
@@ -833,7 +870,7 @@ class pts_test_installer
 					}
 				}
 
-				pts_module_manager::module_process('__post_test_install', $identifier);
+				pts_module_manager::module_process('__post_test_install', $test_install_request);
 				$installed = true;
 
 				if(pts_config::read_bool_config('PhoronixTestSuite/Options/Installation/RemoveDownloadFiles', 'FALSE'))
@@ -868,7 +905,7 @@ class pts_test_installer
 		$identifier = $test_install_request->test_profile->get_identifier();
 		pts_file_io::mkdir($test_install_request->test_profile->get_install_dir());
 
-		if($remove_old_files)
+		if($remove_old_files && $test_install_request->test_profile->do_remove_test_install_directory_on_reinstall())
 		{
 			// Remove any (old) files that were installed
 			$ignore_files = array('pts-install.xml', 'install-failed.log');